Character Encoding and Charsets in Java

Computers store and send bytes, not letters. Every time text goes into a file, over a network or into a database, it has to be turned into bytes, and when it comes back, the bytes have to be turned into text again. The rule that maps characters to bytes is called a character encoding (or charset). Use the same rule at both ends and everything works; mix them up and "naïve" turns into "naïve". This lesson shows why that happens and how to prevent it.

Characters, Codes and Bytes

Three layers are involved:

1. Characters: what people read, such as ï or न.
2. Unicode code points: a number for every character in every language. ï is U+00EF and the grinning-face emoji is U+1F600.
3. Bytes: how those numbers are stored. UTF-8, the encoding of the modern web, uses 1 byte for English letters and digits, 2 bytes for letters like ï, 3 bytes for most Indian scripts and 4 bytes for emoji.

import java.nio.charset.StandardCharsets;
import java.util.Arrays;
 
public class Bytes {
    public static void main(String[] args) {
        String word = "naïve";
        byte[] utf8 = word.getBytes(StandardCharsets.UTF_8);
        System.out.println(word.length() + " characters, "
            + utf8.length + " bytes");
        System.out.println(Arrays.toString(utf8));
 
        System.out.println(new String(utf8, StandardCharsets.UTF_8));
        System.out.println(new String(utf8, StandardCharsets.ISO_8859_1));
 
        String hindi = "नमस्ते";
        System.out.println(hindi.length() + " chars, "
            + hindi.getBytes(StandardCharsets.UTF_8).length + " bytes");
    }
}

5 characters, 6 bytes
[110, 97, -61, -81, 118, 101]
naïve
naïve
6 chars, 18 bytes

How it works

1. getBytes(UTF_8) turns 5 characters into 6 bytes: ï needs two (-61, -81 are the bytes C3 AF, shown as signed numbers because Java's byte is signed).
2. Decoding with the same charset gives back naïve.
3. Decoding with the wrong charset (ISO-8859-1, which uses one byte per character) reads C3 and AF as two separate characters, Ã and ¯. That garbled text has a name: mojibake.
4. The Hindi greeting has 6 Java chars but needs 18 UTF-8 bytes, 3 per character.

Why You Should Always Name the Charset

Methods like getBytes() and new String(bytes) without a charset use the platform default. Since Java 18, that default is UTF-8 everywhere; before that, it depended on the operating system (often Windows-1252 on Windows), which caused bugs that only appeared on some machines. Writing StandardCharsets.UTF_8 makes your code behave the same on every computer and every Java version.

Java's char Is UTF-16

A Java char is 16 bits, enough for most characters but not all. Characters above U+FFFF, like most emoji, take two chars, called a surrogate pair. So length() counts chars, not visible characters:

public class Emoji {
    public static void main(String[] args) {
        String text = "Hi 😀";
        System.out.println(text.length());
        System.out.println(text.codePointCount(0, text.length()));
    }
}

5
4

The text shows 4 characters (H, i, a space and the emoji), but length() says 5 because the emoji uses two chars. codePointCount gives the real count. This matters for "maximum 10 characters" checks and for reversing strings (see "How to Reverse a String in Java").

Common Mistakes

Calling getBytes() or new String(bytes) with no charset. Always pass StandardCharsets.UTF_8.

Counting characters with length() in text that may contain emoji. Use codePointCount.

"Fixing" mojibake by replacing characters. Fix the charset at the point where bytes are decoded.

Interview Corner

Practice

1. Easy: Print the number of UTF-8 bytes for "Navika", "café" and "नमस्ते". Explain each number.
2. Harder: Encode "Srihitha ₹500" with UTF-8, decode the bytes with ISO-8859-1, then show how to recover the original text.

Key Takeaways