NoFussKit field guide
Unicode characters and UTF-8 bytes: practical examples
Modern text supports far more than English letters. That flexibility means visual characters, Unicode code points and UTF-8 bytes can produce different counts.
ASCII is the simple case
Basic English letters, digits and common punctuation generally occupy one byte each in UTF-8. The word Test therefore has four visible characters and four bytes.
This simple relationship does not hold for the full range of human writing. Systems designed around ASCII assumptions can underestimate multilingual storage.
Accents and non-Latin scripts
An accented character may occupy two or more UTF-8 bytes even when it appears as one letter. Arabic and Devanagari characters also commonly require multiple bytes.
The same-looking accented letter can sometimes be represented as one precomposed code point or as a base letter plus a combining mark. Normalization matters in search, comparison and strict database limits.
Emoji can be sequences
Many emoji use several code points joined together. Skin-tone modifiers, flags and family emoji can look like one glyph while requiring a longer encoded sequence.
A visual counter, JavaScript length and UTF-8 byte counter can therefore disagree without any being inherently wrong. They are measuring different layers of the text.
Choose the right validation
For a social post, the platform's own character rule is authoritative. For a database column or request limit, use the encoding specified by its documentation.
Test representative multilingual and emoji input before launch. A field that works for English sample data can fail once real users enter names and messages in other scripts.
- Do not assume one character equals one byte
- Test combining accents
- Include emoji in validation samples
- Follow the destination's documented metric