The encyclopedia · Software & IT · Technical decision · 1992-2003
UTF-8 won by making every byte below 128 identical to ASCII
In 1992 Ken Thompson designed UTF-8 so bytes 0-127 stayed pure ASCII; existing files and parsers kept working, and the encoding conquered the Web.
Bell Labs (Plan 9) · X/Open Consortium
the move
By the early 1990s the world's writing systems were being unified in ISO/IEC 10646 and Unicode, but the proposed fixed 16-bit encodings were hostile to existing systems: every ASCII text file would grow and software that treated bytes 0-127 as ASCII would break. Plan 9 had already used a 16-bit UTF and hated it, as Rob Pike later recalled.
In September 1992, Ken Thompson and Rob Pike designed a variable-length encoding on a placemat in a New Jersey diner after an X/Open committee asked them to vet a competing design. Their scheme kept bytes 0-127 exactly equal to ASCII while representing every other code point in multi-byte sequences; within days Plan 9 was running entirely on the new encoding.
X/Open adopted their proposal, and in 2003 the IETF made it an Internet Standard (STD 63, RFC 3629), which notes that UTF-8 preserves the full US-ASCII range, providing compatibility with file systems, parsers and other software that rely on US-ASCII values.
why it works
- Every existing ASCII file and C-string parser kept working unchanged, so there was nothing to migrate.
- Byte-oriented tools such as grep, mail and file systems could carry UTF-8 without modification.
- X/Open gave the design an institutional push before rival proposals matured.
- The IETF later standardized it, locking in the choice for internet protocols.
what transfers
Backward compatibility is the cheapest adoption strategy: a standard that preserves what works deploys without a migration campaign; a cleaner one that breaks it faces years of resistance.
what came after
X/Open adopted UTF-8 in the early 1990s, and Plan 9's conversion proved it could be implemented in days, not years. The IETF made it an Internet Standard in 2003, and it became the default encoding of the Web and of Unix-like systems. Its dominance is why ASCII compatibility is still assumed by most software written today.
references
spotted an error? The archive wants to know.