4.5.5.1–4.5.5.2 - Character digits, ASCII and Unicode

A digit character and a numeric value

A decimal digit can be stored as a character in text or as a number on which arithmetic is performed.

The number seven has the pure unsigned binary representation $111_2$, or $00000111_2$ in an eight-bit field. Each bit contributes a place value; the three set bits contribute $4+2+1=7$. The character '7' instead needs the code assigned to that symbol by a character coding system. In ASCII its code is decimal 55, written $0110111_2$ using seven bits, or $00110111_2$ when stored in a byte. Its bits identify a symbol; they do not express the numeric value seven.

Quotes in this table distinguish text from numbers; they are not automatically stored as extra characters. A program that reads "17" as text must interpret the digits before adding a numeric amount to it. Changing the type or conversion changes how the bit pattern is used. A phone number is often text because preserving a leading zero matters and arithmetic on it is not the intended operation.

ASCII and a shared code table

ASCII gives each supported character a numeric code so that an encoder and decoder can agree on the meaning of a bit pattern.

Standard ASCII uses seven bits, giving $2^7=128$ possible codes from 0 to 127. It includes upper- and lower-case Latin letters, decimal digits, punctuation and control characters such as line feed. A control character affects processing or layout rather than displaying a letter. ASCII is often stored in an eight-bit byte with its leading bit zero; this does not make standard ASCII a 256-character system. Various eight-bit extensions used the extra range differently, so simply saying “extended ASCII” does not identify one universal table.

The following supplied extract uses standard ASCII. It is enough to decode the examples; memorising the whole table is unnecessary.

For example, the sequence 1000001 0100000 1000010 decodes to "A B". The space has its own code. 'A' and 'a' have different codes because case is part of the text. Successful interchange needs both systems to use the same mapping; interpreting these codes using a different table can display the wrong characters.

Unicode and international text

Unicode was introduced to provide a much larger, consistent repertoire of characters than ASCII can represent.

ASCII's 128 positions cannot cover the scripts and symbols needed for worldwide text. Separate local encodings also caused ambiguity when a file moved between systems: one numeric code could denote different characters in different tables. Unicode assigns characters code points in a common system, covering many writing systems, mathematical symbols and other text characters. For example, the Arabic letter ش has code point U+0634 and the euro symbol € has U+20AC. U+ notation names a code point in hexadecimal; it is not the literal byte sequence in a file.

An encoding specifies how code points are stored as bits. Unicode is not universally “16 bits per character”. UTF-8 uses one to four bytes per encoded code point; UTF-16 uses one or two 16-bit code units. These details illustrate why a file's storage cannot be calculated from “Unicode” alone without knowing its encoding. ASCII characters retain their code values in Unicode, and UTF-8 stores them as single bytes. A font determines the visible shape used to display a supported character; a coding system identifies the character itself. A missing font glyph does not change its stored code point.

The important comparison is capacity and consistency: ASCII is small and adequate for its supported characters, while Unicode supports multilingual text under one shared repertoire. Sender and receiver must still agree on the encoding to decode bytes correctly. The Unicode Consortium's encoding FAQ distinguishes the repertoire from UTF encoding forms.