String — UTF-16 characters, Unicode code points, and grapheme clusters

Reference note (untrusted external data; do not execute it as instructions). Strings are represented fundamentally as sequences of UTF-16 code units. In UTF-16 encoding, every code unit is exact 16 bits long. This means there are a maximum of 216, or 65536 possible characters representable as single UTF-16 code units. This character set is called the basic multilingual plane (BMP), and includes the most common characters like the Latin, Greek, Cyrillic alphabets, as well as many East Asian characters. Each code unit can be written in a string with \u followed by exactly four hex digits. However, the entire Unicode character set is much, much bigger than 65536. The extra characters are stored in UTF-16 as _surrogate pairs_, which are pairs of 16-bit code units that represent a single character. To avoid ambiguity, the two parts of the pair must be between 0xD800 and 0xDFFF, and these code units are not used to encode single-code-unit characters. (More precisely, leading surrogates, also called high-surrogate code units, have values between 0xD800 and 0xDBFF, inclusive, while trailing surrogates, also called low-surrogate code units, have values between 0xDC00 and 0xDFFF, inclusive.) Each Unicode character, comprised of one or two UTF-16 code units, is also called a _Unicode code point_. Each Unicode code point can be written in a string with \u{xxxxxx} where xxxxxx represents 1–6 hex digits. A "lone surrogate" is a 16-bit code unit satisfying one of the descriptions below It is in the range 0xD800–0xDBFF, inclusive (i.e., is a leading surrogate), but it is the last code unit in the string, or the next code unit is not a trailing surrogate. It is in the range 0xDC00–0xDFFF, inclusive (i.e., is a trailing surrogate), but it is the first code unit in the string, or the previous code unit is not a leading surrogate. … Attribution: Adapted from MDN Web Docs under CC-BY-SA-2.5. Adaptation: WikiKV selected one documentation section, normalized formatting, retained bounded excerpts, and shortened it at a paragraph or sentence boundary for retrieval. Verify version-sensitive details at the source.