About

Overview

• Unicode X-Ray is a tool for exploring Unicode text.
• Each grapheme cluster is displayed in a row.
• Each grapheme cluster is broken down into its characters in columns.
• The normalization options let you see how the same grapheme cluster can be represented with different characters.

Grapheme Clusters

Grapheme clusters are what we typically think of as individual characters. For example, we can break the string café into the following grapheme clusters:

The notion of grapheme clusters is important for the implementation of text editors and viewers: Text selection, cursor movement, and deletion all operate on grapheme clusters.

For café, grapheme cluster rules would let a text editor know the following:

• The number of characters in café is four.
• With a cursor starting at the beginning of the string café, pressing the right arrow key four times will move the cursor to the end of the string.
• Pressing the backspace key at the end of the string will delete the é grapheme cluster, leaving caf.

Grapheme clusters are described further in 3.6.2 Grapheme Clusters and UAX #29.

Characters

Grapheme clusters can be further broken down into characters, the smallest components of written language.

In Unicode, characters can be different than the grapheme clusters described above. This abstraction is useful because it allows characters to be combined: instead of defining a separate character for every grapheme cluster (such as an accented letter), we can define constituent characters and combine them.

For example, the grapheme cluster é can be broken into two characters, providing the following characters in café:

Text processing systems know how to spatially combine the e and the ◌́, so that the user sees a single é.

The circle in ◌́ is not part of the text. U+25CC DOTTED CIRCLE stands in as a base, because a combining character has nothing of its own to attach to. Unicode X-Ray adds this base to every combining character it displays.

(The é is a somewhat poor choice here for illustration because it can also be represented as a single character under Unicode. However, there are many instances where the only way to represent a grapheme cluster is with multiple characters, but they are not familiar to English readers.)

Also worth mentioning: Characters are uniquely identified by code points, numbers in the range 0 to 10FFFF₁₆. Code points are written in hexadecimal notation, prefixed with U+.

The code points for café are as follows:

When text is stored in a computer, a Unicode encoding turns each code point into a sequence of bytes.

Normalization

Above, it was noted that the é grapheme could be equivalently represented as either a single character or as two separate ones. But, with two encodings, processors would be unable to tell they are the same. This creates the need for normalization: a regular way of picking the encoding.

There are two axes of normalization, both of which are applied simultaneously:

Composition (C) vs. Decomposition (D)

In general, composition seeks to combine characters into a single code point, while decomposition seeks to break characters into multiple code points.

For example, see the codepoints of é under NFC and é under NFD.

Canonical (no initial) vs. Compatibility (K)

In general, canonical normalization seeks to preserve the meaning of the characters, while compatibility normalization reduces the characters to a more basic form, which may lose some of the original meaning.

For example, see the codepoints of ℝ under NFD and ℝ under NFKD.

A few characters change under every form. For one example, open ẛ and step through the normalization menu.

Read more about normalization in Section 3.11 Normalization Forms.

Control for normalization is offered on Unicode X-Ray because users may be interested in the different representations that text can take.

Naming

Unicode X-Ray gives each assigned code point a useful name when the Unicode Character Database provides enough information. It combines several official files and gives some sources priority over others.

1. Start with the published name. DerivedName.txt supplies the name for almost every assigned code point. For most code points, this name is the final result.

2. Label code points without individual names. Surrogates and private-use code points do not have individual Unicode names. UnicodeData.txt identifies these ranges, and Blocks.txt supplies each block name.

   Unicode X-Ray uses the block name as a useful label.

3. Add control names. UnicodeData.txt records the Unicode 1.0 name for many control code points. The generator uses this name when no selected alias exists.

4. Apply selected aliases. Unicode keeps each published character name stable, even if the name contains an error. NameAliases.txt publishes corrections and other aliases separately.

   Unicode X-Ray prefers corrections, then alternate names, control names, and finally figments. It does not use abbreviations as character names.

   The selected alias replaces the published name or the Unicode 1.0 control name.

5. Add a definition when one exists. Some names identify an ideograph by code point or a hieroglyph by catalog number. These names identify the character, but they do not explain it.

   kDefinition in Unihan_Readings.txt from Unihan.zip supplies glosses for Chinese, Japanese, and Korean ideographs. kEH_Desc in Unikemet.txt describes Egyptian hieroglyphs.

   Unicode X-Ray shows a definition below the name.

Versions

Your browser may use a different version of Unicode than Unicode X-Ray, and therefore, Unicode X-Ray may not be able to provide a name for every character you provide.

In the case that you have entered a code point not yet in the Unicode version used by Unicode X-Ray, its name will be <unknown, Block Name>.