Category talk:Wren-upc: Difference between revisions
m
→User-perceived characters: Updated blurb to Unicode version 15.0.
(Have given up trying to post the source code to RC and have posted it to a GitHub gist instead.) |
m (→User-perceived characters: Updated blurb to Unicode version 15.0.) |
||
| (5 intermediate revisions by the same user not shown) | |||
Line 2:
In Unicode a ''user-perceived character'' (or ''grapheme cluster'') can comprise one or more codepoints and the process of splitting a string into such grapheme clusters is described in [http://unicode.org/reports/tr29/ Unicode Standard Annex #29].
Given the complexity of this process, Wren doesn't have built-in support for it and this module aims to remedy that situation. It is based on Oliver Kuederle's [https://github.com/rivo/uniseg Unicode Text Segmentation for Go library] version 0.1.0 which is subject to the [https://github.com/rivo/uniseg/blob/master/LICENSE.txt MIT License] and is
At the time of writing (November 2023) there has been a subsequent release for this library (version 0.3.0) which includes support for word breaking, sentence breaking and line breaking and is based on Unicode version 14.0 (subsequently updated to version 15.0 though not yet released). These additional features are unlikely to see much (if any) use on Rosetta Code and, given the amount of code needed to deal with them, I have decided not to support them in upc.wren. However, I have updated the property table so it now includes characters added up to and including Unicode version 15.0.
If anyone is interested in using the additional features of version 0.3.0, then it would probably be easier to wrap the library using a Go host rather than attempting to translate all of it to Wren.
Although the source code file is large by Wren library standards (nearly 2200 lines), approximately 1900 lines of this are needed to describe the property table which provides the raw material for text segmentation. In the interests of brevity, I have omitted the comments which accompanied the original table which should be referred to if any explanation is needed.
===Source code===
<syntaxhighlight lang="wren">/* Module "upc.wren" */
/*
Properties encapsulates the Unicode properties relevant to this module and, as well as, exposing
those properties themselves contains a single public method to return the relevant property
for a given Unicode codepoint.
*/
class Properties {
// Unicode properties relevant to this module.
static any { 0 }
static prepend { 1 }
static cr { 2 }
static lf { 3 }
static control { 4 }
static extend { 5 }
static regionalIndicator { 6 }
static spacingMark { 7 }
static l { 8 }
static v { 9 }
static t { 10 }
static lv { 11 }
static lvt { 12 }
static zwj { 13 }
static extendedPictographic { 14 }
// Private method to initialize codepoints table.
// The latter maps an inclusive range of codepoints to a Unicode property.
// The ranges are in order to enable a binary search of the table by codepoint.
static init_() {
__codePoints = [
[0x0000, 0x0009, control],
[0x000A, 0x000A, lf],
[0x000B, 0x000C, control],
[0x000D, 0x000D, cr],
[0x000E, 0x001F, control],
[0x007F, 0x009F, control],
[0x00A9, 0x00A9, extendedPictographic],
[0x00AD, 0x00AD, control],
[0x00AE, 0x00AE, extendedPictographic],
[0x0300, 0x036F, extend],
[0x0483, 0x0487, extend],
[0x0488, 0x0489, extend],
[0x0591, 0x05BD, extend],
[0x05BF, 0x05BF, extend],
[0x05C1, 0x05C2, extend],
[0x05C4, 0x05C5, extend],
[0x05C7, 0x05C7, extend],
[0x0600, 0x0605, prepend],
[0x0610, 0x061A, extend],
[0x061C, 0x061C, control],
[0x064B, 0x065F, extend],
[0x0670, 0x0670, extend],
[0x06D6, 0x06DC, extend],
[0x06DD, 0x06DD, prepend],
[0x06DF, 0x06E4, extend],
[0x06E7, 0x06E8, extend],
[0x06EA, 0x06ED, extend],
[0x070F, 0x070F, prepend],
[0x0711, 0x0711, extend],
[0x0730, 0x074A, extend],
[0x07A6, 0x07B0, extend],
[0x07EB, 0x07F3, extend],
[0x07FD, 0x07FD, extend],
[0x0816, 0x0819, extend],
[0x081B, 0x0823, extend],
[0x0825, 0x0827, extend],
[0x0829, 0x082D, extend],
[0x0859, 0x085B, extend],
[0x0890, 0x0891, prepend],
[0x0898, 0x089F, extend],
[0x08CA, 0x08E1, extend],
[0x08E2, 0x08E2, prepend],
[0x08E3, 0x0902, extend],
[0x0903, 0x0903, spacingMark],
[0x093A, 0x093A, extend],
[0x093B, 0x093B, spacingMark],
[0x093C, 0x093C, extend],
[0x093E, 0x0940, spacingMark],
[0x0941, 0x0948, extend],
[0x0949, 0x094C, spacingMark],
[0x094D, 0x094D, extend],
[0x094E, 0x094F, spacingMark],
[0x0951, 0x0957, extend],
[0x0962, 0x0963, extend],
[0x0981, 0x0981, extend],
[0x0982, 0x0983, spacingMark],
[0x09BC, 0x09BC, extend],
[0x09BE, 0x09BE, extend],
[0x09BF, 0x09C0, spacingMark],
[0x09C1, 0x09C4, extend],
[0x09C7, 0x09C8, spacingMark],
[0x09CB, 0x09CC, spacingMark],
[0x09CD, 0x09CD, extend],
[0x09D7, 0x09D7, extend],
[0x09E2, 0x09E3, extend],
[0x09FE, 0x09FE, extend],
[0x0A01, 0x0A02, extend],
[0x0A03, 0x0A03, spacingMark],
[0x0A3C, 0x0A3C, extend],
[0x0A3E, 0x0A40, spacingMark],
[0x0A41, 0x0A42, extend],
[0x0A47, 0x0A48, extend],
[0x0A4B, 0x0A4D, extend],
[0x0A51, 0x0A51, extend],
[0x0A70, 0x0A71, extend],
[0x0A75, 0x0A75, extend],
[0x0A81, 0x0A82, extend],
[0x0A83, 0x0A83, spacingMark],
[0x0ABC, 0x0ABC, extend],
[0x0ABE, 0x0AC0, spacingMark],
[0x0AC1, 0x0AC5, extend],
[0x0AC7, 0x0AC8, extend],
[0x0AC9, 0x0AC9, spacingMark],
[0x0ACB, 0x0ACC, spacingMark],
[0x0ACD, 0x0ACD, extend],
[0x0AE2, 0x0AE3, extend],
[0x0AFA, 0x0AFF, extend],
[0x0B01, 0x0B01, extend],
[0x0B02, 0x0B03, spacingMark],
[0x0B3C, 0x0B3C, extend],
[0x0B3E, 0x0B3E, extend],
[0x0B3F, 0x0B3F, extend],
[0x0B40, 0x0B40, spacingMark],
[0x0B41, 0x0B44, extend],
[0x0B47, 0x0B48, spacingMark],
[0x0B4B, 0x0B4C, spacingMark],
[0x0B4D, 0x0B4D, extend],
[0x0B55, 0x0B56, extend],
[0x0B57, 0x0B57, extend],
[0x0B62, 0x0B63, extend],
[0x0B82, 0x0B82, extend],
[0x0BBE, 0x0BBE, extend],
[0x0BBF, 0x0BBF, spacingMark],
[0x0BC0, 0x0BC0, extend],
[0x0BC1, 0x0BC2, spacingMark],
[0x0BC6, 0x0BC8, spacingMark],
[0x0BCA, 0x0BCC, spacingMark],
[0x0BCD, 0x0BCD, extend],
[0x0BD7, 0x0BD7, extend],
[0x0C00, 0x0C00, extend],
[0x0C01, 0x0C03, spacingMark],
[0x0C04, 0x0C04, extend],
[0x0C3C, 0x0C3C, extend],
[0x0C3E, 0x0C40, extend],
[0x0C41, 0x0C44, spacingMark],
[0x0C46, 0x0C48, extend],
[0x0C4A, 0x0C4D, extend],
[0x0C55, 0x0C56, extend],
[0x0C62, 0x0C63, extend],
[0x0C81, 0x0C81, extend],
[0x0C82, 0x0C83, spacingMark],
[0x0CBC, 0x0CBC, extend],
[0x0CBE, 0x0CBE, spacingMark],
[0x0CBF, 0x0CBF, extend],
[0x0CC0, 0x0CC1, spacingMark],
[0x0CC2, 0x0CC2, extend],
[0x0CC3, 0x0CC4, spacingMark],
[0x0CC6, 0x0CC6, extend],
[0x0CC7, 0x0CC8, spacingMark],
[0x0CCA, 0x0CCB, spacingMark],
[0x0CCC, 0x0CCD, extend],
[0x0CD5, 0x0CD6, extend],
[0x0CE2, 0x0CE3, extend],
[0x0CF3, 0x0CF3, spacingMark],
[0x0D00, 0x0D01, extend],
[0x0D02, 0x0D03, spacingMark],
[0x0D3B, 0x0D3C, extend],
[0x0D3E, 0x0D3E, extend],
[0x0D3F, 0x0D40, spacingMark],
[0x0D41, 0x0D44, extend],
[0x0D46, 0x0D48, spacingMark],
[0x0D4A, 0x0D4C, spacingMark],
[0x0D4D, 0x0D4D, extend],
[0x0D4E, 0x0D4E, prepend],
[0x0D57, 0x0D57, extend],
[0x0D62, 0x0D63, extend],
[0x0D81, 0x0D81, extend],
[0x0D82, 0x0D83, spacingMark],
[0x0DCA, 0x0DCA, extend],
[0x0DCF, 0x0DCF, extend],
[0x0DD0, 0x0DD1, spacingMark],
[0x0DD2, 0x0DD4, extend],
[0x0DD6, 0x0DD6, extend],
[0x0DD8, 0x0DDE, spacingMark],
[0x0DDF, 0x0DDF, extend],
[0x0DF2, 0x0DF3, spacingMark],
[0x0E31, 0x0E31, extend],
[0x0E33, 0x0E33, spacingMark],
[0x0E34, 0x0E3A, extend],
[0x0E47, 0x0E4E, extend],
[0x0EB1, 0x0EB1, extend],
[0x0EB3, 0x0EB3, spacingMark],
[0x0EB4, 0x0EBC, extend],
[0x0EC8, 0x0ECE, extend],
[0x0F18, 0x0F19, extend],
[0x0F35, 0x0F35, extend],
[0x0F37, 0x0F37, extend],
[0x0F39, 0x0F39, extend],
[0x0F3E, 0x0F3F, spacingMark],
[0x0F71, 0x0F7E, extend],
[0x0F7F, 0x0F7F, spacingMark],
[0x0F80, 0x0F84, extend],
[0x0F86, 0x0F87, extend],
[0x0F8D, 0x0F97, extend],
[0x0F99, 0x0FBC, extend],
[0x0FC6, 0x0FC6, extend],
[0x102D, 0x1030, extend],
[0x1031, 0x1031, spacingMark],
[0x1032, 0x1037, extend],
[0x1039, 0x103A, extend],
[0x103B, 0x103C, spacingMark],
[0x103D, 0x103E, extend],
[0x1056, 0x1057, spacingMark],
[0x1058, 0x1059, extend],
[0x105E, 0x1060, extend],
[0x1071, 0x1074, extend],
[0x1082, 0x1082, extend],
[0x1084, 0x1084, spacingMark],
[0x1085, 0x1086, extend],
[0x108D, 0x108D, extend],
[0x109D, 0x109D, extend],
[0x1100, 0x115F, l],
[0x1160, 0x11A7, v],
[0x11A8, 0x11FF, t],
[0x135D, 0x135F, extend],
[0x1712, 0x1714, extend],
[0x1715, 0x1715, spacingMark],
[0x1732, 0x1733, extend],
[0x1734, 0x1734, spacingMark],
[0x1752, 0x1753, extend],
[0x1772, 0x1773, extend],
[0x17B4, 0x17B5, extend],
[0x17B6, 0x17B6, spacingMark],
[0x17B7, 0x17BD, extend],
[0x17BE, 0x17C5, spacingMark],
[0x17C6, 0x17C6, extend],
[0x17C7, 0x17C8, spacingMark],
[0x17C9, 0x17D3, extend],
[0x17DD, 0x17DD, extend],
[0x180B, 0x180D, extend],
[0x180E, 0x180E, control],
[0x180F, 0x180F, extend],
[0x1885, 0x1886, extend],
[0x18A9, 0x18A9, extend],
[0x1920, 0x1922, extend],
[0x1923, 0x1926, spacingMark],
[0x1927, 0x1928, extend],
[0x1929, 0x192B, spacingMark],
[0x1930, 0x1931, spacingMark],
[0x1932, 0x1932, extend],
[0x1933, 0x1938, spacingMark],
[0x1939, 0x193B, extend],
[0x1A17, 0x1A18, extend],
[0x1A19, 0x1A1A, spacingMark],
[0x1A1B, 0x1A1B, extend],
[0x1A55, 0x1A55, spacingMark],
[0x1A56, 0x1A56, extend],
[0x1A57, 0x1A57, spacingMark],
[0x1A58, 0x1A5E, extend],
[0x1A60, 0x1A60, extend],
[0x1A62, 0x1A62, extend],
[0x1A65, 0x1A6C, extend],
[0x1A6D, 0x1A72, spacingMark],
[0x1A73, 0x1A7C, extend],
[0x1A7F, 0x1A7F, extend],
[0x1AB0, 0x1ABD, extend],
[0x1ABE, 0x1ABE, extend],
[0x1ABF, 0x1ACE, extend],
[0x1B00, 0x1B03, extend],
[0x1B04, 0x1B04, spacingMark],
[0x1B34, 0x1B34, extend],
[0x1B35, 0x1B35, extend],
[0x1B36, 0x1B3A, extend],
[0x1B3B, 0x1B3B, spacingMark],
[0x1B3C, 0x1B3C, extend],
[0x1B3D, 0x1B41, spacingMark],
[0x1B42, 0x1B42, extend],
[0x1B43, 0x1B44, spacingMark],
[0x1B6B, 0x1B73, extend],
[0x1B80, 0x1B81, extend],
[0x1B82, 0x1B82, spacingMark],
[0x1BA1, 0x1BA1, spacingMark],
[0x1BA2, 0x1BA5, extend],
[0x1BA6, 0x1BA7, spacingMark],
[0x1BA8, 0x1BA9, extend],
[0x1BAA, 0x1BAA, spacingMark],
[0x1BAB, 0x1BAD, extend],
[0x1BE6, 0x1BE6, extend],
[0x1BE7, 0x1BE7, spacingMark],
[0x1BE8, 0x1BE9, extend],
[0x1BEA, 0x1BEC, spacingMark],
[0x1BED, 0x1BED, extend],
[0x1BEE, 0x1BEE, spacingMark],
[0x1BEF, 0x1BF1, extend],
[0x1BF2, 0x1BF3, spacingMark],
[0x1C24, 0x1C2B, spacingMark],
[0x1C2C, 0x1C33, extend],
[0x1C34, 0x1C35, spacingMark],
[0x1C36, 0x1C37, extend],
[0x1CD0, 0x1CD2, extend],
[0x1CD4, 0x1CE0, extend],
[0x1CE1, 0x1CE1, spacingMark],
[0x1CE2, 0x1CE8, extend],
[0x1CED, 0x1CED, extend],
[0x1CF4, 0x1CF4, extend],
[0x1CF7, 0x1CF7, spacingMark],
[0x1CF8, 0x1CF9, extend],
[0x1DC0, 0x1DFF, extend],
[0x200B, 0x200B, control],
[0x200C, 0x200C, extend],
[0x200D, 0x200D, zwj],
[0x200E, 0x200F, control],
[0x2028, 0x2028, control],
[0x2029, 0x2029, control],
[0x202A, 0x202E, control],
[0x203C, 0x203C, extendedPictographic],
[0x2049, 0x2049, extendedPictographic],
[0x2060, 0x2064, control],
[0x2065, 0x2065, control],
[0x2066, 0x206F, control],
[0x20D0, 0x20DC, extend],
[0x20DD, 0x20E0, extend],
[0x20E1, 0x20E1, extend],
[0x20E2, 0x20E4, extend],
[0x20E5, 0x20F0, extend],
[0x2122, 0x2122, extendedPictographic],
[0x2139, 0x2139, extendedPictographic],
[0x2194, 0x2199, extendedPictographic],
[0x21A9, 0x21AA, extendedPictographic],
[0x231A, 0x231B, extendedPictographic],
[0x2328, 0x2328, extendedPictographic],
[0x2388, 0x2388, extendedPictographic],
[0x23CF, 0x23CF, extendedPictographic],
[0x23E9, 0x23EC, extendedPictographic],
[0x23ED, 0x23EE, extendedPictographic],
[0x23EF, 0x23EF, extendedPictographic],
[0x23F0, 0x23F0, extendedPictographic],
[0x23F1, 0x23F2, extendedPictographic],
[0x23F3, 0x23F3, extendedPictographic],
[0x23F8, 0x23FA, extendedPictographic],
[0x24C2, 0x24C2, extendedPictographic],
[0x25AA, 0x25AB, extendedPictographic],
[0x25B6, 0x25B6, extendedPictographic],
[0x25C0, 0x25C0, extendedPictographic],
[0x25FB, 0x25FE, extendedPictographic],
[0x2600, 0x2601, extendedPictographic],
[0x2602, 0x2603, extendedPictographic],
[0x2604, 0x2604, extendedPictographic],
[0x2605, 0x2605, extendedPictographic],
[0x2607, 0x260D, extendedPictographic],
[0x260E, 0x260E, extendedPictographic],
[0x260F, 0x2610, extendedPictographic],
[0x2611, 0x2611, extendedPictographic],
[0x2612, 0x2612, extendedPictographic],
[0x2614, 0x2615, extendedPictographic],
[0x2616, 0x2617, extendedPictographic],
[0x2618, 0x2618, extendedPictographic],
[0x2619, 0x261C, extendedPictographic],
[0x261D, 0x261D, extendedPictographic],
[0x261E, 0x261F, extendedPictographic],
[0x2620, 0x2620, extendedPictographic],
[0x2621, 0x2621, extendedPictographic],
[0x2622, 0x2623, extendedPictographic],
[0x2624, 0x2625, extendedPictographic],
[0x2626, 0x2626, extendedPictographic],
[0x2627, 0x2629, extendedPictographic],
[0x262A, 0x262A, extendedPictographic],
[0x262B, 0x262D, extendedPictographic],
[0x262E, 0x262E, extendedPictographic],
[0x262F, 0x262F, extendedPictographic],
[0x2630, 0x2637, extendedPictographic],
[0x2638, 0x2639, extendedPictographic],
[0x263A, 0x263A, extendedPictographic],
[0x263B, 0x263F, extendedPictographic],
[0x2640, 0x2640, extendedPictographic],
[0x2641, 0x2641, extendedPictographic],
[0x2642, 0x2642, extendedPictographic],
[0x2643, 0x2647, extendedPictographic],
[0x2648, 0x2653, extendedPictographic],
[0x2654, 0x265E, extendedPictographic],
[0x265F, 0x265F, extendedPictographic],
[0x2660, 0x2660, extendedPictographic],
[0x2661, 0x2662, extendedPictographic],
[0x2663, 0x2663, extendedPictographic],
[0x2664, 0x2664, extendedPictographic],
[0x2665, 0x2666, extendedPictographic],
[0x2667, 0x2667, extendedPictographic],
[0x2668, 0x2668, extendedPictographic],
[0x2669, 0x267A, extendedPictographic],
[0x267B, 0x267B, extendedPictographic],
[0x267C, 0x267D, extendedPictographic],
[0x267E, 0x267E, extendedPictographic],
[0x267F, 0x267F, extendedPictographic],
[0x2680, 0x2685, extendedPictographic],
[0x2690, 0x2691, extendedPictographic],
[0x2692, 0x2692, extendedPictographic],
[0x2693, 0x2693, extendedPictographic],
[0x2694, 0x2694, extendedPictographic],
[0x2695, 0x2695, extendedPictographic],
[0x2696, 0x2697, extendedPictographic],
[0x2698, 0x2698, extendedPictographic],
[0x2699, 0x2699, extendedPictographic],
[0x269A, 0x269A, extendedPictographic],
[0x269B, 0x269C, extendedPictographic],
[0x269D, 0x269F, extendedPictographic],
[0x26A0, 0x26A1, extendedPictographic],
[0x26A2, 0x26A6, extendedPictographic],
[0x26A7, 0x26A7, extendedPictographic],
[0x26A8, 0x26A9, extendedPictographic],
[0x26AA, 0x26AB, extendedPictographic],
[0x26AC, 0x26AF, extendedPictographic],
[0x26B0, 0x26B1, extendedPictographic],
[0x26B2, 0x26BC, extendedPictographic],
[0x26BD, 0x26BE, extendedPictographic],
[0x26BF, 0x26C3, extendedPictographic],
[0x26C4, 0x26C5, extendedPictographic],
[0x26C6, 0x26C7, extendedPictographic],
[0x26C8, 0x26C8, extendedPictographic],
[0x26C9, 0x26CD, extendedPictographic],
[0x26CE, 0x26CE, extendedPictographic],
[0x26CF, 0x26CF, extendedPictographic],
[0x26D0, 0x26D0, extendedPictographic],
[0x26D1, 0x26D1, extendedPictographic],
[0x26D2, 0x26D2, extendedPictographic],
[0x26D3, 0x26D3, extendedPictographic],
[0x26D4, 0x26D4, extendedPictographic],
[0x26D5, 0x26E8, extendedPictographic],
[0x26E9, 0x26E9, extendedPictographic],
[0x26EA, 0x26EA, extendedPictographic],
[0x26EB, 0x26EF, extendedPictographic],
[0x26F0, 0x26F1, extendedPictographic],
[0x26F2, 0x26F3, extendedPictographic],
[0x26F4, 0x26F4, extendedPictographic],
[0x26F5, 0x26F5, extendedPictographic],
[0x26F6, 0x26F6, extendedPictographic],
[0x26F7, 0x26F9, extendedPictographic],
[0x26FA, 0x26FA, extendedPictographic],
[0x26FB, 0x26FC, extendedPictographic],
[0x26FD, 0x26FD, extendedPictographic],
[0x26FE, 0x2701, extendedPictographic],
[0x2702, 0x2702, extendedPictographic],
[0x2703, 0x2704, extendedPictographic],
[0x2705, 0x2705, extendedPictographic],
[0x2708, 0x270C, extendedPictographic],
[0x270D, 0x270D, extendedPictographic],
[0x270E, 0x270E, extendedPictographic],
[0x270F, 0x270F, extendedPictographic],
[0x2710, 0x2711, extendedPictographic],
[0x2712, 0x2712, extendedPictographic],
[0x2714, 0x2714, extendedPictographic],
[0x2716, 0x2716, extendedPictographic],
[0x271D, 0x271D, extendedPictographic],
[0x2721, 0x2721, extendedPictographic],
[0x2728, 0x2728, extendedPictographic],
[0x2733, 0x2734, extendedPictographic],
[0x2744, 0x2744, extendedPictographic],
[0x2747, 0x2747, extendedPictographic],
[0x274C, 0x274C, extendedPictographic],
[0x274E, 0x274E, extendedPictographic],
[0x2753, 0x2755, extendedPictographic],
[0x2757, 0x2757, extendedPictographic],
[0x2763, 0x2763, extendedPictographic],
[0x2764, 0x2764, extendedPictographic],
[0x2765, 0x2767, extendedPictographic],
[0x2795, 0x2797, extendedPictographic],
[0x27A1, 0x27A1, extendedPictographic],
[0x27B0, 0x27B0, extendedPictographic],
[0x27BF, 0x27BF, extendedPictographic],
[0x2934, 0x2935, extendedPictographic],
[0x2B05, 0x2B07, extendedPictographic],
[0x2B1B, 0x2B1C, extendedPictographic],
[0x2B50, 0x2B50, extendedPictographic],
[0x2B55, 0x2B55, extendedPictographic],
[0x2CEF, 0x2CF1, extend],
[0x2D7F, 0x2D7F, extend],
[0x2DE0, 0x2DFF, extend],
[0x302A, 0x302D, extend],
[0x302E, 0x302F, extend],
[0x3030, 0x3030, extendedPictographic],
[0x303D, 0x303D, extendedPictographic],
[0x3099, 0x309A, extend],
[0x3297, 0x3297, extendedPictographic],
[0x3299, 0x3299, extendedPictographic],
[0xA66F, 0xA66F, extend],
[0xA670, 0xA672, extend],
[0xA674, 0xA67D, extend],
[0xA69E, 0xA69F, extend],
[0xA6F0, 0xA6F1, extend],
[0xA802, 0xA802, extend],
[0xA806, 0xA806, extend],
[0xA80B, 0xA80B, extend],
[0xA823, 0xA824, spacingMark],
[0xA825, 0xA826, extend],
[0xA827, 0xA827, spacingMark],
[0xA82C, 0xA82C, extend],
[0xA880, 0xA881, spacingMark],
[0xA8B4, 0xA8C3, spacingMark],
[0xA8C4, 0xA8C5, extend],
[0xA8E0, 0xA8F1, extend],
[0xA8FF, 0xA8FF, extend],
[0xA926, 0xA92D, extend],
[0xA947, 0xA951, extend],
[0xA952, 0xA953, spacingMark],
[0xA960, 0xA97C, l],
[0xA980, 0xA982, extend],
[0xA983, 0xA983, spacingMark],
[0xA9B3, 0xA9B3, extend],
[0xA9B4, 0xA9B5, spacingMark],
[0xA9B6, 0xA9B9, extend],
[0xA9BA, 0xA9BB, spacingMark],
[0xA9BC, 0xA9BD, extend],
[0xA9BE, 0xA9C0, spacingMark],
[0xA9E5, 0xA9E5, extend],
[0xAA29, 0xAA2E, extend],
[0xAA2F, 0xAA30, spacingMark],
[0xAA31, 0xAA32, extend],
[0xAA33, 0xAA34, spacingMark],
[0xAA35, 0xAA36, extend],
[0xAA43, 0xAA43, extend],
[0xAA4C, 0xAA4C, extend],
[0xAA4D, 0xAA4D, spacingMark],
[0xAA7C, 0xAA7C, extend],
[0xAAB0, 0xAAB0, extend],
[0xAAB2, 0xAAB4, extend],
[0xAAB7, 0xAAB8, extend],
[0xAABE, 0xAABF, extend],
[0xAAC1, 0xAAC1, extend],
[0xAAEB, 0xAAEB, spacingMark],
[0xAAEC, 0xAAED, extend],
[0xAAEE, 0xAAEF, spacingMark],
[0xAAF5, 0xAAF5, spacingMark],
[0xAAF6, 0xAAF6, extend],
[0xABE3, 0xABE4, spacingMark],
[0xABE5, 0xABE5, extend],
[0xABE6, 0xABE7, spacingMark],
[0xABE8, 0xABE8, extend],
[0xABE9, 0xABEA, spacingMark],
[0xABEC, 0xABEC, spacingMark],
[0xABED, 0xABED, extend],
[0xAC00, 0xAC00, lv],
[0xAC01, 0xAC1B, lvt],
[0xAC1C, 0xAC1C, lv],
[0xAC1D, 0xAC37, lvt],
[0xAC38, 0xAC38, lv],
[0xAC39, 0xAC53, lvt],
[0xAC54, 0xAC54, lv],
[0xAC55, 0xAC6F, lvt],
[0xAC70, 0xAC70, lv],
[0xAC71, 0xAC8B, lvt],
[0xAC8C, 0xAC8C, lv],
[0xAC8D, 0xACA7, lvt],
[0xACA8, 0xACA8, lv],
[0xACA9, 0xACC3, lvt],
[0xACC4, 0xACC4, lv],
[0xACC5, 0xACDF, lvt],
[0xACE0, 0xACE0, lv],
[0xACE1, 0xACFB, lvt],
[0xACFC, 0xACFC, lv],
[0xACFD, 0xAD17, lvt],
[0xAD18, 0xAD18, lv],
[0xAD19, 0xAD33, lvt],
[0xAD34, 0xAD34, lv],
[0xAD35, 0xAD4F, lvt],
[0xAD50, 0xAD50, lv],
[0xAD51, 0xAD6B, lvt],
[0xAD6C, 0xAD6C, lv],
[0xAD6D, 0xAD87, lvt],
[0xAD88, 0xAD88, lv],
[0xAD89, 0xADA3, lvt],
[0xADA4, 0xADA4, lv],
[0xADA5, 0xADBF, lvt],
[0xADC0, 0xADC0, lv],
[0xADC1, 0xADDB, lvt],
[0xADDC, 0xADDC, lv],
[0xADDD, 0xADF7, lvt],
[0xADF8, 0xADF8, lv],
[0xADF9, 0xAE13, lvt],
[0xAE14, 0xAE14, lv],
[0xAE15, 0xAE2F, lvt],
[0xAE30, 0xAE30, lv],
[0xAE31, 0xAE4B, lvt],
[0xAE4C, 0xAE4C, lv],
[0xAE4D, 0xAE67, lvt],
[0xAE68, 0xAE68, lv],
[0xAE69, 0xAE83, lvt],
[0xAE84, 0xAE84, lv],
[0xAE85, 0xAE9F, lvt],
[0xAEA0, 0xAEA0, lv],
[0xAEA1, 0xAEBB, lvt],
[0xAEBC, 0xAEBC, lv],
[0xAEBD, 0xAED7, lvt],
[0xAED8, 0xAED8, lv],
[0xAED9, 0xAEF3, lvt],
[0xAEF4, 0xAEF4, lv],
[0xAEF5, 0xAF0F, lvt],
[0xAF10, 0xAF10, lv],
[0xAF11, 0xAF2B, lvt],
[0xAF2C, 0xAF2C, lv],
[0xAF2D, 0xAF47, lvt],
[0xAF48, 0xAF48, lv],
[0xAF49, 0xAF63, lvt],
[0xAF64, 0xAF64, lv],
[0xAF65, 0xAF7F, lvt],
[0xAF80, 0xAF80, lv],
[0xAF81, 0xAF9B, lvt],
[0xAF9C, 0xAF9C, lv],
[0xAF9D, 0xAFB7, lvt],
[0xAFB8, 0xAFB8, lv],
[0xAFB9, 0xAFD3, lvt],
[0xAFD4, 0xAFD4, lv],
[0xAFD5, 0xAFEF, lvt],
[0xAFF0, 0xAFF0, lv],
[0xAFF1, 0xB00B, lvt],
[0xB00C, 0xB00C, lv],
[0xB00D, 0xB027, lvt],
[0xB028, 0xB028, lv],
[0xB029, 0xB043, lvt],
[0xB044, 0xB044, lv],
[0xB045, 0xB05F, lvt],
[0xB060, 0xB060, lv],
[0xB061, 0xB07B, lvt],
[0xB07C, 0xB07C, lv],
[0xB07D, 0xB097, lvt],
[0xB098, 0xB098, lv],
[0xB099, 0xB0B3, lvt],
[0xB0B4, 0xB0B4, lv],
[0xB0B5, 0xB0CF, lvt],
[0xB0D0, 0xB0D0, lv],
[0xB0D1, 0xB0EB, lvt],
[0xB0EC, 0xB0EC, lv],
[0xB0ED, 0xB107, lvt],
[0xB108, 0xB108, lv],
[0xB109, 0xB123, lvt],
[0xB124, 0xB124, lv],
[0xB125, 0xB13F, lvt],
[0xB140, 0xB140, lv],
[0xB141, 0xB15B, lvt],
[0xB15C, 0xB15C, lv],
[0xB15D, 0xB177, lvt],
[0xB178, 0xB178, lv],
[0xB179, 0xB193, lvt],
[0xB194, 0xB194, lv],
[0xB195, 0xB1AF, lvt],
[0xB1B0, 0xB1B0, lv],
[0xB1B1, 0xB1CB, lvt],
[0xB1CC, 0xB1CC, lv],
[0xB1CD, 0xB1E7, lvt],
[0xB1E8, 0xB1E8, lv],
[0xB1E9, 0xB203, lvt],
[0xB204, 0xB204, lv],
[0xB205, 0xB21F, lvt],
[0xB220, 0xB220, lv],
[0xB221, 0xB23B, lvt],
[0xB23C, 0xB23C, lv],
[0xB23D, 0xB257, lvt],
[0xB258, 0xB258, lv],
[0xB259, 0xB273, lvt],
[0xB274, 0xB274, lv],
[0xB275, 0xB28F, lvt],
[0xB290, 0xB290, lv],
[0xB291, 0xB2AB, lvt],
[0xB2AC, 0xB2AC, lv],
[0xB2AD, 0xB2C7, lvt],
[0xB2C8, 0xB2C8, lv],
[0xB2C9, 0xB2E3, lvt],
[0xB2E4, 0xB2E4, lv],
[0xB2E5, 0xB2FF, lvt],
[0xB300, 0xB300, lv],
[0xB301, 0xB31B, lvt],
[0xB31C, 0xB31C, lv],
[0xB31D, 0xB337, lvt],
[0xB338, 0xB338, lv],
[0xB339, 0xB353, lvt],
[0xB354, 0xB354, lv],
[0xB355, 0xB36F, lvt],
[0xB370, 0xB370, lv],
[0xB371, 0xB38B, lvt],
[0xB38C, 0xB38C, lv],
[0xB38D, 0xB3A7, lvt],
[0xB3A8, 0xB3A8, lv],
[0xB3A9, 0xB3C3, lvt],
[0xB3C4, 0xB3C4, lv],
[0xB3C5, 0xB3DF, lvt],
[0xB3E0, 0xB3E0, lv],
[0xB3E1, 0xB3FB, lvt],
[0xB3FC, 0xB3FC, lv],
[0xB3FD, 0xB417, lvt],
[0xB418, 0xB418, lv],
[0xB419, 0xB433, lvt],
[0xB434, 0xB434, lv],
[0xB435, 0xB44F, lvt],
[0xB450, 0xB450, lv],
[0xB451, 0xB46B, lvt],
[0xB46C, 0xB46C, lv],
[0xB46D, 0xB487, lvt],
[0xB488, 0xB488, lv],
[0xB489, 0xB4A3, lvt],
[0xB4A4, 0xB4A4, lv],
[0xB4A5, 0xB4BF, lvt],
[0xB4C0, 0xB4C0, lv],
[0xB4C1, 0xB4DB, lvt],
[0xB4DC, 0xB4DC, lv],
[0xB4DD, 0xB4F7, lvt],
[0xB4F8, 0xB4F8, lv],
[0xB4F9, 0xB513, lvt],
[0xB514, 0xB514, lv],
[0xB515, 0xB52F, lvt],
[0xB530, 0xB530, lv],
[0xB531, 0xB54B, lvt],
[0xB54C, 0xB54C, lv],
[0xB54D, 0xB567, lvt],
[0xB568, 0xB568, lv],
[0xB569, 0xB583, lvt],
[0xB584, 0xB584, lv],
[0xB585, 0xB59F, lvt],
[0xB5A0, 0xB5A0, lv],
[0xB5A1, 0xB5BB, lvt],
[0xB5BC, 0xB5BC, lv],
[0xB5BD, 0xB5D7, lvt],
[0xB5D8, 0xB5D8, lv],
[0xB5D9, 0xB5F3, lvt],
[0xB5F4, 0xB5F4, lv],
[0xB5F5, 0xB60F, lvt],
[0xB610, 0xB610, lv],
[0xB611, 0xB62B, lvt],
[0xB62C, 0xB62C, lv],
[0xB62D, 0xB647, lvt],
[0xB648, 0xB648, lv],
[0xB649, 0xB663, lvt],
[0xB664, 0xB664, lv],
[0xB665, 0xB67F, lvt],
[0xB680, 0xB680, lv],
[0xB681, 0xB69B, lvt],
[0xB69C, 0xB69C, lv],
[0xB69D, 0xB6B7, lvt],
[0xB6B8, 0xB6B8, lv],
[0xB6B9, 0xB6D3, lvt],
[0xB6D4, 0xB6D4, lv],
[0xB6D5, 0xB6EF, lvt],
[0xB6F0, 0xB6F0, lv],
[0xB6F1, 0xB70B, lvt],
[0xB70C, 0xB70C, lv],
[0xB70D, 0xB727, lvt],
[0xB728, 0xB728, lv],
[0xB729, 0xB743, lvt],
[0xB744, 0xB744, lv],
[0xB745, 0xB75F, lvt],
[0xB760, 0xB760, lv],
[0xB761, 0xB77B, lvt],
[0xB77C, 0xB77C, lv],
[0xB77D, 0xB797, lvt],
[0xB798, 0xB798, lv],
[0xB799, 0xB7B3, lvt],
[0xB7B4, 0xB7B4, lv],
[0xB7B5, 0xB7CF, lvt],
[0xB7D0, 0xB7D0, lv],
[0xB7D1, 0xB7EB, lvt],
[0xB7EC, 0xB7EC, lv],
[0xB7ED, 0xB807, lvt],
[0xB808, 0xB808, lv],
[0xB809, 0xB823, lvt],
[0xB824, 0xB824, lv],
[0xB825, 0xB83F, lvt],
[0xB840, 0xB840, lv],
[0xB841, 0xB85B, lvt],
[0xB85C, 0xB85C, lv],
[0xB85D, 0xB877, lvt],
[0xB878, 0xB878, lv],
[0xB879, 0xB893, lvt],
[0xB894, 0xB894, lv],
[0xB895, 0xB8AF, lvt],
[0xB8B0, 0xB8B0, lv],
[0xB8B1, 0xB8CB, lvt],
[0xB8CC, 0xB8CC, lv],
[0xB8CD, 0xB8E7, lvt],
[0xB8E8, 0xB8E8, lv],
[0xB8E9, 0xB903, lvt],
[0xB904, 0xB904, lv],
[0xB905, 0xB91F, lvt],
[0xB920, 0xB920, lv],
[0xB921, 0xB93B, lvt],
[0xB93C, 0xB93C, lv],
[0xB93D, 0xB957, lvt],
[0xB958, 0xB958, lv],
[0xB959, 0xB973, lvt],
[0xB974, 0xB974, lv],
[0xB975, 0xB98F, lvt],
[0xB990, 0xB990, lv],
[0xB991, 0xB9AB, lvt],
[0xB9AC, 0xB9AC, lv],
[0xB9AD, 0xB9C7, lvt],
[0xB9C8, 0xB9C8, lv],
[0xB9C9, 0xB9E3, lvt],
[0xB9E4, 0xB9E4, lv],
[0xB9E5, 0xB9FF, lvt],
[0xBA00, 0xBA00, lv],
[0xBA01, 0xBA1B, lvt],
[0xBA1C, 0xBA1C, lv],
[0xBA1D, 0xBA37, lvt],
[0xBA38, 0xBA38, lv],
[0xBA39, 0xBA53, lvt],
[0xBA54, 0xBA54, lv],
[0xBA55, 0xBA6F, lvt],
[0xBA70, 0xBA70, lv],
[0xBA71, 0xBA8B, lvt],
[0xBA8C, 0xBA8C, lv],
[0xBA8D, 0xBAA7, lvt],
[0xBAA8, 0xBAA8, lv],
[0xBAA9, 0xBAC3, lvt],
[0xBAC4, 0xBAC4, lv],
[0xBAC5, 0xBADF, lvt],
[0xBAE0, 0xBAE0, lv],
[0xBAE1, 0xBAFB, lvt],
[0xBAFC, 0xBAFC, lv],
[0xBAFD, 0xBB17, lvt],
[0xBB18, 0xBB18, lv],
[0xBB19, 0xBB33, lvt],
[0xBB34, 0xBB34, lv],
[0xBB35, 0xBB4F, lvt],
[0xBB50, 0xBB50, lv],
[0xBB51, 0xBB6B, lvt],
[0xBB6C, 0xBB6C, lv],
[0xBB6D, 0xBB87, lvt],
[0xBB88, 0xBB88, lv],
[0xBB89, 0xBBA3, lvt],
[0xBBA4, 0xBBA4, lv],
[0xBBA5, 0xBBBF, lvt],
[0xBBC0, 0xBBC0, lv],
[0xBBC1, 0xBBDB, lvt],
[0xBBDC, 0xBBDC, lv],
[0xBBDD, 0xBBF7, lvt],
[0xBBF8, 0xBBF8, lv],
[0xBBF9, 0xBC13, lvt],
[0xBC14, 0xBC14, lv],
[0xBC15, 0xBC2F, lvt],
[0xBC30, 0xBC30, lv],
[0xBC31, 0xBC4B, lvt],
[0xBC4C, 0xBC4C, lv],
[0xBC4D, 0xBC67, lvt],
[0xBC68, 0xBC68, lv],
[0xBC69, 0xBC83, lvt],
[0xBC84, 0xBC84, lv],
[0xBC85, 0xBC9F, lvt],
[0xBCA0, 0xBCA0, lv],
[0xBCA1, 0xBCBB, lvt],
[0xBCBC, 0xBCBC, lv],
[0xBCBD, 0xBCD7, lvt],
[0xBCD8, 0xBCD8, lv],
[0xBCD9, 0xBCF3, lvt],
[0xBCF4, 0xBCF4, lv],
[0xBCF5, 0xBD0F, lvt],
[0xBD10, 0xBD10, lv],
[0xBD11, 0xBD2B, lvt],
[0xBD2C, 0xBD2C, lv],
[0xBD2D, 0xBD47, lvt],
[0xBD48, 0xBD48, lv],
[0xBD49, 0xBD63, lvt],
[0xBD64, 0xBD64, lv],
[0xBD65, 0xBD7F, lvt],
[0xBD80, 0xBD80, lv],
[0xBD81, 0xBD9B, lvt],
[0xBD9C, 0xBD9C, lv],
[0xBD9D, 0xBDB7, lvt],
[0xBDB8, 0xBDB8, lv],
[0xBDB9, 0xBDD3, lvt],
[0xBDD4, 0xBDD4, lv],
[0xBDD5, 0xBDEF, lvt],
[0xBDF0, 0xBDF0, lv],
[0xBDF1, 0xBE0B, lvt],
[0xBE0C, 0xBE0C, lv],
[0xBE0D, 0xBE27, lvt],
[0xBE28, 0xBE28, lv],
[0xBE29, 0xBE43, lvt],
[0xBE44, 0xBE44, lv],
[0xBE45, 0xBE5F, lvt],
[0xBE60, 0xBE60, lv],
[0xBE61, 0xBE7B, lvt],
[0xBE7C, 0xBE7C, lv],
[0xBE7D, 0xBE97, lvt],
[0xBE98, 0xBE98, lv],
[0xBE99, 0xBEB3, lvt],
[0xBEB4, 0xBEB4, lv],
[0xBEB5, 0xBECF, lvt],
[0xBED0, 0xBED0, lv],
[0xBED1, 0xBEEB, lvt],
[0xBEEC, 0xBEEC, lv],
[0xBEED, 0xBF07, lvt],
[0xBF08, 0xBF08, lv],
[0xBF09, 0xBF23, lvt],
[0xBF24, 0xBF24, lv],
[0xBF25, 0xBF3F, lvt],
[0xBF40, 0xBF40, lv],
[0xBF41, 0xBF5B, lvt],
[0xBF5C, 0xBF5C, lv],
[0xBF5D, 0xBF77, lvt],
[0xBF78, 0xBF78, lv],
[0xBF79, 0xBF93, lvt],
[0xBF94, 0xBF94, lv],
[0xBF95, 0xBFAF, lvt],
[0xBFB0, 0xBFB0, lv],
[0xBFB1, 0xBFCB, lvt],
[0xBFCC, 0xBFCC, lv],
[0xBFCD, 0xBFE7, lvt],
[0xBFE8, 0xBFE8, lv],
[0xBFE9, 0xC003, lvt],
[0xC004, 0xC004, lv],
[0xC005, 0xC01F, lvt],
[0xC020, 0xC020, lv],
[0xC021, 0xC03B, lvt],
[0xC03C, 0xC03C, lv],
[0xC03D, 0xC057, lvt],
[0xC058, 0xC058, lv],
[0xC059, 0xC073, lvt],
[0xC074, 0xC074, lv],
[0xC075, 0xC08F, lvt],
[0xC090, 0xC090, lv],
[0xC091, 0xC0AB, lvt],
[0xC0AC, 0xC0AC, lv],
[0xC0AD, 0xC0C7, lvt],
[0xC0C8, 0xC0C8, lv],
[0xC0C9, 0xC0E3, lvt],
[0xC0E4, 0xC0E4, lv],
[0xC0E5, 0xC0FF, lvt],
[0xC100, 0xC100, lv],
[0xC101, 0xC11B, lvt],
[0xC11C, 0xC11C, lv],
[0xC11D, 0xC137, lvt],
[0xC138, 0xC138, lv],
[0xC139, 0xC153, lvt],
[0xC154, 0xC154, lv],
[0xC155, 0xC16F, lvt],
[0xC170, 0xC170, lv],
[0xC171, 0xC18B, lvt],
[0xC18C, 0xC18C, lv],
[0xC18D, 0xC1A7, lvt],
[0xC1A8, 0xC1A8, lv],
[0xC1A9, 0xC1C3, lvt],
[0xC1C4, 0xC1C4, lv],
[0xC1C5, 0xC1DF, lvt],
[0xC1E0, 0xC1E0, lv],
[0xC1E1, 0xC1FB, lvt],
[0xC1FC, 0xC1FC, lv],
[0xC1FD, 0xC217, lvt],
[0xC218, 0xC218, lv],
[0xC219, 0xC233, lvt],
[0xC234, 0xC234, lv],
[0xC235, 0xC24F, lvt],
[0xC250, 0xC250, lv],
[0xC251, 0xC26B, lvt],
[0xC26C, 0xC26C, lv],
[0xC26D, 0xC287, lvt],
[0xC288, 0xC288, lv],
[0xC289, 0xC2A3, lvt],
[0xC2A4, 0xC2A4, lv],
[0xC2A5, 0xC2BF, lvt],
[0xC2C0, 0xC2C0, lv],
[0xC2C1, 0xC2DB, lvt],
[0xC2DC, 0xC2DC, lv],
[0xC2DD, 0xC2F7, lvt],
[0xC2F8, 0xC2F8, lv],
[0xC2F9, 0xC313, lvt],
[0xC314, 0xC314, lv],
[0xC315, 0xC32F, lvt],
[0xC330, 0xC330, lv],
[0xC331, 0xC34B, lvt],
[0xC34C, 0xC34C, lv],
[0xC34D, 0xC367, lvt],
[0xC368, 0xC368, lv],
[0xC369, 0xC383, lvt],
[0xC384, 0xC384, lv],
[0xC385, 0xC39F, lvt],
[0xC3A0, 0xC3A0, lv],
[0xC3A1, 0xC3BB, lvt],
[0xC3BC, 0xC3BC, lv],
[0xC3BD, 0xC3D7, lvt],
[0xC3D8, 0xC3D8, lv],
[0xC3D9, 0xC3F3, lvt],
[0xC3F4, 0xC3F4, lv],
[0xC3F5, 0xC40F, lvt],
[0xC410, 0xC410, lv],
[0xC411, 0xC42B, lvt],
[0xC42C, 0xC42C, lv],
[0xC42D, 0xC447, lvt],
[0xC448, 0xC448, lv],
[0xC449, 0xC463, lvt],
[0xC464, 0xC464, lv],
[0xC465, 0xC47F, lvt],
[0xC480, 0xC480, lv],
[0xC481, 0xC49B, lvt],
[0xC49C, 0xC49C, lv],
[0xC49D, 0xC4B7, lvt],
[0xC4B8, 0xC4B8, lv],
[0xC4B9, 0xC4D3, lvt],
[0xC4D4, 0xC4D4, lv],
[0xC4D5, 0xC4EF, lvt],
[0xC4F0, 0xC4F0, lv],
[0xC4F1, 0xC50B, lvt],
[0xC50C, 0xC50C, lv],
[0xC50D, 0xC527, lvt],
[0xC528, 0xC528, lv],
[0xC529, 0xC543, lvt],
[0xC544, 0xC544, lv],
[0xC545, 0xC55F, lvt],
[0xC560, 0xC560, lv],
[0xC561, 0xC57B, lvt],
[0xC57C, 0xC57C, lv],
[0xC57D, 0xC597, lvt],
[0xC598, 0xC598, lv],
[0xC599, 0xC5B3, lvt],
[0xC5B4, 0xC5B4, lv],
[0xC5B5, 0xC5CF, lvt],
[0xC5D0, 0xC5D0, lv],
[0xC5D1, 0xC5EB, lvt],
[0xC5EC, 0xC5EC, lv],
[0xC5ED, 0xC607, lvt],
[0xC608, 0xC608, lv],
[0xC609, 0xC623, lvt],
[0xC624, 0xC624, lv],
[0xC625, 0xC63F, lvt],
[0xC640, 0xC640, lv],
[0xC641, 0xC65B, lvt],
[0xC65C, 0xC65C, lv],
[0xC65D, 0xC677, lvt],
[0xC678, 0xC678, lv],
[0xC679, 0xC693, lvt],
[0xC694, 0xC694, lv],
[0xC695, 0xC6AF, lvt],
[0xC6B0, 0xC6B0, lv],
[0xC6B1, 0xC6CB, lvt],
[0xC6CC, 0xC6CC, lv],
[0xC6CD, 0xC6E7, lvt],
[0xC6E8, 0xC6E8, lv],
[0xC6E9, 0xC703, lvt],
[0xC704, 0xC704, lv],
[0xC705, 0xC71F, lvt],
[0xC720, 0xC720, lv],
[0xC721, 0xC73B, lvt],
[0xC73C, 0xC73C, lv],
[0xC73D, 0xC757, lvt],
[0xC758, 0xC758, lv],
[0xC759, 0xC773, lvt],
[0xC774, 0xC774, lv],
[0xC775, 0xC78F, lvt],
[0xC790, 0xC790, lv],
[0xC791, 0xC7AB, lvt],
[0xC7AC, 0xC7AC, lv],
[0xC7AD, 0xC7C7, lvt],
[0xC7C8, 0xC7C8, lv],
[0xC7C9, 0xC7E3, lvt],
[0xC7E4, 0xC7E4, lv],
[0xC7E5, 0xC7FF, lvt],
[0xC800, 0xC800, lv],
[0xC801, 0xC81B, lvt],
[0xC81C, 0xC81C, lv],
[0xC81D, 0xC837, lvt],
[0xC838, 0xC838, lv],
[0xC839, 0xC853, lvt],
[0xC854, 0xC854, lv],
[0xC855, 0xC86F, lvt],
[0xC870, 0xC870, lv],
[0xC871, 0xC88B, lvt],
[0xC88C, 0xC88C, lv],
[0xC88D, 0xC8A7, lvt],
[0xC8A8, 0xC8A8, lv],
[0xC8A9, 0xC8C3, lvt],
[0xC8C4, 0xC8C4, lv],
[0xC8C5, 0xC8DF, lvt],
[0xC8E0, 0xC8E0, lv],
[0xC8E1, 0xC8FB, lvt],
[0xC8FC, 0xC8FC, lv],
[0xC8FD, 0xC917, lvt],
[0xC918, 0xC918, lv],
[0xC919, 0xC933, lvt],
[0xC934, 0xC934, lv],
[0xC935, 0xC94F, lvt],
[0xC950, 0xC950, lv],
[0xC951, 0xC96B, lvt],
[0xC96C, 0xC96C, lv],
[0xC96D, 0xC987, lvt],
[0xC988, 0xC988, lv],
[0xC989, 0xC9A3, lvt],
[0xC9A4, 0xC9A4, lv],
[0xC9A5, 0xC9BF, lvt],
[0xC9C0, 0xC9C0, lv],
[0xC9C1, 0xC9DB, lvt],
[0xC9DC, 0xC9DC, lv],
[0xC9DD, 0xC9F7, lvt],
[0xC9F8, 0xC9F8, lv],
[0xC9F9, 0xCA13, lvt],
[0xCA14, 0xCA14, lv],
[0xCA15, 0xCA2F, lvt],
[0xCA30, 0xCA30, lv],
[0xCA31, 0xCA4B, lvt],
[0xCA4C, 0xCA4C, lv],
[0xCA4D, 0xCA67, lvt],
[0xCA68, 0xCA68, lv],
[0xCA69, 0xCA83, lvt],
[0xCA84, 0xCA84, lv],
[0xCA85, 0xCA9F, lvt],
[0xCAA0, 0xCAA0, lv],
[0xCAA1, 0xCABB, lvt],
[0xCABC, 0xCABC, lv],
[0xCABD, 0xCAD7, lvt],
[0xCAD8, 0xCAD8, lv],
[0xCAD9, 0xCAF3, lvt],
[0xCAF4, 0xCAF4, lv],
[0xCAF5, 0xCB0F, lvt],
[0xCB10, 0xCB10, lv],
[0xCB11, 0xCB2B, lvt],
[0xCB2C, 0xCB2C, lv],
[0xCB2D, 0xCB47, lvt],
[0xCB48, 0xCB48, lv],
[0xCB49, 0xCB63, lvt],
[0xCB64, 0xCB64, lv],
[0xCB65, 0xCB7F, lvt],
[0xCB80, 0xCB80, lv],
[0xCB81, 0xCB9B, lvt],
[0xCB9C, 0xCB9C, lv],
[0xCB9D, 0xCBB7, lvt],
[0xCBB8, 0xCBB8, lv],
[0xCBB9, 0xCBD3, lvt],
[0xCBD4, 0xCBD4, lv],
[0xCBD5, 0xCBEF, lvt],
[0xCBF0, 0xCBF0, lv],
[0xCBF1, 0xCC0B, lvt],
[0xCC0C, 0xCC0C, lv],
[0xCC0D, 0xCC27, lvt],
[0xCC28, 0xCC28, lv],
[0xCC29, 0xCC43, lvt],
[0xCC44, 0xCC44, lv],
[0xCC45, 0xCC5F, lvt],
[0xCC60, 0xCC60, lv],
[0xCC61, 0xCC7B, lvt],
[0xCC7C, 0xCC7C, lv],
[0xCC7D, 0xCC97, lvt],
[0xCC98, 0xCC98, lv],
[0xCC99, 0xCCB3, lvt],
[0xCCB4, 0xCCB4, lv],
[0xCCB5, 0xCCCF, lvt],
[0xCCD0, 0xCCD0, lv],
[0xCCD1, 0xCCEB, lvt],
[0xCCEC, 0xCCEC, lv],
[0xCCED, 0xCD07, lvt],
[0xCD08, 0xCD08, lv],
[0xCD09, 0xCD23, lvt],
[0xCD24, 0xCD24, lv],
[0xCD25, 0xCD3F, lvt],
[0xCD40, 0xCD40, lv],
[0xCD41, 0xCD5B, lvt],
[0xCD5C, 0xCD5C, lv],
[0xCD5D, 0xCD77, lvt],
[0xCD78, 0xCD78, lv],
[0xCD79, 0xCD93, lvt],
[0xCD94, 0xCD94, lv],
[0xCD95, 0xCDAF, lvt],
[0xCDB0, 0xCDB0, lv],
[0xCDB1, 0xCDCB, lvt],
[0xCDCC, 0xCDCC, lv],
[0xCDCD, 0xCDE7, lvt],
[0xCDE8, 0xCDE8, lv],
[0xCDE9, 0xCE03, lvt],
[0xCE04, 0xCE04, lv],
[0xCE05, 0xCE1F, lvt],
[0xCE20, 0xCE20, lv],
[0xCE21, 0xCE3B, lvt],
[0xCE3C, 0xCE3C, lv],
[0xCE3D, 0xCE57, lvt],
[0xCE58, 0xCE58, lv],
[0xCE59, 0xCE73, lvt],
[0xCE74, 0xCE74, lv],
[0xCE75, 0xCE8F, lvt],
[0xCE90, 0xCE90, lv],
[0xCE91, 0xCEAB, lvt],
[0xCEAC, 0xCEAC, lv],
[0xCEAD, 0xCEC7, lvt],
[0xCEC8, 0xCEC8, lv],
[0xCEC9, 0xCEE3, lvt],
[0xCEE4, 0xCEE4, lv],
[0xCEE5, 0xCEFF, lvt],
[0xCF00, 0xCF00, lv],
[0xCF01, 0xCF1B, lvt],
[0xCF1C, 0xCF1C, lv],
[0xCF1D, 0xCF37, lvt],
[0xCF38, 0xCF38, lv],
[0xCF39, 0xCF53, lvt],
[0xCF54, 0xCF54, lv],
[0xCF55, 0xCF6F, lvt],
[0xCF70, 0xCF70, lv],
[0xCF71, 0xCF8B, lvt],
[0xCF8C, 0xCF8C, lv],
[0xCF8D, 0xCFA7, lvt],
[0xCFA8, 0xCFA8, lv],
[0xCFA9, 0xCFC3, lvt],
[0xCFC4, 0xCFC4, lv],
[0xCFC5, 0xCFDF, lvt],
[0xCFE0, 0xCFE0, lv],
[0xCFE1, 0xCFFB, lvt],
[0xCFFC, 0xCFFC, lv],
[0xCFFD, 0xD017, lvt],
[0xD018, 0xD018, lv],
[0xD019, 0xD033, lvt],
[0xD034, 0xD034, lv],
[0xD035, 0xD04F, lvt],
[0xD050, 0xD050, lv],
[0xD051, 0xD06B, lvt],
[0xD06C, 0xD06C, lv],
[0xD06D, 0xD087, lvt],
[0xD088, 0xD088, lv],
[0xD089, 0xD0A3, lvt],
[0xD0A4, 0xD0A4, lv],
[0xD0A5, 0xD0BF, lvt],
[0xD0C0, 0xD0C0, lv],
[0xD0C1, 0xD0DB, lvt],
[0xD0DC, 0xD0DC, lv],
[0xD0DD, 0xD0F7, lvt],
[0xD0F8, 0xD0F8, lv],
[0xD0F9, 0xD113, lvt],
[0xD114, 0xD114, lv],
[0xD115, 0xD12F, lvt],
[0xD130, 0xD130, lv],
[0xD131, 0xD14B, lvt],
[0xD14C, 0xD14C, lv],
[0xD14D, 0xD167, lvt],
[0xD168, 0xD168, lv],
[0xD169, 0xD183, lvt],
[0xD184, 0xD184, lv],
[0xD185, 0xD19F, lvt],
[0xD1A0, 0xD1A0, lv],
[0xD1A1, 0xD1BB, lvt],
[0xD1BC, 0xD1BC, lv],
[0xD1BD, 0xD1D7, lvt],
[0xD1D8, 0xD1D8, lv],
[0xD1D9, 0xD1F3, lvt],
[0xD1F4, 0xD1F4, lv],
[0xD1F5, 0xD20F, lvt],
[0xD210, 0xD210, lv],
[0xD211, 0xD22B, lvt],
[0xD22C, 0xD22C, lv],
[0xD22D, 0xD247, lvt],
[0xD248, 0xD248, lv],
[0xD249, 0xD263, lvt],
[0xD264, 0xD264, lv],
[0xD265, 0xD27F, lvt],
[0xD280, 0xD280, lv],
[0xD281, 0xD29B, lvt],
[0xD29C, 0xD29C, lv],
[0xD29D, 0xD2B7, lvt],
[0xD2B8, 0xD2B8, lv],
[0xD2B9, 0xD2D3, lvt],
[0xD2D4, 0xD2D4, lv],
[0xD2D5, 0xD2EF, lvt],
[0xD2F0, 0xD2F0, lv],
[0xD2F1, 0xD30B, lvt],
[0xD30C, 0xD30C, lv],
[0xD30D, 0xD327, lvt],
[0xD328, 0xD328, lv],
[0xD329, 0xD343, lvt],
[0xD344, 0xD344, lv],
[0xD345, 0xD35F, lvt],
[0xD360, 0xD360, lv],
[0xD361, 0xD37B, lvt],
[0xD37C, 0xD37C, lv],
[0xD37D, 0xD397, lvt],
[0xD398, 0xD398, lv],
[0xD399, 0xD3B3, lvt],
[0xD3B4, 0xD3B4, lv],
[0xD3B5, 0xD3CF, lvt],
[0xD3D0, 0xD3D0, lv],
[0xD3D1, 0xD3EB, lvt],
[0xD3EC, 0xD3EC, lv],
[0xD3ED, 0xD407, lvt],
[0xD408, 0xD408, lv],
[0xD409, 0xD423, lvt],
[0xD424, 0xD424, lv],
[0xD425, 0xD43F, lvt],
[0xD440, 0xD440, lv],
[0xD441, 0xD45B, lvt],
[0xD45C, 0xD45C, lv],
[0xD45D, 0xD477, lvt],
[0xD478, 0xD478, lv],
[0xD479, 0xD493, lvt],
[0xD494, 0xD494, lv],
[0xD495, 0xD4AF, lvt],
[0xD4B0, 0xD4B0, lv],
[0xD4B1, 0xD4CB, lvt],
[0xD4CC, 0xD4CC, lv],
[0xD4CD, 0xD4E7, lvt],
[0xD4E8, 0xD4E8, lv],
[0xD4E9, 0xD503, lvt],
[0xD504, 0xD504, lv],
[0xD505, 0xD51F, lvt],
[0xD520, 0xD520, lv],
[0xD521, 0xD53B, lvt],
[0xD53C, 0xD53C, lv],
[0xD53D, 0xD557, lvt],
[0xD558, 0xD558, lv],
[0xD559, 0xD573, lvt],
[0xD574, 0xD574, lv],
[0xD575, 0xD58F, lvt],
[0xD590, 0xD590, lv],
[0xD591, 0xD5AB, lvt],
[0xD5AC, 0xD5AC, lv],
[0xD5AD, 0xD5C7, lvt],
[0xD5C8, 0xD5C8, lv],
[0xD5C9, 0xD5E3, lvt],
[0xD5E4, 0xD5E4, lv],
[0xD5E5, 0xD5FF, lvt],
[0xD600, 0xD600, lv],
[0xD601, 0xD61B, lvt],
[0xD61C, 0xD61C, lv],
[0xD61D, 0xD637, lvt],
[0xD638, 0xD638, lv],
[0xD639, 0xD653, lvt],
[0xD654, 0xD654, lv],
[0xD655, 0xD66F, lvt],
[0xD670, 0xD670, lv],
[0xD671, 0xD68B, lvt],
[0xD68C, 0xD68C, lv],
[0xD68D, 0xD6A7, lvt],
[0xD6A8, 0xD6A8, lv],
[0xD6A9, 0xD6C3, lvt],
[0xD6C4, 0xD6C4, lv],
[0xD6C5, 0xD6DF, lvt],
[0xD6E0, 0xD6E0, lv],
[0xD6E1, 0xD6FB, lvt],
[0xD6FC, 0xD6FC, lv],
[0xD6FD, 0xD717, lvt],
[0xD718, 0xD718, lv],
[0xD719, 0xD733, lvt],
[0xD734, 0xD734, lv],
[0xD735, 0xD74F, lvt],
[0xD750, 0xD750, lv],
[0xD751, 0xD76B, lvt],
[0xD76C, 0xD76C, lv],
[0xD76D, 0xD787, lvt],
[0xD788, 0xD788, lv],
[0xD789, 0xD7A3, lvt],
[0xD7B0, 0xD7C6, v],
[0xD7CB, 0xD7FB, t],
[0xFB1E, 0xFB1E, extend],
[0xFE00, 0xFE0F, extend],
[0xFE20, 0xFE2F, extend],
[0xFEFF, 0xFEFF, control],
[0xFF9E, 0xFF9F, extend],
[0xFFF0, 0xFFF8, control],
[0xFFF9, 0xFFFB, control],
[0x101FD, 0x101FD, extend],
[0x102E0, 0x102E0, extend],
[0x10376, 0x1037A, extend],
[0x10A01, 0x10A03, extend],
[0x10A05, 0x10A06, extend],
[0x10A0C, 0x10A0F, extend],
[0x10A38, 0x10A3A, extend],
[0x10A3F, 0x10A3F, extend],
[0x10AE5, 0x10AE6, extend],
[0x10D24, 0x10D27, extend],
[0x10EAB, 0x10EAC, extend],
[0x10EFD, 0x10EFF, extend],
[0x10F46, 0x10F50, extend],
[0x10F82, 0x10F85, extend],
[0x11000, 0x11000, spacingMark],
[0x11001, 0x11001, extend],
[0x11002, 0x11002, spacingMark],
[0x11038, 0x11046, extend],
[0x11070, 0x11070, extend],
[0x11073, 0x11074, extend],
[0x1107F, 0x11081, extend],
[0x11082, 0x11082, spacingMark],
[0x110B0, 0x110B2, spacingMark],
[0x110B3, 0x110B6, extend],
[0x110B7, 0x110B8, spacingMark],
[0x110B9, 0x110BA, extend],
[0x110BD, 0x110BD, prepend],
[0x110C2, 0x110C2, extend],
[0x110CD, 0x110CD, prepend],
[0x11100, 0x11102, extend],
[0x11127, 0x1112B, extend],
[0x1112C, 0x1112C, spacingMark],
[0x1112D, 0x11134, extend],
[0x11145, 0x11146, spacingMark],
[0x11173, 0x11173, extend],
[0x11180, 0x11181, extend],
[0x11182, 0x11182, spacingMark],
[0x111B3, 0x111B5, spacingMark],
[0x111B6, 0x111BE, extend],
[0x111BF, 0x111C0, spacingMark],
[0x111C2, 0x111C3, prepend],
[0x111C9, 0x111CC, extend],
[0x111CE, 0x111CE, spacingMark],
[0x111CF, 0x111CF, extend],
[0x1122C, 0x1122E, spacingMark],
[0x1122F, 0x11231, extend],
[0x11232, 0x11233, spacingMark],
[0x11234, 0x11234, extend],
[0x11235, 0x11235, spacingMark],
[0x11236, 0x11237, extend],
[0x1123E, 0x1123E, extend],
[0x11241, 0x11241, extend],
[0x112DF, 0x112DF, extend],
[0x112E0, 0x112E2, spacingMark],
[0x112E3, 0x112EA, extend],
[0x11300, 0x11301, extend],
[0x11302, 0x11303, spacingMark],
[0x1133B, 0x1133C, extend],
[0x1133E, 0x1133E, extend],
[0x1133F, 0x1133F, spacingMark],
[0x11340, 0x11340, extend],
[0x11341, 0x11344, spacingMark],
[0x11347, 0x11348, spacingMark],
[0x1134B, 0x1134D, spacingMark],
[0x11357, 0x11357, extend],
[0x11362, 0x11363, spacingMark],
[0x11366, 0x1136C, extend],
[0x11370, 0x11374, extend],
[0x11435, 0x11437, spacingMark],
[0x11438, 0x1143F, extend],
[0x11440, 0x11441, spacingMark],
[0x11442, 0x11444, extend],
[0x11445, 0x11445, spacingMark],
[0x11446, 0x11446, extend],
[0x1145E, 0x1145E, extend],
[0x114B0, 0x114B0, extend],
[0x114B1, 0x114B2, spacingMark],
[0x114B3, 0x114B8, extend],
[0x114B9, 0x114B9, spacingMark],
[0x114BA, 0x114BA, extend],
[0x114BB, 0x114BC, spacingMark],
[0x114BD, 0x114BD, extend],
[0x114BE, 0x114BE, spacingMark],
[0x114BF, 0x114C0, extend],
[0x114C1, 0x114C1, spacingMark],
[0x114C2, 0x114C3, extend],
[0x115AF, 0x115AF, extend],
[0x115B0, 0x115B1, spacingMark],
[0x115B2, 0x115B5, extend],
[0x115B8, 0x115BB, spacingMark],
[0x115BC, 0x115BD, extend],
[0x115BE, 0x115BE, spacingMark],
[0x115BF, 0x115C0, extend],
[0x115DC, 0x115DD, extend],
[0x11630, 0x11632, spacingMark],
[0x11633, 0x1163A, extend],
[0x1163B, 0x1163C, spacingMark],
[0x1163D, 0x1163D, extend],
[0x1163E, 0x1163E, spacingMark],
[0x1163F, 0x11640, extend],
[0x116AB, 0x116AB, extend],
[0x116AC, 0x116AC, spacingMark],
[0x116AD, 0x116AD, extend],
[0x116AE, 0x116AF, spacingMark],
[0x116B0, 0x116B5, extend],
[0x116B6, 0x116B6, spacingMark],
[0x116B7, 0x116B7, extend],
[0x1171D, 0x1171F, extend],
[0x11722, 0x11725, extend],
[0x11726, 0x11726, spacingMark],
[0x11727, 0x1172B, extend],
[0x1182C, 0x1182E, spacingMark],
[0x1182F, 0x11837, extend],
[0x11838, 0x11838, spacingMark],
[0x11839, 0x1183A, extend],
[0x11930, 0x11930, extend],
[0x11931, 0x11935, spacingMark],
[0x11937, 0x11938, spacingMark],
[0x1193B, 0x1193C, extend],
[0x1193D, 0x1193D, spacingMark],
[0x1193E, 0x1193E, extend],
[0x1193F, 0x1193F, prepend],
[0x11940, 0x11940, spacingMark],
[0x11941, 0x11941, prepend],
[0x11942, 0x11942, spacingMark],
[0x11943, 0x11943, extend],
[0x119D1, 0x119D3, spacingMark],
[0x119D4, 0x119D7, extend],
[0x119DA, 0x119DB, extend],
[0x119DC, 0x119DF, spacingMark],
[0x119E0, 0x119E0, extend],
[0x119E4, 0x119E4, spacingMark],
[0x11A01, 0x11A0A, extend],
[0x11A33, 0x11A38, extend],
[0x11A39, 0x11A39, spacingMark],
[0x11A3A, 0x11A3A, prepend],
[0x11A3B, 0x11A3E, extend],
[0x11A47, 0x11A47, extend],
[0x11A51, 0x11A56, extend],
[0x11A57, 0x11A58, spacingMark],
[0x11A59, 0x11A5B, extend],
[0x11A84, 0x11A89, prepend],
[0x11A8A, 0x11A96, extend],
[0x11A97, 0x11A97, spacingMark],
[0x11A98, 0x11A99, extend],
[0x11C2F, 0x11C2F, spacingMark],
[0x11C30, 0x11C36, extend],
[0x11C38, 0x11C3D, extend],
[0x11C3E, 0x11C3E, spacingMark],
[0x11C3F, 0x11C3F, extend],
[0x11C92, 0x11CA7, extend],
[0x11CA9, 0x11CA9, spacingMark],
[0x11CAA, 0x11CB0, extend],
[0x11CB1, 0x11CB1, spacingMark],
[0x11CB2, 0x11CB3, extend],
[0x11CB4, 0x11CB4, spacingMark],
[0x11CB5, 0x11CB6, extend],
[0x11D31, 0x11D36, extend],
[0x11D3A, 0x11D3A, extend],
[0x11D3C, 0x11D3D, extend],
[0x11D3F, 0x11D45, extend],
[0x11D46, 0x11D46, prepend],
[0x11D47, 0x11D47, extend],
[0x11D8A, 0x11D8E, spacingMark],
[0x11D90, 0x11D91, extend],
[0x11D93, 0x11D94, spacingMark],
[0x11D95, 0x11D95, extend],
[0x11D96, 0x11D96, spacingMark],
[0x11D97, 0x11D97, extend],
[0x11EF3, 0x11EF4, extend],
[0x11EF5, 0x11EF6, spacingMark],
[0x11F00, 0x11F01, extend],
[0x11F02, 0x11F02, prepend],
[0x11F03, 0x11F03, spacingMark],
[0x11F34, 0x11F35, spacingMark],
[0x11F36, 0x11F3A, extend],
[0x11F3E, 0x11F3F, spacingMark],
[0x11F40, 0x11F40, extend],
[0x11F41, 0x11F41, spacingMark],
[0x11F42, 0x11F42, extend],
[0x13430, 0x1343F, control],
[0x13440, 0x13440, extend],
[0x13447, 0x13455, extend],
[0x16AF0, 0x16AF4, extend],
[0x16B30, 0x16B36, extend],
[0x16F4F, 0x16F4F, extend],
[0x16F51, 0x16F87, spacingMark],
[0x16F8F, 0x16F92, extend],
[0x16FE4, 0x16FE4, extend],
[0x16FF0, 0x16FF1, spacingMark],
[0x1BC9D, 0x1BC9E, extend],
[0x1BCA0, 0x1BCA3, control],
[0x1CF00, 0x1CF2D, extend],
[0x1CF30, 0x1CF46, extend],
[0x1D165, 0x1D165, extend],
[0x1D166, 0x1D166, spacingMark],
[0x1D167, 0x1D169, extend],
[0x1D16D, 0x1D16D, spacingMark],
[0x1D16E, 0x1D172, extend],
[0x1D173, 0x1D17A, control],
[0x1D17B, 0x1D182, extend],
[0x1D185, 0x1D18B, extend],
[0x1D1AA, 0x1D1AD, extend],
[0x1D242, 0x1D244, extend],
[0x1DA00, 0x1DA36, extend],
[0x1DA3B, 0x1DA6C, extend],
[0x1DA75, 0x1DA75, extend],
[0x1DA84, 0x1DA84, extend],
[0x1DA9B, 0x1DA9F, extend],
[0x1DAA1, 0x1DAAF, extend],
[0x1E000, 0x1E006, extend],
[0x1E008, 0x1E018, extend],
[0x1E01B, 0x1E021, extend],
[0x1E023, 0x1E024, extend],
[0x1E026, 0x1E02A, extend],
[0x1E08F, 0x1E08F, extend],
[0x1E130, 0x1E136, extend],
[0x1E2AE, 0x1E2AE, extend],
[0x1E2EC, 0x1E2EF, extend],
[0x1E4EC, 0x1E4EF, extend],
[0x1E8D0, 0x1E8D6, extend],
[0x1E944, 0x1E94A, extend],
[0x1F000, 0x1F003, extendedPictographic],
[0x1F004, 0x1F004, extendedPictographic],
[0x1F005, 0x1F0CE, extendedPictographic],
[0x1F0CF, 0x1F0CF, extendedPictographic],
[0x1F0D0, 0x1F0FF, extendedPictographic],
[0x1F10D, 0x1F10F, extendedPictographic],
[0x1F12F, 0x1F12F, extendedPictographic],
[0x1F16C, 0x1F16F, extendedPictographic],
[0x1F170, 0x1F171, extendedPictographic],
[0x1F17E, 0x1F17F, extendedPictographic],
[0x1F18E, 0x1F18E, extendedPictographic],
[0x1F191, 0x1F19A, extendedPictographic],
[0x1F1AD, 0x1F1E5, extendedPictographic],
[0x1F1E6, 0x1F1FF, regionalIndicator],
[0x1F201, 0x1F202, extendedPictographic],
[0x1F203, 0x1F20F, extendedPictographic],
[0x1F21A, 0x1F21A, extendedPictographic],
[0x1F22F, 0x1F22F, extendedPictographic],
[0x1F232, 0x1F23A, extendedPictographic],
[0x1F23C, 0x1F23F, extendedPictographic],
[0x1F249, 0x1F24F, extendedPictographic],
[0x1F250, 0x1F251, extendedPictographic],
[0x1F252, 0x1F2FF, extendedPictographic],
[0x1F300, 0x1F30C, extendedPictographic],
[0x1F30D, 0x1F30E, extendedPictographic],
[0x1F30F, 0x1F30F, extendedPictographic],
[0x1F310, 0x1F310, extendedPictographic],
[0x1F311, 0x1F311, extendedPictographic],
[0x1F312, 0x1F312, extendedPictographic],
[0x1F313, 0x1F315, extendedPictographic],
[0x1F316, 0x1F318, extendedPictographic],
[0x1F319, 0x1F319, extendedPictographic],
[0x1F31A, 0x1F31A, extendedPictographic],
[0x1F31B, 0x1F31B, extendedPictographic],
[0x1F31C, 0x1F31C, extendedPictographic],
[0x1F31D, 0x1F31E, extendedPictographic],
[0x1F31F, 0x1F320, extendedPictographic],
[0x1F321, 0x1F321, extendedPictographic],
[0x1F322, 0x1F323, extendedPictographic],
[0x1F324, 0x1F32C, extendedPictographic],
[0x1F32D, 0x1F32F, extendedPictographic],
[0x1F330, 0x1F331, extendedPictographic],
[0x1F332, 0x1F333, extendedPictographic],
[0x1F334, 0x1F335, extendedPictographic],
[0x1F336, 0x1F336, extendedPictographic],
[0x1F337, 0x1F34A, extendedPictographic],
[0x1F34B, 0x1F34B, extendedPictographic],
[0x1F34C, 0x1F34F, extendedPictographic],
[0x1F350, 0x1F350, extendedPictographic],
[0x1F351, 0x1F37B, extendedPictographic],
[0x1F37C, 0x1F37C, extendedPictographic],
[0x1F37D, 0x1F37D, extendedPictographic],
[0x1F37E, 0x1F37F, extendedPictographic],
[0x1F380, 0x1F393, extendedPictographic],
[0x1F394, 0x1F395, extendedPictographic],
[0x1F396, 0x1F397, extendedPictographic],
[0x1F398, 0x1F398, extendedPictographic],
[0x1F399, 0x1F39B, extendedPictographic],
[0x1F39C, 0x1F39D, extendedPictographic],
[0x1F39E, 0x1F39F, extendedPictographic],
[0x1F3A0, 0x1F3C4, extendedPictographic],
[0x1F3C5, 0x1F3C5, extendedPictographic],
[0x1F3C6, 0x1F3C6, extendedPictographic],
[0x1F3C7, 0x1F3C7, extendedPictographic],
[0x1F3C8, 0x1F3C8, extendedPictographic],
[0x1F3C9, 0x1F3C9, extendedPictographic],
[0x1F3CA, 0x1F3CA, extendedPictographic],
[0x1F3CB, 0x1F3CE, extendedPictographic],
[0x1F3CF, 0x1F3D3, extendedPictographic],
[0x1F3D4, 0x1F3DF, extendedPictographic],
[0x1F3E0, 0x1F3E3, extendedPictographic],
[0x1F3E4, 0x1F3E4, extendedPictographic],
[0x1F3E5, 0x1F3F0, extendedPictographic],
[0x1F3F1, 0x1F3F2, extendedPictographic],
[0x1F3F3, 0x1F3F3, extendedPictographic],
[0x1F3F4, 0x1F3F4, extendedPictographic],
[0x1F3F5, 0x1F3F5, extendedPictographic],
[0x1F3F6, 0x1F3F6, extendedPictographic],
[0x1F3F7, 0x1F3F7, extendedPictographic],
[0x1F3F8, 0x1F3FA, extendedPictographic],
[0x1F3FB, 0x1F3FF, extend],
[0x1F400, 0x1F407, extendedPictographic],
[0x1F408, 0x1F408, extendedPictographic],
[0x1F409, 0x1F40B, extendedPictographic],
[0x1F40C, 0x1F40E, extendedPictographic],
[0x1F40F, 0x1F410, extendedPictographic],
[0x1F411, 0x1F412, extendedPictographic],
[0x1F413, 0x1F413, extendedPictographic],
[0x1F414, 0x1F414, extendedPictographic],
[0x1F415, 0x1F415, extendedPictographic],
[0x1F416, 0x1F416, extendedPictographic],
[0x1F417, 0x1F429, extendedPictographic],
[0x1F42A, 0x1F42A, extendedPictographic],
[0x1F42B, 0x1F43E, extendedPictographic],
[0x1F43F, 0x1F43F, extendedPictographic],
[0x1F440, 0x1F440, extendedPictographic],
[0x1F441, 0x1F441, extendedPictographic],
[0x1F442, 0x1F464, extendedPictographic],
[0x1F465, 0x1F465, extendedPictographic],
[0x1F466, 0x1F46B, extendedPictographic],
[0x1F46C, 0x1F46D, extendedPictographic],
[0x1F46E, 0x1F4AC, extendedPictographic],
[0x1F4AD, 0x1F4AD, extendedPictographic],
[0x1F4AE, 0x1F4B5, extendedPictographic],
[0x1F4B6, 0x1F4B7, extendedPictographic],
[0x1F4B8, 0x1F4EB, extendedPictographic],
[0x1F4EC, 0x1F4ED, extendedPictographic],
[0x1F4EE, 0x1F4EE, extendedPictographic],
[0x1F4EF, 0x1F4EF, extendedPictographic],
[0x1F4F0, 0x1F4F4, extendedPictographic],
[0x1F4F5, 0x1F4F5, extendedPictographic],
[0x1F4F6, 0x1F4F7, extendedPictographic],
[0x1F4F8, 0x1F4F8, extendedPictographic],
[0x1F4F9, 0x1F4FC, extendedPictographic],
[0x1F4FD, 0x1F4FD, extendedPictographic],
[0x1F4FE, 0x1F4FE, extendedPictographic],
[0x1F4FF, 0x1F502, extendedPictographic],
[0x1F503, 0x1F503, extendedPictographic],
[0x1F504, 0x1F507, extendedPictographic],
[0x1F508, 0x1F508, extendedPictographic],
[0x1F509, 0x1F509, extendedPictographic],
[0x1F50A, 0x1F514, extendedPictographic],
[0x1F515, 0x1F515, extendedPictographic],
[0x1F516, 0x1F52B, extendedPictographic],
[0x1F52C, 0x1F52D, extendedPictographic],
[0x1F52E, 0x1F53D, extendedPictographic],
[0x1F546, 0x1F548, extendedPictographic],
[0x1F549, 0x1F54A, extendedPictographic],
[0x1F54B, 0x1F54E, extendedPictographic],
[0x1F54F, 0x1F54F, extendedPictographic],
[0x1F550, 0x1F55B, extendedPictographic],
[0x1F55C, 0x1F567, extendedPictographic],
[0x1F568, 0x1F56E, extendedPictographic],
[0x1F56F, 0x1F570, extendedPictographic],
[0x1F571, 0x1F572, extendedPictographic],
[0x1F573, 0x1F579, extendedPictographic],
[0x1F57A, 0x1F57A, extendedPictographic],
[0x1F57B, 0x1F586, extendedPictographic],
[0x1F587, 0x1F587, extendedPictographic],
[0x1F588, 0x1F589, extendedPictographic],
[0x1F58A, 0x1F58D, extendedPictographic],
[0x1F58E, 0x1F58F, extendedPictographic],
[0x1F590, 0x1F590, extendedPictographic],
[0x1F591, 0x1F594, extendedPictographic],
[0x1F595, 0x1F596, extendedPictographic],
[0x1F597, 0x1F5A3, extendedPictographic],
[0x1F5A4, 0x1F5A4, extendedPictographic],
[0x1F5A5, 0x1F5A5, extendedPictographic],
[0x1F5A6, 0x1F5A7, extendedPictographic],
[0x1F5A8, 0x1F5A8, extendedPictographic],
[0x1F5A9, 0x1F5B0, extendedPictographic],
[0x1F5B1, 0x1F5B2, extendedPictographic],
[0x1F5B3, 0x1F5BB, extendedPictographic],
[0x1F5BC, 0x1F5BC, extendedPictographic],
[0x1F5BD, 0x1F5C1, extendedPictographic],
[0x1F5C2, 0x1F5C4, extendedPictographic],
[0x1F5C5, 0x1F5D0, extendedPictographic],
[0x1F5D1, 0x1F5D3, extendedPictographic],
[0x1F5D4, 0x1F5DB, extendedPictographic],
[0x1F5DC, 0x1F5DE, extendedPictographic],
[0x1F5DF, 0x1F5E0, extendedPictographic],
[0x1F5E1, 0x1F5E1, extendedPictographic],
[0x1F5E2, 0x1F5E2, extendedPictographic],
[0x1F5E3, 0x1F5E3, extendedPictographic],
[0x1F5E4, 0x1F5E7, extendedPictographic],
[0x1F5E8, 0x1F5E8, extendedPictographic],
[0x1F5E9, 0x1F5EE, extendedPictographic],
[0x1F5EF, 0x1F5EF, extendedPictographic],
[0x1F5F0, 0x1F5F2, extendedPictographic],
[0x1F5F3, 0x1F5F3, extendedPictographic],
[0x1F5F4, 0x1F5F9, extendedPictographic],
[0x1F5FA, 0x1F5FA, extendedPictographic],
[0x1F5FB, 0x1F5FF, extendedPictographic],
[0x1F600, 0x1F600, extendedPictographic],
[0x1F601, 0x1F606, extendedPictographic],
[0x1F607, 0x1F608, extendedPictographic],
[0x1F609, 0x1F60D, extendedPictographic],
[0x1F60E, 0x1F60E, extendedPictographic],
[0x1F60F, 0x1F60F, extendedPictographic],
[0x1F610, 0x1F610, extendedPictographic],
[0x1F611, 0x1F611, extendedPictographic],
[0x1F612, 0x1F614, extendedPictographic],
[0x1F615, 0x1F615, extendedPictographic],
[0x1F616, 0x1F616, extendedPictographic],
[0x1F617, 0x1F617, extendedPictographic],
[0x1F618, 0x1F618, extendedPictographic],
[0x1F619, 0x1F619, extendedPictographic],
[0x1F61A, 0x1F61A, extendedPictographic],
[0x1F61B, 0x1F61B, extendedPictographic],
[0x1F61C, 0x1F61E, extendedPictographic],
[0x1F61F, 0x1F61F, extendedPictographic],
[0x1F620, 0x1F625, extendedPictographic],
[0x1F626, 0x1F627, extendedPictographic],
[0x1F628, 0x1F62B, extendedPictographic],
[0x1F62C, 0x1F62C, extendedPictographic],
[0x1F62D, 0x1F62D, extendedPictographic],
[0x1F62E, 0x1F62F, extendedPictographic],
[0x1F630, 0x1F633, extendedPictographic],
[0x1F634, 0x1F634, extendedPictographic],
[0x1F635, 0x1F635, extendedPictographic],
[0x1F636, 0x1F636, extendedPictographic],
[0x1F637, 0x1F640, extendedPictographic],
[0x1F641, 0x1F644, extendedPictographic],
[0x1F645, 0x1F64F, extendedPictographic],
[0x1F680, 0x1F680, extendedPictographic],
[0x1F681, 0x1F682, extendedPictographic],
[0x1F683, 0x1F685, extendedPictographic],
[0x1F686, 0x1F686, extendedPictographic],
[0x1F687, 0x1F687, extendedPictographic],
[0x1F688, 0x1F688, extendedPictographic],
[0x1F689, 0x1F689, extendedPictographic],
[0x1F68A, 0x1F68B, extendedPictographic],
[0x1F68C, 0x1F68C, extendedPictographic],
[0x1F68D, 0x1F68D, extendedPictographic],
[0x1F68E, 0x1F68E, extendedPictographic],
[0x1F68F, 0x1F68F, extendedPictographic],
[0x1F690, 0x1F690, extendedPictographic],
[0x1F691, 0x1F693, extendedPictographic],
[0x1F694, 0x1F694, extendedPictographic],
[0x1F695, 0x1F695, extendedPictographic],
[0x1F696, 0x1F696, extendedPictographic],
[0x1F697, 0x1F697, extendedPictographic],
[0x1F698, 0x1F698, extendedPictographic],
[0x1F699, 0x1F69A, extendedPictographic],
[0x1F69B, 0x1F6A1, extendedPictographic],
[0x1F6A2, 0x1F6A2, extendedPictographic],
[0x1F6A3, 0x1F6A3, extendedPictographic],
[0x1F6A4, 0x1F6A5, extendedPictographic],
[0x1F6A6, 0x1F6A6, extendedPictographic],
[0x1F6A7, 0x1F6AD, extendedPictographic],
[0x1F6AE, 0x1F6B1, extendedPictographic],
[0x1F6B2, 0x1F6B2, extendedPictographic],
[0x1F6B3, 0x1F6B5, extendedPictographic],
[0x1F6B6, 0x1F6B6, extendedPictographic],
[0x1F6B7, 0x1F6B8, extendedPictographic],
[0x1F6B9, 0x1F6BE, extendedPictographic],
[0x1F6BF, 0x1F6BF, extendedPictographic],
[0x1F6C0, 0x1F6C0, extendedPictographic],
[0x1F6C1, 0x1F6C5, extendedPictographic],
[0x1F6C6, 0x1F6CA, extendedPictographic],
[0x1F6CB, 0x1F6CB, extendedPictographic],
[0x1F6CC, 0x1F6CC, extendedPictographic],
[0x1F6CD, 0x1F6CF, extendedPictographic],
[0x1F6D0, 0x1F6D0, extendedPictographic],
[0x1F6D1, 0x1F6D2, extendedPictographic],
[0x1F6D3, 0x1F6D4, extendedPictographic],
[0x1F6D5, 0x1F6D5, extendedPictographic],
[0x1F6D6, 0x1F6D7, extendedPictographic],
[0x1F6D8, 0x1F6DB, extendedPictographic],
[0x1F6DC, 0x1F6DC, extendedPictographic],
[0x1F6DD, 0x1F6DF, extendedPictographic],
[0x1F6E0, 0x1F6E5, extendedPictographic],
[0x1F6E6, 0x1F6E8, extendedPictographic],
[0x1F6E9, 0x1F6E9, extendedPictographic],
[0x1F6EA, 0x1F6EA, extendedPictographic],
[0x1F6EB, 0x1F6EC, extendedPictographic],
[0x1F6ED, 0x1F6EF, extendedPictographic],
[0x1F6F0, 0x1F6F0, extendedPictographic],
[0x1F6F1, 0x1F6F2, extendedPictographic],
[0x1F6F3, 0x1F6F3, extendedPictographic],
[0x1F6F4, 0x1F6F6, extendedPictographic],
[0x1F6F7, 0x1F6F8, extendedPictographic],
[0x1F6F9, 0x1F6F9, extendedPictographic],
[0x1F6FA, 0x1F6FA, extendedPictographic],
[0x1F6FB, 0x1F6FC, extendedPictographic],
[0x1F6FD, 0x1F6FF, extendedPictographic],
[0x1F774, 0x1F77F, extendedPictographic],
[0x1F7D5, 0x1F7DF, extendedPictographic],
[0x1F7E0, 0x1F7EB, extendedPictographic],
[0x1F7EC, 0x1F7EF, extendedPictographic],
[0x1F7F0, 0x1F7F0, extendedPictographic],
[0x1F7F1, 0x1F7FF, extendedPictographic],
[0x1F80C, 0x1F80F, extendedPictographic],
[0x1F848, 0x1F84F, extendedPictographic],
[0x1F85A, 0x1F85F, extendedPictographic],
[0x1F888, 0x1F88F, extendedPictographic],
[0x1F8AE, 0x1F8FF, extendedPictographic],
[0x1F90C, 0x1F90C, extendedPictographic],
[0x1F90D, 0x1F90F, extendedPictographic],
[0x1F910, 0x1F918, extendedPictographic],
[0x1F919, 0x1F91E, extendedPictographic],
[0x1F91F, 0x1F91F, extendedPictographic],
[0x1F920, 0x1F927, extendedPictographic],
[0x1F928, 0x1F92F, extendedPictographic],
[0x1F930, 0x1F930, extendedPictographic],
[0x1F931, 0x1F932, extendedPictographic],
[0x1F933, 0x1F93A, extendedPictographic],
[0x1F93C, 0x1F93E, extendedPictographic],
[0x1F93F, 0x1F93F, extendedPictographic],
[0x1F940, 0x1F945, extendedPictographic],
[0x1F947, 0x1F94B, extendedPictographic],
[0x1F94C, 0x1F94C, extendedPictographic],
[0x1F94D, 0x1F94F, extendedPictographic],
[0x1F950, 0x1F95E, extendedPictographic],
[0x1F95F, 0x1F96B, extendedPictographic],
[0x1F96C, 0x1F970, extendedPictographic],
[0x1F971, 0x1F971, extendedPictographic],
[0x1F972, 0x1F972, extendedPictographic],
[0x1F973, 0x1F976, extendedPictographic],
[0x1F977, 0x1F978, extendedPictographic],
[0x1F979, 0x1F979, extendedPictographic],
[0x1F97A, 0x1F97A, extendedPictographic],
[0x1F97B, 0x1F97B, extendedPictographic],
[0x1F97C, 0x1F97F, extendedPictographic],
[0x1F980, 0x1F984, extendedPictographic],
[0x1F985, 0x1F991, extendedPictographic],
[0x1F992, 0x1F997, extendedPictographic],
[0x1F998, 0x1F9A2, extendedPictographic],
[0x1F9A3, 0x1F9A4, extendedPictographic],
[0x1F9A5, 0x1F9AA, extendedPictographic],
[0x1F9AB, 0x1F9AD, extendedPictographic],
[0x1F9AE, 0x1F9AF, extendedPictographic],
[0x1F9B0, 0x1F9B9, extendedPictographic],
[0x1F9BA, 0x1F9BF, extendedPictographic],
[0x1F9C0, 0x1F9C0, extendedPictographic],
[0x1F9C1, 0x1F9C2, extendedPictographic],
[0x1F9C3, 0x1F9CA, extendedPictographic],
[0x1F9CB, 0x1F9CB, extendedPictographic],
[0x1F9CC, 0x1F9CC, extendedPictographic],
[0x1F9CD, 0x1F9CF, extendedPictographic],
[0x1F9D0, 0x1F9E6, extendedPictographic],
[0x1F9E7, 0x1F9FF, extendedPictographic],
[0x1FA00, 0x1FA6F, extendedPictographic],
[0x1FA70, 0x1FA73, extendedPictographic],
[0x1FA74, 0x1FA74, extendedPictographic],
[0x1FA75, 0x1FA77, extendedPictographic],
[0x1FA78, 0x1FA7A, extendedPictographic],
[0x1FA7B, 0x1FA7C, extendedPictographic],
[0x1FA7D, 0x1FA7F, extendedPictographic],
[0x1FA80, 0x1FA82, extendedPictographic],
[0x1FA83, 0x1FA86, extendedPictographic],
[0x1FA87, 0x1FA88, extendedPictographic],
[0x1FA89, 0x1FA8F, extendedPictographic],
[0x1FA90, 0x1FA95, extendedPictographic],
[0x1FA96, 0x1FAA8, extendedPictographic],
[0x1FAA9, 0x1FAAC, extendedPictographic],
[0x1FAAD, 0x1FAAF, extendedPictographic],
[0x1FAB0, 0x1FAB6, extendedPictographic],
[0x1FAB7, 0x1FABA, extendedPictographic],
[0x1FABB, 0x1FABD, extendedPictographic],
[0x1FABE, 0x1FABE, extendedPictographic],
[0x1FABF, 0x1FABF, extendedPictographic],
[0x1FAC0, 0x1FAC2, extendedPictographic],
[0x1FAC3, 0x1FAC5, extendedPictographic],
[0x1FAC6, 0x1FACD, extendedPictographic],
[0x1FACE, 0x1FACF, extendedPictographic],
[0x1FAD0, 0x1FAD6, extendedPictographic],
[0x1FAD7, 0x1FAD9, extendedPictographic],
[0x1FADA, 0x1FADB, extendedPictographic],
[0x1FADC, 0x1FADF, extendedPictographic],
[0x1FAE0, 0x1FAE7, extendedPictographic],
[0x1FAE8, 0x1FAE8, extendedPictographic],
[0x1FAE9, 0x1FAEF, extendedPictographic],
[0x1FAF0, 0x1FAF6, extendedPictographic],
[0x1FAF7, 0x1FAF8, extendedPictographic],
[0x1FAF9, 0x1FAFF, extendedPictographic],
[0x1FC00, 0x1FFFD, extendedPictographic],
[0xE0000, 0xE0000, control],
[0xE0001, 0xE0001, control],
[0xE0002, 0xE001F, control],
[0xE0020, 0xE007F, extend],
[0xE0080, 0xE00FF, control],
[0xE0100, 0xE01EF, extend],
[0xE01F0, 0xE0FFF, control],
]
}
// Returns the unicode property of its codePoint argument using a binary search.
// If 'cp' isn't included in the above table, 'any' is returned.
static fromCP(cp) {
var from = 0
var to = __codePoints.count
while (to > from) {
var middle = ((from + to)/2).floor
var cpRange = __codePoints[middle]
if (cp < cpRange[0]) {
to = middle
} else if (cp > cpRange[1]) {
from = middle + 1
} else {
return cpRange[2]
}
}
return any
}
}
/*
Graphemes repesents a sequence of 'user-perceived characters' (or grapheme clusters) for
a given string.
*/
class Graphemes is Sequence {
// The states of the grapheme cluster parser.
static any { 0 }
static cr { 1 }
static controlLF { 2 }
static l { 3 }
static lvv { 4 }
static lvtt { 5 }
static prepend { 6 }
static extendedPictographic { 7 }
static extendedPictographicZWJ { 8 }
static riOdd { 9 }
static riEven { 10 }
// The grapheme cluster parser's breaking instructions.
static noBoundary { 0 }
static boundary { 1 }
// Private method to initialize static fields.
static init_() {
__np = 15 // number of propetry types (0 to 14)
// The grapheme cluster parser's state transitions. They map (state, property) to
// (new state, breaking instruction, rule number). The breaking instruction
// always refers to the boundary between the last and next codepoint.
// This map is queried as follows:
// 1. Find specific state + specific property. Stop if found.
// 2. Find specific state + any property.
// 3. Find any state + specific property.
// 4. If only (2) or (3) (but not both) was found, stop.
// 5. If both (2) and (3) were found, use state and breaking instruction from
// the transition with the lower rule number, prefer (3) if rule numbers
// are equal. Stop.
// 6. Assume 'any' and 'boundary'.
__transitions = {
// GB5
(any * __np + Properties.cr) : [cr, boundary, 50],
(any * __np + Properties.lf) : [controlLF, boundary, 50],
(any * __np + Properties.control) : [controlLF, boundary, 50],
// GB4
(cr * __np + Properties.any) : [any, boundary, 40],
(controlLF * __np + Properties.any) : [any, boundary, 40],
// GB3
(cr * __np + Properties.lf) : [controlLF, noBoundary, 30],
// GB6
(any * __np + Properties.l) : [l, boundary, 9990],
(l * __np + Properties.l) : [l, noBoundary, 60],
(l * __np + Properties.v) : [lvv, noBoundary, 60],
(l * __np + Properties.lv) : [lvv, noBoundary, 60],
(l * __np + Properties.lvt) : [lvtt, noBoundary, 60],
// GB7
(any * __np + Properties.lv) : [lvv, boundary, 9990],
(any * __np + Properties.v) : [lvv, boundary, 9990],
(lvv * __np + Properties.v) : [lvv, noBoundary, 70],
(lvv * __np + Properties.t) : [lvtt, noBoundary, 70],
// GB8
(any * __np + Properties.lvt) : [lvtt, boundary, 9990],
(any * __np + Properties.t) : [lvtt, boundary, 9990],
(lvtt * __np + Properties.t) : [lvtt, noBoundary, 80],
// GB9
(any * __np + Properties.extend) : [any, noBoundary, 90],
(any * __np + Properties.zwj) : [any, noBoundary, 90],
// GB9a
(any * __np + Properties.spacingMark) : [any, noBoundary, 91],
// GB9b
(any * __np + Properties.prepend) : [prepend, boundary, 9990],
(prepend * __np + Properties.any) : [any, noBoundary, 92],
// GB11
(any * __np + Properties.extendedPictographic) :
[extendedPictographic, boundary, 9990],
(extendedPictographic * __np + Properties.extend) :
[extendedPictographic, noBoundary, 110],
(extendedPictographic * __np + Properties.zwj) :
[extendedPictographicZWJ, noBoundary, 110],
(extendedPictographicZWJ * __np + Properties.extendedPictographic) :
[extendedPictographic, noBoundary, 110],
// GB12 / GB13
(any * __np + Properties.regionalIndicator) : [riOdd, boundary, 9990],
(riOdd * __np + Properties.regionalIndicator) : [riEven, noBoundary, 120],
(riEven * __np + Properties.regionalIndicator) : [riOdd, boundary, 120]
}
}
// Returns the number of grapheme clusters for the given string. This is found by
// iterating through the string and counting the clusters.
static clusterCount(s) {
var g = Graphemes.new(s)
var n = 0
while (g.iterate(true)) n = n + 1
return n
}
// Private helper method to return number of UTF8 bytes in a given codepoint.
static byteCount_(cp) {
if (cp < 0x80) return 1
if (cp < 0x800) return 2
if (cp < 0x10000) return 3
return 4
}
// Constructs a new Graphemes object from a string. There are six instance fields:
// 1. _codePoints stores the string's codePoints in order.
// 2. _indices stores the byte index corresponding to the start of each codePoint
// plus the byte length of the string.
// 3. _start stores the starting index of the 'current' grapheme cluster.
// 4. _end stores the final index + 1 of the 'current' grapheme cluster
// 5. _pos stores the index of the next codepoint to be parsed.
// 6. _state stores the current state of the codepoint parser.
// Note that if _start == _end, either the iteration has not begun or has already completed.
construct new(s) {
_codePoints = s.codePoints.toList
_indices = []
var index = 0
for (cp in _codePoints) {
_indices.add(index)
index = index + Graphemes.byteCount_(cp)
}
_indices.add(index)
_start = 0
_end = 0
_pos = 0
_state = Graphemes.any
iterate(true) // move the iterator to the first grapheme cluster.
}
// Advances the iterator by one grapheme cluster and returns false if no clusters are left.
// Must be called before the first cluster is accessed.
// The parameter 'iter' is not used but is needed to satisfy the iterator protocol.
iterate(iter) {
_start = _end
// The state transition gives us a boundary instruction BEFORE the next codepoint
// so we always need to stay ahead by one codepoint.
// Parse the next codepoint.
var cpCount = _codePoints.count
while (_pos <= cpCount) {
// GB2
if (_pos == cpCount) {
_end = _pos
_pos = _pos + 1
break
}
// Determine the property of the next codepoint.
var nextProp = Properties.fromCP(_codePoints[_pos])
_pos = _pos + 1
// Find the applicable transition.
var boundary = false
var trans = __transitions[_state * __np + nextProp]
if (trans) {
// We have a specific transition so use it.
_state = trans[0]
boundary = trans[1] == Graphemes.boundary
} else {
// No specific transition found so try the less specific ones.
var transAnyProp = __transitions[_state * __np + Properties.any]
var transAnyState = __transitions[Graphemes.any * __np + nextProp]
if (transAnyProp && transAnyState) {
// Both apply so we use a mix.
_state = transAnyState[0]
boundary = transAnyState[1] == Graphemes.boundary
if (transAnyProp[2] < transAnyState[2]) {
_state = transAnyProp[0]
boundary = transAnyProp[1] == Graphemes.boundary
}
} else if (transAnyProp) {
// We only have a specific state.
_state = transAnyProp[0]
boundary = transAnyProp[1] == Graphemes.boundary
} else if (transAnyState) {
// We only have a specific property.
_state = transAnyState[0]
boundary = transAnyState[1] == Graphemes.boundary
} else {
// No known transition. GB999: any x any.
_state = Graphemes.any
boundary = true
}
}
// If a cluster boundary was found, stop here. The current cluster will be
// the one that just ended.
if ((_pos - 1 == 0) /* GB1 */ || boundary) {
_end = _pos - 1
break
}
}
return _start != _end
}
// The value (in string form) of the 'current' grapheme cluster.
// The parameter 'iter' is not used but is needed to satisfy the iterator protocol.
iteratorValue(iter) { currentStr }
// Returns a list of codePoints corresponding to the 'current' grapheme cluster.
// Returns null if the iterator is already past the end or iterate() has not yet been called.
currentCluster {
if (_start == _end) return null
return _codePoints[_start..._end].toList
}
// Returns a list of bytes corresponding to the 'current' grapheme cluster.
// Returns null if the iterator is already past the end or iterate() has not yet been called.
currentBytes {
if (_start == _end) return null
return currentStr.bytes.toList
}
// Returns the 'current' grapheme cluster in string form i.e. as a sub-string of the
// original string. Returns an empty string if the iterator is already past the end or
// iterate() has not yet been called.
currentStr {
if (_start == _end) return ""
return currentCluster.map { |c| String.fromCodePoint(c) }.join()
}
// Returns a list of the starting and (ending + 1) indices of the 'current' grapheme cluster.
// If iterate() has not yet been called, both values are 0.
// If the iterator is already past the end, both values are 1.
currentSpan { [_indices[_start], _indices[_end]] }
// Puts the iterator into its initial state and moves to the first grapheme cluster again.
reset() {
_start = 0
_end = 0
_pos = 0
_state = Graphemes.any
iterate(true)
}
}
// Initialize static fields
Properties.init_()
Graphemes.init_()</syntaxhighlight>
| |||