Reduce letter folds to a fixed point in gen_folds.py
CI / ci (push) Successful in 1m32s
CI / ci (pull_request) Successful in 1m33s

A fold result could itself be foldable: ǿ folded to ø and ǽ to æ, while
the canonically equivalent ø+U+0301 and æ+U+0301 reached o and ae.
Normalize writes a result without folding it again, so the generator now
folds every result letter by letter until it settles (and fails if it does
not), making the committed table fully reduced. TestLetterFoldsFullyReduced
pins that, and the canonical-equivalence tests now cover ǿ and ǽ through
Normalize, Similar and Deduper.IsDuplicate.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-05 09:07:02 -04:00
co-authored by Claude Opus 5.5
parent 7729488cde
commit d2d5e9c742
3 changed files with 48 additions and 9 deletions
+6 -6
View File
@@ -248,8 +248,8 @@ var letterFolds = map[rune]string{
0x01DF: "a", // LATIN SMALL LETTER A WITH DIAERESIS AND MACRON
0x01E0: "a", // LATIN CAPITAL LETTER A WITH DOT ABOVE AND MACRON
0x01E1: "a", // LATIN SMALL LETTER A WITH DOT ABOVE AND MACRON
0x01E2: "\u00e6", // LATIN CAPITAL LETTER AE WITH MACRON
0x01E3: "\u00e6", // LATIN SMALL LETTER AE WITH MACRON
0x01E2: "ae", // LATIN CAPITAL LETTER AE WITH MACRON
0x01E3: "ae", // LATIN SMALL LETTER AE WITH MACRON
0x01E4: "g", // LATIN CAPITAL LETTER G WITH STROKE
0x01E5: "g", // LATIN SMALL LETTER G WITH STROKE
0x01E6: "g", // LATIN CAPITAL LETTER G WITH CARON
@@ -270,10 +270,10 @@ var letterFolds = map[rune]string{
0x01F9: "n", // LATIN SMALL LETTER N WITH GRAVE
0x01FA: "a", // LATIN CAPITAL LETTER A WITH RING ABOVE AND ACUTE
0x01FB: "a", // LATIN SMALL LETTER A WITH RING ABOVE AND ACUTE
0x01FC: "\u00e6", // LATIN CAPITAL LETTER AE WITH ACUTE
0x01FD: "\u00e6", // LATIN SMALL LETTER AE WITH ACUTE
0x01FE: "\u00f8", // LATIN CAPITAL LETTER O WITH STROKE AND ACUTE
0x01FF: "\u00f8", // LATIN SMALL LETTER O WITH STROKE AND ACUTE
0x01FC: "ae", // LATIN CAPITAL LETTER AE WITH ACUTE
0x01FD: "ae", // LATIN SMALL LETTER AE WITH ACUTE
0x01FE: "o", // LATIN CAPITAL LETTER O WITH STROKE AND ACUTE
0x01FF: "o", // LATIN SMALL LETTER O WITH STROKE AND ACUTE
0x0200: "a", // LATIN CAPITAL LETTER A WITH DOUBLE GRAVE
0x0201: "a", // LATIN SMALL LETTER A WITH DOUBLE GRAVE
0x0202: "a", // LATIN CAPITAL LETTER A WITH INVERTED BREVE