Fix review findings R1–R3 on Regular grounding and name folding
R1: Migration 0010 now writes revision 0 into every ground_dinner job payload that has none. Jobs queued before the upgrade carry only dinner_id, so the boot cleanup, which matches a pending dinner to its job by revision, failed those dinners as interrupted while their jobs were still queued. The migration runs before the job runner starts. A new test migrates a database to 0009, adds a pending Regular with a legacy queued job, upgrades and boots, and checks the dinner stays pending and is then grounded. R2: Archiving a dinner, or renaming a pending one, now cancels the grounding of any ground_dinner job running for it at an older revision, so no further search, page fetch, photo download or model call is made. The job is registered before it reads the dinner, so an archive or rename committed after that read always finds it. A stopped job succeeds and reports the dinner's current grounding status. The archive and rename tests now check that the job ends while its search is still held, with no page fetched and no model call for the stale name. R3: gen_folds.py writes web/src/dinners/letterFolds.json from the same table as folds_table.go, and names.ts folds through it instead of a hand-copied subset. names.ts also lowercases letter by letter, as Go's strings.ToLower does, rather than by context. TestLetterFoldsMatchPWA fails when the two tables differ. The shared cases add "Ƈhili", "Ꞔarbanzo Stew" and "ΜΟΥΣΑΚΑΣ" (final sigma). Maestro task 5037ff86-181d-448e-927b-c2fd2258a25d. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
File diff suppressed because it is too large
Load Diff
+15
-31
@@ -3,37 +3,17 @@
|
||||
// API would refuse a name as a duplicate. The server still decides.
|
||||
// internal/ai/harness/testdata/dedupe_cases.json is run against both.
|
||||
|
||||
import folds from './letterFolds.json'
|
||||
|
||||
// SIMILAR_THRESHOLD is harness.SimilarThreshold.
|
||||
const SIMILAR_THRESHOLD = 0.85
|
||||
|
||||
// fillerWords is harness's fillerWords.
|
||||
const fillerWords = new Set(['easy', 'best', 'homemade', 'quick', 'simple', 'the', 'a', 'recipe', 'classic'])
|
||||
|
||||
// letterFolds are the letters harness folds that canonical decomposition
|
||||
// leaves whole: its ligatures and extra letters, and the common stroked
|
||||
// Latin letters. Letters with accents fold by decomposing and dropping the
|
||||
// marks, as harness's table does; rarer stroked letters are compared as
|
||||
// they are.
|
||||
const letterFolds: Record<string, string> = {
|
||||
ß: 'ss',
|
||||
æ: 'ae',
|
||||
œ: 'oe',
|
||||
þ: 'th',
|
||||
ð: 'd',
|
||||
ı: 'i',
|
||||
ʼn: 'n',
|
||||
ij: 'ij',
|
||||
ø: 'o',
|
||||
đ: 'd',
|
||||
ħ: 'h',
|
||||
ŀ: 'l',
|
||||
ł: 'l',
|
||||
ŧ: 't',
|
||||
ƀ: 'b',
|
||||
ƶ: 'z',
|
||||
ǥ: 'g',
|
||||
ɨ: 'i',
|
||||
}
|
||||
// letterFolds is harness's letterFolds: gen_folds.py writes both tables,
|
||||
// and harness's TestLetterFoldsMatchPWA fails when they differ.
|
||||
const letterFolds: Record<string, string> = folds
|
||||
|
||||
const combiningMark = /\p{Mn}/u
|
||||
const wordChar = /[\p{L}\p{Nd}]/u
|
||||
@@ -44,12 +24,16 @@ const wordChar = /[\p{L}\p{Nd}]/u
|
||||
// "chicken tikka masala".
|
||||
export function normalizeName(name: string): string {
|
||||
let out = ''
|
||||
for (const ch of name.toLowerCase().normalize('NFD')) {
|
||||
if (combiningMark.test(ch) || ch === "'" || ch === '’') continue
|
||||
const fold = letterFolds[ch]
|
||||
if (fold !== undefined) out += fold
|
||||
else if (wordChar.test(ch)) out += ch
|
||||
else out += ' '
|
||||
// Lowercased letter by letter, as Go's strings.ToLower does, so no
|
||||
// letter is lowered by its context (a final Σ is σ, not ς).
|
||||
for (const letter of name) {
|
||||
for (const ch of letter.toLowerCase()) {
|
||||
if (combiningMark.test(ch) || ch === "'" || ch === '’') continue
|
||||
const fold = letterFolds[ch]
|
||||
if (fold !== undefined) out += fold
|
||||
else if (wordChar.test(ch)) out += ch
|
||||
else out += ' '
|
||||
}
|
||||
}
|
||||
return out
|
||||
.split(' ')
|
||||
|
||||
Reference in New Issue
Block a user