Name normalization and dedupe (#59)

Co-authored-by: dev_agent <dev_agent@cubecraftlabs.com>
2026-10-05 09:16:16 -04:00
parent 6ff6443db7
commit 5d5404b3cb
5 changed files with 1567 additions and 0 deletions
@@ -164,6 +164,15 @@ automatic** (D-021).
  last 60 passed Side quests and the plans of the last four ISO weeks. A
  character budget trims the oldest plans first, then the oldest passes;
  Regulars and the Quest log are never trimmed.
+- **Name dedupe** (`harness.Normalize`, `harness.Similar`, `harness.Deduper`):
+  a dinner name is normalized by lowercasing, dropping combining marks and
+  folding precomposed letters to their base letters through a table generated
+  from Unicode data (`gen_folds.py`), so composed and decomposed spellings
+  match, stripping punctuation, dropping filler words (easy, best, homemade, quick, simple,
+  the, a, recipe, classic) and collapsing whitespace. Two names are the same
+  dinner when the token-set (Jaccard) similarity of their words is at least
+  0.85. A `Deduper` over the Regulars, Quest log, pending suggestions and
+  passed names rejects a suggestion that repeats any of them.

 ## Recipe grounding