Integration fix: Feature 12 feature-review findings F1-F8 (#95)
CI / ci (pull_request) Successful in 2m23s
CI / ci (push) Successful in 2m27s

Co-authored-by: orchestrator-agent <20+orchestrator-agent@code.cubecraftcreations.com>
This commit was merged in pull request #95.
This commit is contained in:
2026-10-06 19:54:47 -04:00
committed by approver_agent
parent 171c9c2d11
commit cc7875c76c
26 changed files with 2402 additions and 257 deletions
+36 -11
View File
@@ -96,7 +96,7 @@ Columns ending in `_ct` hold AES-256-GCM ciphertext (via `secretbox`).
| `recipe_ingredients` | recipe_id, position, raw_text, name, qty, unit, note. | | `recipe_ingredients` | recipe_id, position, raw_text, name, qty, unit, note. |
| `recipe_steps` | recipe_id, position, text. | | `recipe_steps` | recipe_id, position, text. |
| `photos` | source_url, local_path, width, height, sha256, fetched_at. | | `photos` | source_url, local_path, width, height, sha256, fetched_at. |
| `dinners` | The meal record and its lifecycle: name, normalized_name, status (`suggested` / `quest_log` / `tried` / `regular` / `archived`), origin, method, minutes_total, difficulty, cuisine, is_favorite, recipe_id, recipe_url (a pasted recipe link awaiting grounding), photo_id, grounding_status (`pending` / `grounded` / `failed` / `manual`), grounding_error (why grounding failed), updated_by. | | `dinners` | The meal record and its lifecycle: name, normalized_name, status (`suggested` / `quest_log` / `tried` / `regular` / `archived`), origin, method, minutes_total, difficulty, cuisine, is_favorite, recipe_id, recipe_url (a pasted recipe link awaiting grounding), photo_id, grounding_status (`pending` / `grounded` / `failed` / `manual`), grounding_error (why grounding failed), grounding_revision (raised when a pending dinner is renamed, so only the latest `ground_dinner` job may write it), updated_by. |
| `suggestion_batches` | trigger, status, ai_run_id. | | `suggestion_batches` | trigger, status, ai_run_id. |
| `suggestions` | dinner_id, batch_id, reason, cited_regular_ids (JSON), status, decided_at. | | `suggestions` | dinner_id, batch_id, reason, cited_regular_ids (JSON), status, decided_at. |
| `week_plans` | week_start (unique), status (`draft` / `synced` / `changed`), ai_note, ai_run_id, calendar_synced_at. | | `week_plans` | week_start (unique), status (`draft` / `synced` / `changed`), ai_note, ai_run_id, calendar_synced_at. |
@@ -187,8 +187,10 @@ automatic** (D-021).
"Beef stew"); a link that names nothing is 400 — add it by name. A name "Beef stew"); a link that names nothing is 400 — add it by name. A name
is trimmed, at most 120 characters and must keep a word after is trimmed, at most 120 characters and must keep a word after
`harness.Normalize`, which sets `normalized_name`. A name `harness.Normalize`, which sets `normalized_name`. A name
`harness.Similar` to an unarchived dinner's, or a link one already has, `harness.Similar` to an unarchived dinner's, or a link one already has
is 409 `duplicate_dinner`; archived names may be added again. (the source_url of the recipe it is grounded in, else its pasted
`recipe_url`), is 409 `duplicate_dinner`; archived names may be added
again.
`GET /api/v1/dinners?status=regular|quest_log` returns `GET /api/v1/dinners?status=regular|quest_log` returns
`{dinners: [...]}` ordered by normalized name, each with `photo_url` `{dinners: [...]}` ordered by normalized name, each with `photo_url`
(`/media/photos/{photo_id}-400.jpg`, or null) and `grounding_status`; (`/media/photos/{photo_id}-400.jpg`, or null) and `grounding_status`;
@@ -211,11 +213,22 @@ automatic** (D-021).
`failed` with `grounding_error` — also the job's failure code — one of `failed` with `grounding_error` — also the job's failure code — one of
`no_recipe_found`, `recipe_unusable` (the pasted link), `search_not_configured`, `no_recipe_found`, `recipe_unusable` (the pasted link), `search_not_configured`,
`search_failed`, `ai_failed`, `interrupted` (the server stopped) or `search_failed`, `ai_failed`, `interrupted` (the server stopped) or
`grounding_error`. It only writes a dinner still `pending`, so a fix-up `grounding_error`. Each job carries the dinner's `grounding_revision`
made meanwhile wins. At boot, after the job runner fails the jobs a from when it was enqueued; renaming a `pending` dinner clears its
previous process left running, every dinner still `pending` with no `recipe_url`, raises the revision and enqueues a job for the new name.
`ground_dinner` job queued or running is set `failed` with `interrupted`, A job for a dinner archived, no longer `pending` or at another revision
so no dinner stays pending with nothing left to ground it. does no work, and a job only writes a dinner still unarchived, `pending`
and at its revision, so a fix-up made meanwhile wins and an earlier job
for an old name — queued or running — never decides the outcome.
Archiving or renaming a dinner also stops its running job where it is,
so no further search, page fetch, photo download or model call is made
for it. A
method, minutes_total or difficulty a member saved while the dinner was
`pending` is kept; grounding fills only those left empty. At boot, after
the job runner fails the jobs a previous process left running, every
unarchived dinner still `pending` with no `ground_dinner` job for its revision
queued or running is set `failed` with `interrupted`, so no dinner stays
pending with nothing left to ground it.
`GET /api/v1/dinners/{id}/recipe-candidates` `GET /api/v1/dinners/{id}/recipe-candidates`
searches for the dinner's name and returns `{candidates: [{candidate_id, searches for the dinner's name and returns `{candidates: [{candidate_id,
title, site_name, photo_url, minutes}]}`, at most 3, ranked as `Ground` title, site_name, photo_url, minutes}]}`, at most 3, ranked as `Ground`
@@ -229,8 +242,9 @@ automatic** (D-021).
— no model runs — as a `recipes` row whose source_url is — no model runs — as a `recipes` row whose source_url is
`mealquest:manual:{recipe_id}` and whose raw_jsonld is a schema.org `mealquest:manual:{recipe_id}` and whose raw_jsonld is a schema.org
Recipe built from it, and sets `manual` with its method and minutes, no Recipe built from it, and sets `manual` with its method and minutes, no
photo and the dinner's difficulty unchanged. It answers 200 with the photo and the dinner's difficulty unchanged. Either clears the dinner's
dinner; an invalid body is 400, an unknown dinner 404, a page that cannot pasted `recipe_url`: the recipe chosen replaces it. It answers 200 with
the dinner; an invalid body is 400, an unknown dinner 404, a page that cannot
ground it 422 `ungroundable` (the dinner is unchanged), recipe search not ground it 422 `ungroundable` (the dinner is unchanged), recipe search not
set up 409 `not_configured`, a search or model failure 502 set up 409 `not_configured`, a search or model failure 502
`search_failed` / `ai_failed` (only the reason code is logged, never the `search_failed` / `ai_failed` (only the reason code is logged, never the
@@ -488,13 +502,24 @@ First run: while `household_settings.onboarding_completed_at` (read from
`GET /api/v1/settings/household`) is null, every route redirects to `GET /api/v1/settings/household`) is null, every route redirects to
`/onboarding`. Onboarding is a full screen outside the app shell, in three `/onboarding`. Onboarding is a full screen outside the app shell, in three
steps: the welcome (`/onboarding`), the Regulars (`/onboarding/regulars`, steps: the welcome (`/onboarding`), the Regulars (`/onboarding/regulars`,
mock 1h, adding and archiving through the dinners API optimistically) and mock 1h, adding and archiving through the dinners API optimistically; Next
waits for three saved Regulars, and a common pick is not offered once a
Regular has a name `harness.Similar` to it — the PWA's copy of the name
rules in `web/src/dinners/names.ts` folds letters with
`web/src/dinners/letterFolds.json`, which `gen_folds.py` writes from the
same table as the server's, and runs the same cases,
`internal/ai/harness/testdata/dedupe_cases.json`, as the server's) and
notifications (`/onboarding/notifications`: what MealQuest notifies about, notifications (`/onboarding/notifications`: what MealQuest notifies about,
the iOS install hint, "Turn on notifications" and "Maybe later"). Either the iOS install hint, "Turn on notifications" and "Maybe later"). Either
button sends `PATCH /api/v1/settings/household` with button sends `PATCH /api/v1/settings/household` with
`{"onboarding_completed": true}`; the server sets `onboarding_completed_at` `{"onboarding_completed": true}`; the server sets `onboarding_completed_at`
to the time of that update (keeping the first time if it is already set; to the time of that update (keeping the first time if it is already set;
the field itself is read-only and `false` is refused) and the app goes Home. the field itself is read-only and `false` is refused) and the app goes Home.
Completing needs at least three Regulars (`household.MinRegulars`); with
fewer the server answers 400 `invalid_settings` naming
`onboarding_completed` and changes nothing. Step 3 can be reached with
fewer — by its URL, or after another member removes a Regular — so it then
shows the server's reason with an "Add Regulars" button back to step 2.
Notification permission is never asked for on page load; until Feature 19 Notification permission is never asked for on page load; until Feature 19
wires Web Push, "Turn on notifications" asks for nothing. Offline on the saved pages the wires Web Push, "Turn on notifications" asks for nothing. Offline on the saved pages the
settings cannot be read (the service worker never caches `/api/v1/settings`), settings cannot be read (the service worker never caches `/api/v1/settings`),
+60
View File
@@ -1,10 +1,70 @@
package harness package harness
import ( import (
"encoding/json"
"os"
"testing" "testing"
"unicode" "unicode"
"unicode/utf8"
) )
// TestSharedDedupeCases runs testdata/dedupe_cases.json, which the PWA's
// copy of Normalize and Similar (web/src/dinners/names.ts) runs too, so the
// two cannot drift apart on these names.
func TestSharedDedupeCases(t *testing.T) {
raw, err := os.ReadFile("testdata/dedupe_cases.json")
if err != nil {
t.Fatal(err)
}
var cases struct {
Normalize []struct{ Name, Want string }
Similar []struct {
A, B string
Want bool
}
}
if err := json.Unmarshal(raw, &cases); err != nil {
t.Fatal(err)
}
if len(cases.Normalize) == 0 || len(cases.Similar) == 0 {
t.Fatal("dedupe_cases.json has no cases")
}
for _, c := range cases.Normalize {
if got := Normalize(c.Name); got != c.Want {
t.Errorf("Normalize(%q) = %q, want %q", c.Name, got, c.Want)
}
}
for _, c := range cases.Similar {
if got := Similar(c.A, c.B); got != c.Want {
t.Errorf("Similar(%q, %q) = %v, want %v", c.A, c.B, got, c.Want)
}
}
}
// TestLetterFoldsMatchPWA checks that the PWA's letter-fold table, which
// gen_folds.py writes beside folds_table.go, is letterFolds exactly; run
// go generate when it fails.
func TestLetterFoldsMatchPWA(t *testing.T) {
raw, err := os.ReadFile("../../../web/src/dinners/letterFolds.json")
if err != nil {
t.Fatal(err)
}
var pwa map[string]string
if err := json.Unmarshal(raw, &pwa); err != nil {
t.Fatal(err)
}
for r, want := range letterFolds {
if got, ok := pwa[string(r)]; !ok || got != want {
t.Errorf("PWA folds %U to %q (present %v), want %q", r, got, ok, want)
}
}
for k := range pwa {
if utf8.RuneCountInString(k) != 1 || letterFolds[[]rune(k)[0]] == "" {
t.Errorf("PWA folds %q, which letterFolds does not", k)
}
}
}
func TestNormalize(t *testing.T) { func TestNormalize(t *testing.T) {
tests := []struct { tests := []struct {
name, want string name, want string
+10 -1
View File
@@ -1,5 +1,8 @@
#!/usr/bin/env python3 #!/usr/bin/env python3
"""Generate folds_table.go, the letter-fold table used by harness.Normalize. """Generate folds_table.go, the letter-fold table used by harness.Normalize,
and its copy web/src/dinners/letterFolds.json, used by the PWA's
normalizeName (web/src/dinners/names.ts). TestLetterFoldsMatchPWA fails when
the two differ.
Run from this directory with `go generate ./internal/ai/harness` (or Run from this directory with `go generate ./internal/ai/harness` (or
`python3 gen_folds.py`; gofmt must be on PATH). It uses only the Python `python3 gen_folds.py`; gofmt must be on PATH). It uses only the Python
@@ -28,11 +31,14 @@ Hangul syllables are left out: their decomposition is into conjoining jamo,
not combining marks. not combining marks.
""" """
import json
import re import re
import subprocess import subprocess
import sys import sys
import unicodedata import unicodedata
PWA_TABLE = "../../../web/src/dinners/letterFolds.json"
EXTRA = { EXTRA = {
"ß": "ss", "ẞ": "ss", "æ": "ae", "Æ": "ae", "œ": "oe", "Œ": "oe", "ß": "ss", "ẞ": "ss", "æ": "ae", "Æ": "ae", "œ": "oe", "Œ": "oe",
"þ": "th", "Þ": "th", "ð": "d", "Ð": "d", "ı": "i", "ʼn": "n", "þ": "th", "Þ": "th", "ð": "d", "Ð": "d", "ı": "i", "ʼn": "n",
@@ -110,6 +116,9 @@ def main():
with open("folds_table.go", "w", encoding="utf-8") as f: with open("folds_table.go", "w", encoding="utf-8") as f:
f.write("\n".join(out) + "\n") f.write("\n".join(out) + "\n")
subprocess.run(["gofmt", "-w", "folds_table.go"], check=True) subprocess.run(["gofmt", "-w", "folds_table.go"], check=True)
with open(PWA_TABLE, "w", encoding="ascii") as f:
json.dump({chr(cp): folded for cp, folded in entries}, f, indent=2)
f.write("\n")
if __name__ == "__main__": if __name__ == "__main__":
+136
View File
@@ -0,0 +1,136 @@
{
"normalize": [
{
"name": "Easy Chicken Tikka-Masala!",
"want": "chicken tikka masala"
},
{
"name": " Homemade Chili ",
"want": "chili"
},
{
"name": "Grandma's Stew",
"want": "grandmas stew"
},
{
"name": "Grandma’s Stew",
"want": "grandmas stew"
},
{
"name": "İskender Kebab",
"want": "iskender kebab"
},
{
"name": "Paella à la Valenciana",
"want": "paella la valenciana"
},
{
"name": "Crème Brûlée",
"want": "creme brulee"
},
{
"name": "Crème Brûlée",
"want": "creme brulee"
},
{
"name": "Smørrebrød",
"want": "smorrebrod"
},
{
"name": "Pierogi z Łososiem",
"want": "pierogi z lososiem"
},
{
"name": "Straße Würstchen",
"want": "strasse wurstchen"
},
{
"name": "Œufs en Cocotte",
"want": "oeufs en cocotte"
},
{
"name": "Æbleskiver",
"want": "aebleskiver"
},
{
"name": "Phở Bò",
"want": "pho bo"
},
{
"name": "The Best Quick Recipe",
"want": ""
},
{
"name": "Chicken stir-fry",
"want": "chicken stir fry"
},
{
"name": "7-layer dip",
"want": "7 layer dip"
},
{
"name": "Ƈhili",
"want": "chili"
},
{
"name": "Ꞔarbanzo Stew",
"want": "carbanzo stew"
},
{
"name": "ΜΟΥΣΑΚΑΣ",
"want": "μουσακασ"
}
],
"similar": [
{
"a": "Homemade chili",
"b": "Chili",
"want": true
},
{
"a": "Easy Spaghetti Bolognese",
"b": "spaghetti bolognese",
"want": true
},
{
"a": "Chicken stir fry",
"b": "Chicken stir-fry",
"want": true
},
{
"a": "Crème brûlée",
"b": "Creme brulee",
"want": true
},
{
"a": "Smørrebrød",
"b": "smorrebrod",
"want": true
},
{
"a": "Beef tacos",
"b": "Fish tacos",
"want": false
},
{
"a": "Chili",
"b": "White chicken chili",
"want": false
},
{
"a": "Pot roast",
"b": "Roast",
"want": false
},
{
"a": "The best",
"b": "Easy recipe",
"want": false
},
{
"a": "Ƈhili",
"b": "Chili",
"want": true
}
]
}
+2 -1
View File
@@ -190,6 +190,7 @@ func TestDinners(t *testing.T) {
Difficulty: str("medium"), Difficulty: str("medium"),
Cuisine: sql.NullString{}, Cuisine: sql.NullString{},
RecipeID: sql.NullString{}, RecipeID: sql.NullString{},
RecipeUrl: str("https://example.com/tacos"),
PhotoID: str(photo.ID), PhotoID: str(photo.ID),
GroundingStatus: "failed", GroundingStatus: "failed",
UpdatedBy: str("sam@example.com"), UpdatedBy: str("sam@example.com"),
@@ -267,7 +268,7 @@ func TestDinners(t *testing.T) {
} }
var unarchivedIDs []string var unarchivedIDs []string
for _, d := range unarchived { for _, d := range unarchived {
unarchivedIDs = append(unarchivedIDs, d.ID) unarchivedIDs = append(unarchivedIDs, d.Dinner.ID)
} }
assertIDs(t, "unarchived", unarchivedIDs, other.ID, third.ID) assertIDs(t, "unarchived", unarchivedIDs, other.ID, third.ID)
_, err = q.ArchiveDinner(ctx, db.ArchiveDinnerParams{ID: "missing", UpdatedAt: archivedAt}) _, err = q.ArchiveDinner(ctx, db.ArchiveDinnerParams{ID: "missing", UpdatedAt: archivedAt})
+119 -81
View File
@@ -16,7 +16,7 @@ SET status = 'archived',
updated_by = ?, updated_by = ?,
updated_at = ? updated_at = ?
WHERE id = ? WHERE id = ?
RETURNING id, name, normalized_name, status, origin, method, minutes_total, difficulty, cuisine, is_favorite, recipe_id, photo_id, grounding_status, updated_by, created_at, updated_at, recipe_url, grounding_error RETURNING id, name, normalized_name, status, origin, method, minutes_total, difficulty, cuisine, is_favorite, recipe_id, photo_id, grounding_status, updated_by, created_at, updated_at, recipe_url, grounding_error, grounding_revision
` `
type ArchiveDinnerParams struct { type ArchiveDinnerParams struct {
@@ -48,6 +48,7 @@ func (q *Queries) ArchiveDinner(ctx context.Context, arg ArchiveDinnerParams) (D
&i.UpdatedAt, &i.UpdatedAt,
&i.RecipeUrl, &i.RecipeUrl,
&i.GroundingError, &i.GroundingError,
&i.GroundingRevision,
) )
return i, err return i, err
} }
@@ -75,7 +76,7 @@ INSERT INTO dinners (
?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?,
?, ?, ?, ? ?, ?, ?, ?
) )
RETURNING id, name, normalized_name, status, origin, method, minutes_total, difficulty, cuisine, is_favorite, recipe_id, photo_id, grounding_status, updated_by, created_at, updated_at, recipe_url, grounding_error RETURNING id, name, normalized_name, status, origin, method, minutes_total, difficulty, cuisine, is_favorite, recipe_id, photo_id, grounding_status, updated_by, created_at, updated_at, recipe_url, grounding_error, grounding_revision
` `
type CreateDinnerParams struct { type CreateDinnerParams struct {
@@ -138,6 +139,7 @@ func (q *Queries) CreateDinner(ctx context.Context, arg CreateDinnerParams) (Din
&i.UpdatedAt, &i.UpdatedAt,
&i.RecipeUrl, &i.RecipeUrl,
&i.GroundingError, &i.GroundingError,
&i.GroundingRevision,
) )
return i, err return i, err
} }
@@ -148,11 +150,13 @@ SET grounding_status = 'failed',
grounding_error = ?1, grounding_error = ?1,
updated_at = ?2 updated_at = ?2
WHERE grounding_status = 'pending' WHERE grounding_status = 'pending'
AND status <> 'archived'
AND NOT EXISTS ( AND NOT EXISTS (
SELECT 1 FROM jobs SELECT 1 FROM jobs
WHERE jobs.kind = ?3 WHERE jobs.kind = ?3
AND jobs.status IN ('queued', 'running') AND jobs.status IN ('queued', 'running')
AND json_extract(jobs.payload, '$.dinner_id') = dinners.id AND json_extract(jobs.payload, '$.dinner_id') = dinners.id
AND json_extract(jobs.payload, '$.revision') = dinners.grounding_revision
) )
` `
@@ -162,10 +166,11 @@ type FailOrphanedPendingDinnerGroundingsParams struct {
Kind string Kind string
} }
// Fails every dinner still pending whose grounding job is no longer queued // Fails every unarchived dinner still pending whose grounding job for its
// or running: the process stopped before the job, or before its handler // current grounding_revision is no longer queued or running: the process
// recorded a result. Run at boot, after the job runner fails the jobs a // stopped before the job, or before its handler recorded a result. An
// previous process left running. // archived dinner is left as it is; archiving stopped its grounding. Run at
// boot, after the job runner fails the jobs a previous process left running.
func (q *Queries) FailOrphanedPendingDinnerGroundings(ctx context.Context, arg FailOrphanedPendingDinnerGroundingsParams) (int64, error) { func (q *Queries) FailOrphanedPendingDinnerGroundings(ctx context.Context, arg FailOrphanedPendingDinnerGroundingsParams) (int64, error) {
result, err := q.db.ExecContext(ctx, failOrphanedPendingDinnerGroundings, arg.GroundingError, arg.UpdatedAt, arg.Kind) result, err := q.db.ExecContext(ctx, failOrphanedPendingDinnerGroundings, arg.GroundingError, arg.UpdatedAt, arg.Kind)
if err != nil { if err != nil {
@@ -179,25 +184,29 @@ UPDATE dinners
SET grounding_status = 'failed', SET grounding_status = 'failed',
grounding_error = ?, grounding_error = ?,
updated_at = ? updated_at = ?
WHERE id = ? AND grounding_status = 'pending' AND normalized_name = ? WHERE id = ?
AND status <> 'archived'
AND grounding_status = 'pending'
AND grounding_revision = ?
` `
type FailPendingDinnerGroundingParams struct { type FailPendingDinnerGroundingParams struct {
GroundingError sql.NullString GroundingError sql.NullString
UpdatedAt string UpdatedAt string
ID string ID string
NormalizedName string GroundingRevision int64
} }
// Records why the background grounding of a dinner still pending under the // Records why the background grounding of an unarchived dinner still
// normalized name it was grounded for failed; a dinner grounded some other // pending at the grounding_revision it was grounded for failed; a dinner
// way or renamed meanwhile is left as it is (0 rows). // grounded some other way, renamed or archived meanwhile is left as it is
// (0 rows).
func (q *Queries) FailPendingDinnerGrounding(ctx context.Context, arg FailPendingDinnerGroundingParams) (int64, error) { func (q *Queries) FailPendingDinnerGrounding(ctx context.Context, arg FailPendingDinnerGroundingParams) (int64, error) {
result, err := q.db.ExecContext(ctx, failPendingDinnerGrounding, result, err := q.db.ExecContext(ctx, failPendingDinnerGrounding,
arg.GroundingError, arg.GroundingError,
arg.UpdatedAt, arg.UpdatedAt,
arg.ID, arg.ID,
arg.NormalizedName, arg.GroundingRevision,
) )
if err != nil { if err != nil {
return 0, err return 0, err
@@ -206,7 +215,7 @@ func (q *Queries) FailPendingDinnerGrounding(ctx context.Context, arg FailPendin
} }
const getDinner = `-- name: GetDinner :one const getDinner = `-- name: GetDinner :one
SELECT id, name, normalized_name, status, origin, method, minutes_total, difficulty, cuisine, is_favorite, recipe_id, photo_id, grounding_status, updated_by, created_at, updated_at, recipe_url, grounding_error FROM dinners SELECT id, name, normalized_name, status, origin, method, minutes_total, difficulty, cuisine, is_favorite, recipe_id, photo_id, grounding_status, updated_by, created_at, updated_at, recipe_url, grounding_error, grounding_revision FROM dinners
WHERE id = ? WHERE id = ?
` `
@@ -232,6 +241,7 @@ func (q *Queries) GetDinner(ctx context.Context, id string) (Dinner, error) {
&i.UpdatedAt, &i.UpdatedAt,
&i.RecipeUrl, &i.RecipeUrl,
&i.GroundingError, &i.GroundingError,
&i.GroundingRevision,
) )
return i, err return i, err
} }
@@ -239,6 +249,7 @@ func (q *Queries) GetDinner(ctx context.Context, id string) (Dinner, error) {
const groundDinner = `-- name: GroundDinner :one const groundDinner = `-- name: GroundDinner :one
UPDATE dinners UPDATE dinners
SET recipe_id = ?, SET recipe_id = ?,
recipe_url = NULL,
photo_id = ?, photo_id = ?,
method = ?, method = ?,
minutes_total = ?, minutes_total = ?,
@@ -248,7 +259,7 @@ SET recipe_id = ?,
updated_by = ?, updated_by = ?,
updated_at = ? updated_at = ?
WHERE id = ? WHERE id = ?
RETURNING id, name, normalized_name, status, origin, method, minutes_total, difficulty, cuisine, is_favorite, recipe_id, photo_id, grounding_status, updated_by, created_at, updated_at, recipe_url, grounding_error RETURNING id, name, normalized_name, status, origin, method, minutes_total, difficulty, cuisine, is_favorite, recipe_id, photo_id, grounding_status, updated_by, created_at, updated_at, recipe_url, grounding_error, grounding_revision
` `
type GroundDinnerParams struct { type GroundDinnerParams struct {
@@ -263,8 +274,9 @@ type GroundDinnerParams struct {
ID string ID string
} }
// Records a dinner's grounding (grounded or manual) and clears any // Records a dinner's grounding by hand (grounded or manual) and clears any
// grounding_error. An unknown id returns sql.ErrNoRows. // grounding_error and pasted recipe_url: the recipe chosen replaces the
// link. An unknown id returns sql.ErrNoRows.
func (q *Queries) GroundDinner(ctx context.Context, arg GroundDinnerParams) (Dinner, error) { func (q *Queries) GroundDinner(ctx context.Context, arg GroundDinnerParams) (Dinner, error) {
row := q.db.QueryRowContext(ctx, groundDinner, row := q.db.QueryRowContext(ctx, groundDinner,
arg.RecipeID, arg.RecipeID,
@@ -297,37 +309,43 @@ func (q *Queries) GroundDinner(ctx context.Context, arg GroundDinnerParams) (Din
&i.UpdatedAt, &i.UpdatedAt,
&i.RecipeUrl, &i.RecipeUrl,
&i.GroundingError, &i.GroundingError,
&i.GroundingRevision,
) )
return i, err return i, err
} }
const groundPendingDinner = `-- name: GroundPendingDinner :execrows const groundPendingDinner = `-- name: GroundPendingDinner :execrows
UPDATE dinners UPDATE dinners
SET recipe_id = ?, SET recipe_id = ?1,
photo_id = ?, photo_id = ?2,
method = ?, method = COALESCE(method, ?3),
minutes_total = ?, minutes_total = COALESCE(minutes_total, ?4),
difficulty = ?, difficulty = COALESCE(difficulty, ?5),
grounding_status = 'grounded', grounding_status = 'grounded',
grounding_error = NULL, grounding_error = NULL,
updated_at = ? updated_at = ?6
WHERE id = ? AND grounding_status = 'pending' AND normalized_name = ? WHERE id = ?7
AND status <> 'archived'
AND grounding_status = 'pending'
AND grounding_revision = ?8
` `
type GroundPendingDinnerParams struct { type GroundPendingDinnerParams struct {
RecipeID sql.NullString RecipeID sql.NullString
PhotoID sql.NullString PhotoID sql.NullString
Method sql.NullString Method sql.NullString
MinutesTotal sql.NullInt64 MinutesTotal sql.NullInt64
Difficulty sql.NullString Difficulty sql.NullString
UpdatedAt string UpdatedAt string
ID string ID string
NormalizedName string GroundingRevision int64
} }
// Records the background grounding of a dinner still pending under the // Records the background grounding of an unarchived dinner still pending at
// normalized name it was grounded for; a dinner grounded some other way or // the grounding_revision it was grounded for; a dinner grounded some other
// renamed meanwhile is left as it is (0 rows). // way, renamed or archived meanwhile is left as it is (0 rows). A method,
// minutes_total or difficulty already set is kept: a pending dinner has
// them only when a member edited them while it was grounding.
func (q *Queries) GroundPendingDinner(ctx context.Context, arg GroundPendingDinnerParams) (int64, error) { func (q *Queries) GroundPendingDinner(ctx context.Context, arg GroundPendingDinnerParams) (int64, error) {
result, err := q.db.ExecContext(ctx, groundPendingDinner, result, err := q.db.ExecContext(ctx, groundPendingDinner,
arg.RecipeID, arg.RecipeID,
@@ -337,7 +355,7 @@ func (q *Queries) GroundPendingDinner(ctx context.Context, arg GroundPendingDinn
arg.Difficulty, arg.Difficulty,
arg.UpdatedAt, arg.UpdatedAt,
arg.ID, arg.ID,
arg.NormalizedName, arg.GroundingRevision,
) )
if err != nil { if err != nil {
return 0, err return 0, err
@@ -346,7 +364,7 @@ func (q *Queries) GroundPendingDinner(ctx context.Context, arg GroundPendingDinn
} }
const listDinnersByStatus = `-- name: ListDinnersByStatus :many const listDinnersByStatus = `-- name: ListDinnersByStatus :many
SELECT id, name, normalized_name, status, origin, method, minutes_total, difficulty, cuisine, is_favorite, recipe_id, photo_id, grounding_status, updated_by, created_at, updated_at, recipe_url, grounding_error FROM dinners SELECT id, name, normalized_name, status, origin, method, minutes_total, difficulty, cuisine, is_favorite, recipe_id, photo_id, grounding_status, updated_by, created_at, updated_at, recipe_url, grounding_error, grounding_revision FROM dinners
WHERE status = ? WHERE status = ?
ORDER BY normalized_name, id ORDER BY normalized_name, id
` `
@@ -379,6 +397,7 @@ func (q *Queries) ListDinnersByStatus(ctx context.Context, status string) ([]Din
&i.UpdatedAt, &i.UpdatedAt,
&i.RecipeUrl, &i.RecipeUrl,
&i.GroundingError, &i.GroundingError,
&i.GroundingRevision,
); err != nil { ); err != nil {
return nil, err return nil, err
} }
@@ -394,40 +413,50 @@ func (q *Queries) ListDinnersByStatus(ctx context.Context, status string) ([]Din
} }
const listUnarchivedDinners = `-- name: ListUnarchivedDinners :many const listUnarchivedDinners = `-- name: ListUnarchivedDinners :many
SELECT id, name, normalized_name, status, origin, method, minutes_total, difficulty, cuisine, is_favorite, recipe_id, photo_id, grounding_status, updated_by, created_at, updated_at, recipe_url, grounding_error FROM dinners SELECT dinners.id, dinners.name, dinners.normalized_name, dinners.status, dinners.origin, dinners.method, dinners.minutes_total, dinners.difficulty, dinners.cuisine, dinners.is_favorite, dinners.recipe_id, dinners.photo_id, dinners.grounding_status, dinners.updated_by, dinners.created_at, dinners.updated_at, dinners.recipe_url, dinners.grounding_error, dinners.grounding_revision, recipes.source_url AS recipe_source_url
WHERE status <> 'archived' FROM dinners
ORDER BY id LEFT JOIN recipes ON recipes.id = dinners.recipe_id
WHERE dinners.status <> 'archived'
ORDER BY dinners.id
` `
// Every dinner a new or renamed dinner must not repeat. type ListUnarchivedDinnersRow struct {
func (q *Queries) ListUnarchivedDinners(ctx context.Context) ([]Dinner, error) { Dinner Dinner
RecipeSourceUrl sql.NullString
}
// Every dinner a new or renamed dinner must not repeat, with the
// source_url of the recipe it is grounded in (NULL while it has none).
func (q *Queries) ListUnarchivedDinners(ctx context.Context) ([]ListUnarchivedDinnersRow, error) {
rows, err := q.db.QueryContext(ctx, listUnarchivedDinners) rows, err := q.db.QueryContext(ctx, listUnarchivedDinners)
if err != nil { if err != nil {
return nil, err return nil, err
} }
defer rows.Close() defer rows.Close()
var items []Dinner var items []ListUnarchivedDinnersRow
for rows.Next() { for rows.Next() {
var i Dinner var i ListUnarchivedDinnersRow
if err := rows.Scan( if err := rows.Scan(
&i.ID, &i.Dinner.ID,
&i.Name, &i.Dinner.Name,
&i.NormalizedName, &i.Dinner.NormalizedName,
&i.Status, &i.Dinner.Status,
&i.Origin, &i.Dinner.Origin,
&i.Method, &i.Dinner.Method,
&i.MinutesTotal, &i.Dinner.MinutesTotal,
&i.Difficulty, &i.Dinner.Difficulty,
&i.Cuisine, &i.Dinner.Cuisine,
&i.IsFavorite, &i.Dinner.IsFavorite,
&i.RecipeID, &i.Dinner.RecipeID,
&i.PhotoID, &i.Dinner.PhotoID,
&i.GroundingStatus, &i.Dinner.GroundingStatus,
&i.UpdatedBy, &i.Dinner.UpdatedBy,
&i.CreatedAt, &i.Dinner.CreatedAt,
&i.UpdatedAt, &i.Dinner.UpdatedAt,
&i.RecipeUrl, &i.Dinner.RecipeUrl,
&i.GroundingError, &i.Dinner.GroundingError,
&i.Dinner.GroundingRevision,
&i.RecipeSourceUrl,
); err != nil { ); err != nil {
return nil, err return nil, err
} }
@@ -448,7 +477,7 @@ SET is_favorite = ?,
updated_by = ?, updated_by = ?,
updated_at = ? updated_at = ?
WHERE id = ? WHERE id = ?
RETURNING id, name, normalized_name, status, origin, method, minutes_total, difficulty, cuisine, is_favorite, recipe_id, photo_id, grounding_status, updated_by, created_at, updated_at, recipe_url, grounding_error RETURNING id, name, normalized_name, status, origin, method, minutes_total, difficulty, cuisine, is_favorite, recipe_id, photo_id, grounding_status, updated_by, created_at, updated_at, recipe_url, grounding_error, grounding_revision
` `
type SetDinnerFavoriteParams struct { type SetDinnerFavoriteParams struct {
@@ -486,6 +515,7 @@ func (q *Queries) SetDinnerFavorite(ctx context.Context, arg SetDinnerFavoritePa
&i.UpdatedAt, &i.UpdatedAt,
&i.RecipeUrl, &i.RecipeUrl,
&i.GroundingError, &i.GroundingError,
&i.GroundingRevision,
) )
return i, err return i, err
} }
@@ -496,7 +526,7 @@ SET is_favorite = 1 - is_favorite,
updated_by = ?, updated_by = ?,
updated_at = ? updated_at = ?
WHERE id = ? WHERE id = ?
RETURNING id, name, normalized_name, status, origin, method, minutes_total, difficulty, cuisine, is_favorite, recipe_id, photo_id, grounding_status, updated_by, created_at, updated_at, recipe_url, grounding_error RETURNING id, name, normalized_name, status, origin, method, minutes_total, difficulty, cuisine, is_favorite, recipe_id, photo_id, grounding_status, updated_by, created_at, updated_at, recipe_url, grounding_error, grounding_revision
` `
type ToggleDinnerFavoriteParams struct { type ToggleDinnerFavoriteParams struct {
@@ -528,6 +558,7 @@ func (q *Queries) ToggleDinnerFavorite(ctx context.Context, arg ToggleDinnerFavo
&i.UpdatedAt, &i.UpdatedAt,
&i.RecipeUrl, &i.RecipeUrl,
&i.GroundingError, &i.GroundingError,
&i.GroundingRevision,
) )
return i, err return i, err
} }
@@ -543,29 +574,33 @@ SET name = ?,
difficulty = ?, difficulty = ?,
cuisine = ?, cuisine = ?,
recipe_id = ?, recipe_id = ?,
recipe_url = ?,
photo_id = ?, photo_id = ?,
grounding_status = ?, grounding_status = ?,
grounding_revision = ?,
updated_by = ?, updated_by = ?,
updated_at = ? updated_at = ?
WHERE id = ? WHERE id = ?
RETURNING id, name, normalized_name, status, origin, method, minutes_total, difficulty, cuisine, is_favorite, recipe_id, photo_id, grounding_status, updated_by, created_at, updated_at, recipe_url, grounding_error RETURNING id, name, normalized_name, status, origin, method, minutes_total, difficulty, cuisine, is_favorite, recipe_id, photo_id, grounding_status, updated_by, created_at, updated_at, recipe_url, grounding_error, grounding_revision
` `
type UpdateDinnerParams struct { type UpdateDinnerParams struct {
Name string Name string
NormalizedName string NormalizedName string
Status string Status string
Origin string Origin string
Method sql.NullString Method sql.NullString
MinutesTotal sql.NullInt64 MinutesTotal sql.NullInt64
Difficulty sql.NullString Difficulty sql.NullString
Cuisine sql.NullString Cuisine sql.NullString
RecipeID sql.NullString RecipeID sql.NullString
PhotoID sql.NullString RecipeUrl sql.NullString
GroundingStatus string PhotoID sql.NullString
UpdatedBy sql.NullString GroundingStatus string
UpdatedAt string GroundingRevision int64
ID string UpdatedBy sql.NullString
UpdatedAt string
ID string
} }
// An unknown id returns sql.ErrNoRows. // An unknown id returns sql.ErrNoRows.
@@ -580,8 +615,10 @@ func (q *Queries) UpdateDinner(ctx context.Context, arg UpdateDinnerParams) (Din
arg.Difficulty, arg.Difficulty,
arg.Cuisine, arg.Cuisine,
arg.RecipeID, arg.RecipeID,
arg.RecipeUrl,
arg.PhotoID, arg.PhotoID,
arg.GroundingStatus, arg.GroundingStatus,
arg.GroundingRevision,
arg.UpdatedBy, arg.UpdatedBy,
arg.UpdatedAt, arg.UpdatedAt,
arg.ID, arg.ID,
@@ -606,6 +643,7 @@ func (q *Queries) UpdateDinner(ctx context.Context, arg UpdateDinnerParams) (Din
&i.UpdatedAt, &i.UpdatedAt,
&i.RecipeUrl, &i.RecipeUrl,
&i.GroundingError, &i.GroundingError,
&i.GroundingRevision,
) )
return i, err return i, err
} }
+19 -18
View File
@@ -51,24 +51,25 @@ type ChatSetting struct {
} }
type Dinner struct { type Dinner struct {
ID string ID string
Name string Name string
NormalizedName string NormalizedName string
Status string Status string
Origin string Origin string
Method sql.NullString Method sql.NullString
MinutesTotal sql.NullInt64 MinutesTotal sql.NullInt64
Difficulty sql.NullString Difficulty sql.NullString
Cuisine sql.NullString Cuisine sql.NullString
IsFavorite int64 IsFavorite int64
RecipeID sql.NullString RecipeID sql.NullString
PhotoID sql.NullString PhotoID sql.NullString
GroundingStatus string GroundingStatus string
UpdatedBy sql.NullString UpdatedBy sql.NullString
CreatedAt string CreatedAt string
UpdatedAt string UpdatedAt string
RecipeUrl sql.NullString RecipeUrl sql.NullString
GroundingError sql.NullString GroundingError sql.NullString
GroundingRevision int64
} }
type HouseholdSetting struct { type HouseholdSetting struct {
+20 -14
View File
@@ -36,14 +36,16 @@ type Querier interface {
DeleteRecipeIngredients(ctx context.Context, recipeID string) error DeleteRecipeIngredients(ctx context.Context, recipeID string) error
DeleteRecipeSteps(ctx context.Context, recipeID string) error DeleteRecipeSteps(ctx context.Context, recipeID string) error
FailJob(ctx context.Context, arg FailJobParams) (int64, error) FailJob(ctx context.Context, arg FailJobParams) (int64, error)
// Fails every dinner still pending whose grounding job is no longer queued // Fails every unarchived dinner still pending whose grounding job for its
// or running: the process stopped before the job, or before its handler // current grounding_revision is no longer queued or running: the process
// recorded a result. Run at boot, after the job runner fails the jobs a // stopped before the job, or before its handler recorded a result. An
// previous process left running. // archived dinner is left as it is; archiving stopped its grounding. Run at
// boot, after the job runner fails the jobs a previous process left running.
FailOrphanedPendingDinnerGroundings(ctx context.Context, arg FailOrphanedPendingDinnerGroundingsParams) (int64, error) FailOrphanedPendingDinnerGroundings(ctx context.Context, arg FailOrphanedPendingDinnerGroundingsParams) (int64, error)
// Records why the background grounding of a dinner still pending under the // Records why the background grounding of an unarchived dinner still
// normalized name it was grounded for failed; a dinner grounded some other // pending at the grounding_revision it was grounded for failed; a dinner
// way or renamed meanwhile is left as it is (0 rows). // grounded some other way, renamed or archived meanwhile is left as it is
// (0 rows).
FailPendingDinnerGrounding(ctx context.Context, arg FailPendingDinnerGroundingParams) (int64, error) FailPendingDinnerGrounding(ctx context.Context, arg FailPendingDinnerGroundingParams) (int64, error)
// Fails every job left running, at boot before any worker starts. // Fails every job left running, at boot before any worker starts.
FailRunningJobs(ctx context.Context, arg FailRunningJobsParams) (int64, error) FailRunningJobs(ctx context.Context, arg FailRunningJobsParams) (int64, error)
@@ -65,12 +67,15 @@ type Querier interface {
GetRecipeBySourceURL(ctx context.Context, sourceUrl string) (Recipe, error) GetRecipeBySourceURL(ctx context.Context, sourceUrl string) (Recipe, error)
// Returns sql.ErrNoRows until the row exists. // Returns sql.ErrNoRows until the row exists.
GetRecipeSearchSettings(ctx context.Context) (RecipeSearchSetting, error) GetRecipeSearchSettings(ctx context.Context) (RecipeSearchSetting, error)
// Records a dinner's grounding (grounded or manual) and clears any // Records a dinner's grounding by hand (grounded or manual) and clears any
// grounding_error. An unknown id returns sql.ErrNoRows. // grounding_error and pasted recipe_url: the recipe chosen replaces the
// link. An unknown id returns sql.ErrNoRows.
GroundDinner(ctx context.Context, arg GroundDinnerParams) (Dinner, error) GroundDinner(ctx context.Context, arg GroundDinnerParams) (Dinner, error)
// Records the background grounding of a dinner still pending under the // Records the background grounding of an unarchived dinner still pending at
// normalized name it was grounded for; a dinner grounded some other way or // the grounding_revision it was grounded for; a dinner grounded some other
// renamed meanwhile is left as it is (0 rows). // way, renamed or archived meanwhile is left as it is (0 rows). A method,
// minutes_total or difficulty already set is kept: a pending dinner has
// them only when a member edited them while it was grounding.
GroundPendingDinner(ctx context.Context, arg GroundPendingDinnerParams) (int64, error) GroundPendingDinner(ctx context.Context, arg GroundPendingDinnerParams) (int64, error)
ListDinnersByStatus(ctx context.Context, status string) ([]Dinner, error) ListDinnersByStatus(ctx context.Context, status string) ([]Dinner, error)
// The planned dinners dated from from_date up to but not including // The planned dinners dated from from_date up to but not including
@@ -81,8 +86,9 @@ type Querier interface {
ListRecentDeclinedNames(ctx context.Context, limit int64) ([]string, error) ListRecentDeclinedNames(ctx context.Context, limit int64) ([]string, error)
ListRecipeIngredients(ctx context.Context, recipeID string) ([]RecipeIngredient, error) ListRecipeIngredients(ctx context.Context, recipeID string) ([]RecipeIngredient, error)
ListRecipeSteps(ctx context.Context, recipeID string) ([]RecipeStep, error) ListRecipeSteps(ctx context.Context, recipeID string) ([]RecipeStep, error)
// Every dinner a new or renamed dinner must not repeat. // Every dinner a new or renamed dinner must not repeat, with the
ListUnarchivedDinners(ctx context.Context) ([]Dinner, error) // source_url of the recipe it is grounded in (NULL while it has none).
ListUnarchivedDinners(ctx context.Context) ([]ListUnarchivedDinnersRow, error)
// Stores a partial result of a running job. // Stores a partial result of a running job.
PublishJobResult(ctx context.Context, arg PublishJobResultParams) (int64, error) PublishJobResult(ctx context.Context, arg PublishJobResultParams) (int64, error)
// An unknown id returns sql.ErrNoRows. // An unknown id returns sql.ErrNoRows.
+71 -46
View File
@@ -11,9 +11,11 @@ import (
"errors" "errors"
"fmt" "fmt"
"log/slog" "log/slog"
"math"
"net/url" "net/url"
"slices" "slices"
"strings" "strings"
"sync"
"unicode" "unicode"
"unicode/utf8" "unicode/utf8"
@@ -130,6 +132,11 @@ type Service struct {
grounder *recipes.Grounder grounder *recipes.Grounder
logger *slog.Logger logger *slog.Logger
groundJob *jobs.Kind[GroundJob] groundJob *jobs.Kind[GroundJob]
// groundingsMu guards groundings: the KindGroundDinner jobs running,
// by dinner id, so archiving or renaming a dinner can stop them.
groundingsMu sync.Mutex
groundings map[string]map[*runningGrounding]struct{}
} }
// NewService returns a service over st that grounds dinners with grounder // NewService returns a service over st that grounds dinners with grounder
@@ -146,7 +153,7 @@ func NewService(st *store.Store, grounder *recipes.Grounder, runner *jobs.Runner
case logger == nil: case logger == nil:
return nil, errors.New("dinners: nil logger") return nil, errors.New("dinners: nil logger")
} }
s := &Service{st: st, grounder: grounder, logger: logger} s := &Service{st: st, grounder: grounder, logger: logger, groundings: make(map[string]map[*runningGrounding]struct{})}
kind, err := jobs.Register(runner, KindGroundDinner, s.runGroundJob, Reasons...) kind, err := jobs.Register(runner, KindGroundDinner, s.runGroundJob, Reasons...)
if err != nil { if err != nil {
return nil, err return nil, err
@@ -176,7 +183,7 @@ func (s *Service) CreateRegular(ctx context.Context, r NewRegular, updatedBy str
if err != nil { if err != nil {
return Created{}, err return Created{}, err
} }
jobID, err := s.enqueueGrounding(ctx, d.ID, harness.Normalize(d.Name)) jobID, err := s.enqueueGrounding(ctx, d.ID, d.GroundingRevision)
if err != nil { if err != nil {
return Created{}, err return Created{}, err
} }
@@ -187,60 +194,59 @@ func (s *Service) CreateRegular(ctx context.Context, r NewRegular, updatedBy str
} }
return Created{Dinner: fromRow(row)}, nil return Created{Dinner: fromRow(row)}, nil
} }
return Created{Dinner: d, GroundingJobID: jobID}, nil return Created{Dinner: fromRow(d), GroundingJobID: jobID}, nil
} }
// enqueueGrounding enqueues the KindGroundDinner job for pending dinner id, // enqueueGrounding enqueues the KindGroundDinner job for pending dinner id
// whose normalized name is normalizedName, and returns its id. When the job // at grounding_revision revision and returns its id. When the job cannot be
// cannot be enqueued, the dinner's grounding is failed with // enqueued, the dinner's grounding is failed with ReasonGroundingError, so
// ReasonGroundingError, so it is not left pending with no job, and the id // it is not left pending with no job, and the id is nil.
// is nil. func (s *Service) enqueueGrounding(ctx context.Context, id string, revision int64) (*string, error) {
func (s *Service) enqueueGrounding(ctx context.Context, id, normalizedName string) (*string, error) { jobID, err := s.groundJob.Enqueue(ctx, GroundJob{DinnerID: id, Revision: revision})
jobID, err := s.groundJob.Enqueue(ctx, GroundJob{DinnerID: id})
if err == nil { if err == nil {
return &jobID, nil return &jobID, nil
} }
// The error is a database error, never a dependency's text. // The error is a database error, never a dependency's text.
s.logger.ErrorContext(ctx, "dinner grounding not enqueued", "dinner_id", id, "error", err) s.logger.ErrorContext(ctx, "dinner grounding not enqueued", "dinner_id", id, "error", err)
if _, err := db.New(s.st.Writer).FailPendingDinnerGrounding(context.WithoutCancel(ctx), db.FailPendingDinnerGroundingParams{ if _, err := db.New(s.st.Writer).FailPendingDinnerGrounding(context.WithoutCancel(ctx), db.FailPendingDinnerGroundingParams{
GroundingError: sql.NullString{String: ReasonGroundingError, Valid: true}, GroundingError: sql.NullString{String: ReasonGroundingError, Valid: true},
UpdatedAt: store.Now().String(), UpdatedAt: store.Now().String(),
ID: id, ID: id,
NormalizedName: normalizedName, GroundingRevision: revision,
}); err != nil { }); err != nil {
return nil, fmt.Errorf("dinners: record failed grounding of %s: %w", id, err) return nil, fmt.Errorf("dinners: record failed grounding of %s: %w", id, err)
} }
return nil, nil return nil, nil
} }
func (s *Service) createRegular(ctx context.Context, r NewRegular, updatedBy string) (Dinner, error) { func (s *Service) createRegular(ctx context.Context, r NewRegular, updatedBy string) (db.Dinner, error) {
if updatedBy == "" { if updatedBy == "" {
return Dinner{}, errors.New("dinners: create: no member to record as updated_by") return db.Dinner{}, errors.New("dinners: create: no member to record as updated_by")
} }
var name string var name string
var recipeURL sql.NullString var recipeURL sql.NullString
switch { switch {
case r.Name != nil && r.RecipeURL != nil: case r.Name != nil && r.RecipeURL != nil:
return Dinner{}, invalid("name", "give a name or a recipe_url, not both") return db.Dinner{}, invalid("name", "give a name or a recipe_url, not both")
case r.Name != nil: case r.Name != nil:
n, err := checkName(*r.Name) n, err := checkName(*r.Name)
if err != nil { if err != nil {
return Dinner{}, err return db.Dinner{}, err
} }
name = n name = n
case r.RecipeURL != nil: case r.RecipeURL != nil:
u, err := checkRecipeURL(*r.RecipeURL) u, err := checkRecipeURL(*r.RecipeURL)
if err != nil { if err != nil {
return Dinner{}, err return db.Dinner{}, err
} }
n, ok := nameFromURL(u) n, ok := nameFromURL(u)
if !ok { if !ok {
return Dinner{}, invalid("recipe_url", "the link does not name a dinner; add it by name instead") return db.Dinner{}, invalid("recipe_url", "the link does not name a dinner; add it by name instead")
} }
name = n name = n
recipeURL = sql.NullString{String: u.String(), Valid: true} recipeURL = sql.NullString{String: u.String(), Valid: true}
default: default:
return Dinner{}, invalid("name", "give a name or a recipe_url") return db.Dinner{}, invalid("name", "give a name or a recipe_url")
} }
var row db.Dinner var row db.Dinner
@@ -269,9 +275,9 @@ func (s *Service) createRegular(ctx context.Context, r NewRegular, updatedBy str
return nil return nil
}) })
if err != nil { if err != nil {
return Dinner{}, err return db.Dinner{}, err
} }
return fromRow(row), nil return row, nil
} }
// List returns the dinners with status, which must be regular or // List returns the dinners with status, which must be regular or
@@ -294,8 +300,10 @@ func (s *Service) List(ctx context.Context, status string) ([]Dinner, error) {
// Update applies p to dinner id, recording updatedBy. An invalid patch is a // Update applies p to dinner id, recording updatedBy. An invalid patch is a
// *ValidationError and a new name repeating another unarchived dinner a // *ValidationError and a new name repeating another unarchived dinner a
// *DuplicateError; either changes nothing. Renaming a dinner still pending // *DuplicateError; either changes nothing. Renaming a dinner still pending
// enqueues a fresh KindGroundDinner job for the new name: the job already // clears its recipe_url, raises its grounding_revision and enqueues a fresh
// running was for the old name, and its result no longer applies. // KindGroundDinner job that grounds the new name: a job queued or running
// for the old name or link carries the old revision, so it can no longer
// write the dinner, and one running is stopped.
func (s *Service) Update(ctx context.Context, id string, p Patch, updatedBy string) (Dinner, error) { func (s *Service) Update(ctx context.Context, id string, p Patch, updatedBy string) (Dinner, error) {
if updatedBy == "" { if updatedBy == "" {
return Dinner{}, errors.New("dinners: update: no member to record as updated_by") return Dinner{}, errors.New("dinners: update: no member to record as updated_by")
@@ -317,20 +325,22 @@ func (s *Service) Update(ctx context.Context, id string, p Patch, updatedBy stri
return fmt.Errorf("dinners: read %s: %w", id, err) return fmt.Errorf("dinners: read %s: %w", id, err)
} }
params := db.UpdateDinnerParams{ params := db.UpdateDinnerParams{
ID: cur.ID, ID: cur.ID,
Name: cur.Name, Name: cur.Name,
NormalizedName: cur.NormalizedName, NormalizedName: cur.NormalizedName,
Status: cur.Status, Status: cur.Status,
Origin: cur.Origin, Origin: cur.Origin,
Method: cur.Method, Method: cur.Method,
MinutesTotal: cur.MinutesTotal, MinutesTotal: cur.MinutesTotal,
Difficulty: cur.Difficulty, Difficulty: cur.Difficulty,
Cuisine: cur.Cuisine, Cuisine: cur.Cuisine,
RecipeID: cur.RecipeID, RecipeID: cur.RecipeID,
PhotoID: cur.PhotoID, RecipeUrl: cur.RecipeUrl,
GroundingStatus: cur.GroundingStatus, PhotoID: cur.PhotoID,
UpdatedBy: sql.NullString{String: updatedBy, Valid: true}, GroundingStatus: cur.GroundingStatus,
UpdatedAt: store.Now().String(), GroundingRevision: cur.GroundingRevision,
UpdatedBy: sql.NullString{String: updatedBy, Valid: true},
UpdatedAt: store.Now().String(),
} }
if p.Name != nil { if p.Name != nil {
if err := checkUnique(ctx, q, cur.ID, name, sql.NullString{}); err != nil { if err := checkUnique(ctx, q, cur.ID, name, sql.NullString{}); err != nil {
@@ -338,6 +348,12 @@ func (s *Service) Update(ctx context.Context, id string, p Patch, updatedBy stri
} }
params.Name = name params.Name = name
params.NormalizedName = harness.Normalize(name) params.NormalizedName = harness.Normalize(name)
if cur.GroundingStatus == GroundingPending && params.NormalizedName != cur.NormalizedName {
// The link was the old dinner's recipe; the new name is
// grounded by name, and a link can be given by hand.
params.RecipeUrl = sql.NullString{}
params.GroundingRevision++
}
} }
if p.Method != nil { if p.Method != nil {
params.Method = sql.NullString{String: *p.Method, Valid: true} params.Method = sql.NullString{String: *p.Method, Valid: true}
@@ -352,14 +368,15 @@ func (s *Service) Update(ctx context.Context, id string, p Patch, updatedBy stri
if err != nil { if err != nil {
return fmt.Errorf("dinners: update %s: %w", id, err) return fmt.Errorf("dinners: update %s: %w", id, err)
} }
regrounding = row.GroundingStatus == GroundingPending && row.NormalizedName != cur.NormalizedName regrounding = row.GroundingRevision != cur.GroundingRevision
return nil return nil
}) })
if err != nil { if err != nil {
return Dinner{}, err return Dinner{}, err
} }
if regrounding { if regrounding {
jobID, err := s.enqueueGrounding(ctx, row.ID, row.NormalizedName) s.stopGroundings(row.ID, row.GroundingRevision)
jobID, err := s.enqueueGrounding(ctx, row.ID, row.GroundingRevision)
if err != nil { if err != nil {
return Dinner{}, err return Dinner{}, err
} }
@@ -374,7 +391,8 @@ func (s *Service) Update(ctx context.Context, id string, p Patch, updatedBy stri
} }
// Archive sets dinner id's status to archived, recording updatedBy. The row // Archive sets dinner id's status to archived, recording updatedBy. The row
// is kept, so its name still counts for Side quest dedupe. // is kept, so its name still counts for Side quest dedupe. A
// KindGroundDinner job running for it is stopped.
func (s *Service) Archive(ctx context.Context, id, updatedBy string) (Dinner, error) { func (s *Service) Archive(ctx context.Context, id, updatedBy string) (Dinner, error) {
if updatedBy == "" { if updatedBy == "" {
return Dinner{}, errors.New("dinners: archive: no member to record as updated_by") return Dinner{}, errors.New("dinners: archive: no member to record as updated_by")
@@ -390,6 +408,7 @@ func (s *Service) Archive(ctx context.Context, id, updatedBy string) (Dinner, er
if err != nil { if err != nil {
return Dinner{}, fmt.Errorf("dinners: archive %s: %w", id, err) return Dinner{}, fmt.Errorf("dinners: archive %s: %w", id, err)
} }
s.stopGroundings(row.ID, math.MaxInt64)
return fromRow(row), nil return fromRow(row), nil
} }
@@ -507,18 +526,24 @@ func nameFromURL(u *url.URL) (name string, ok bool) {
} }
// checkUnique returns a *DuplicateError when name is harness.Similar to an // checkUnique returns a *DuplicateError when name is harness.Similar to an
// unarchived dinner other than self, or recipeURL is one's recipe link. // unarchived dinner other than self, or recipeURL is one's recipe link: the
// source_url of the recipe it is grounded in, else the recipe_url still
// awaiting grounding.
func checkUnique(ctx context.Context, q *db.Queries, self, name string, recipeURL sql.NullString) error { func checkUnique(ctx context.Context, q *db.Queries, self, name string, recipeURL sql.NullString) error {
rows, err := q.ListUnarchivedDinners(ctx) rows, err := q.ListUnarchivedDinners(ctx)
if err != nil { if err != nil {
return fmt.Errorf("dinners: list for dedupe: %w", err) return fmt.Errorf("dinners: list for dedupe: %w", err)
} }
for _, row := range rows { for _, row := range rows {
if row.ID == self { if row.Dinner.ID == self {
continue continue
} }
if harness.Similar(name, row.Name) || (recipeURL.Valid && row.RecipeUrl == recipeURL) { link := row.Dinner.RecipeUrl
return &DuplicateError{Existing: fromRow(row)} if row.RecipeSourceUrl.Valid {
link = row.RecipeSourceUrl
}
if harness.Similar(name, row.Dinner.Name) || (recipeURL.Valid && link == recipeURL) {
return &DuplicateError{Existing: fromRow(row.Dinner)}
} }
} }
return nil return nil
+9
View File
@@ -1,6 +1,7 @@
package dinners package dinners
import ( import (
"context"
"testing" "testing"
"time" "time"
) )
@@ -12,3 +13,11 @@ func SetGroundTimeout(t *testing.T, d time.Duration) {
groundTimeout = d groundTimeout = d
t.Cleanup(func() { groundTimeout = old }) t.Cleanup(func() { groundTimeout = old })
} }
// RunGroundJob runs the KindGroundDinner handler for p in the caller's
// goroutine, as the job runner would, and returns the grounding status it
// reports.
func (s *Service) RunGroundJob(ctx context.Context, p GroundJob) (string, error) {
r, err := s.runGroundJob(ctx, p, nil)
return r.GroundingStatus, err
}
+101 -33
View File
@@ -76,14 +76,17 @@ var groundTimeout = GroundTimeout
// runs past GroundTimeout; the dinner is left as it was. // runs past GroundTimeout; the dinner is left as it was.
var ErrGroundTimeout = errors.New("dinners: grounding the recipe took too long") var ErrGroundTimeout = errors.New("dinners: grounding the recipe took too long")
// GroundJob is the KindGroundDinner payload. // GroundJob is the KindGroundDinner payload: the dinner to ground and its
// grounding_revision when the job was enqueued.
type GroundJob struct { type GroundJob struct {
DinnerID string `json:"dinner_id"` DinnerID string `json:"dinner_id"`
Revision int64 `json:"revision"`
} }
// groundResult is the KindGroundDinner result: the dinner's grounding // groundResult is the KindGroundDinner result: the dinner's grounding
// status once the job ends. It is not grounded when the dinner was // status once the job ends. It is not grounded by this job when the dinner
// grounded by hand before the job finished. // was grounded by hand before the job finished, renamed (a later job
// grounds it) or archived; the job then reports the status it found.
type groundResult struct { type groundResult struct {
DinnerID string `json:"dinner_id"` DinnerID string `json:"dinner_id"`
GroundingStatus string `json:"grounding_status"` GroundingStatus string `json:"grounding_status"`
@@ -93,26 +96,38 @@ func (r groundResult) PublicJSON() ([]byte, error) { return json.Marshal(r) }
// runGroundJob grounds a pending dinner as its creation asked: its // runGroundJob grounds a pending dinner as its creation asked: its
// recipe_url with GroundURL, else its name with Ground. Success records // recipe_url with GroundURL, else its name with Ground. Success records
// the recipe, photo, method, minutes and difficulty with grounding_status // the recipe and photo with grounding_status grounded, and the recipe's
// grounded; failure records grounding_status failed with its Reason, which // method, minutes and difficulty where a member has not set them; failure
// is also the job's failure code. Either is recorded only while the dinner // records grounding_status failed with its Reason, which is also the job's
// is still pending under the normalized name read here: a dinner grounded // failure code. A dinner archived, no longer pending, or at a
// by hand meanwhile is left as it is, and one renamed meanwhile is left // grounding_revision other than p.Revision is not grounded at all, and
// pending for the job its rename enqueued. // either result is recorded only while the dinner is still unarchived,
// pending and at p.Revision: a dinner grounded by hand meanwhile is left as
// it is, and one renamed or archived meanwhile is left pending — for the
// job its rename enqueued, or for none. Archiving or renaming the dinner
// stops the grounding where it is, so no further search, fetch, photo
// download or model call is made for it.
func (s *Service) runGroundJob(ctx context.Context, p GroundJob, _ func(groundResult) error) (groundResult, error) { func (s *Service) runGroundJob(ctx context.Context, p GroundJob, _ func(groundResult) error) (groundResult, error) {
// Tracked before the dinner is read: an archive or rename committed
// after the read finds this job to stop.
groundCtx, done := s.trackGrounding(ctx, p.DinnerID, p.Revision)
defer done()
row, err := db.New(s.st.Reader).GetDinner(ctx, p.DinnerID) row, err := db.New(s.st.Reader).GetDinner(ctx, p.DinnerID)
if err != nil { if err != nil {
return groundResult{}, fmt.Errorf("dinners: ground: read %s: %w", p.DinnerID, err) return groundResult{}, fmt.Errorf("dinners: ground: read %s: %w", p.DinnerID, err)
} }
if row.GroundingStatus != GroundingPending { if row.Status == StatusArchived || row.GroundingStatus != GroundingPending || row.GroundingRevision != p.Revision {
return groundResult{DinnerID: row.ID, GroundingStatus: row.GroundingStatus}, nil return groundResult{DinnerID: row.ID, GroundingStatus: row.GroundingStatus}, nil
} }
var gr recipes.Grounded var gr recipes.Grounded
if row.RecipeUrl.Valid { if row.RecipeUrl.Valid {
gr, err = s.grounder.GroundURL(ctx, row.RecipeUrl.String) gr, err = s.grounder.GroundURL(groundCtx, row.RecipeUrl.String)
} else { } else {
gr, err = s.grounder.Ground(ctx, row.Name, row.Name) gr, err = s.grounder.Ground(groundCtx, row.Name, row.Name)
}
if err != nil && ctx.Err() == nil && errors.Is(context.Cause(groundCtx), errGroundingStopped) {
return s.currentGrounding(ctx, row.ID)
} }
if err != nil { if err != nil {
reason := reasonFor(err, row.RecipeUrl.Valid) reason := reasonFor(err, row.RecipeUrl.Valid)
@@ -124,10 +139,10 @@ func (s *Service) runGroundJob(ctx context.Context, p GroundJob, _ func(groundRe
s.logger.InfoContext(ctx, "dinner grounding failed", "dinner_id", row.ID, "reason", reason) s.logger.InfoContext(ctx, "dinner grounding failed", "dinner_id", row.ID, "reason", reason)
write := context.WithoutCancel(ctx) write := context.WithoutCancel(ctx)
if _, failErr := db.New(s.st.Writer).FailPendingDinnerGrounding(write, db.FailPendingDinnerGroundingParams{ if _, failErr := db.New(s.st.Writer).FailPendingDinnerGrounding(write, db.FailPendingDinnerGroundingParams{
GroundingError: sql.NullString{String: reason, Valid: true}, GroundingError: sql.NullString{String: reason, Valid: true},
UpdatedAt: store.Now().String(), UpdatedAt: store.Now().String(),
ID: row.ID, ID: row.ID,
NormalizedName: row.NormalizedName, GroundingRevision: p.Revision,
}); failErr != nil { }); failErr != nil {
return groundResult{}, fmt.Errorf("dinners: ground: record failure of %s: %w", row.ID, failErr) return groundResult{}, fmt.Errorf("dinners: ground: record failure of %s: %w", row.ID, failErr)
} }
@@ -138,32 +153,85 @@ func (s *Service) runGroundJob(ctx context.Context, p GroundJob, _ func(groundRe
} }
n, err := db.New(s.st.Writer).GroundPendingDinner(ctx, db.GroundPendingDinnerParams{ n, err := db.New(s.st.Writer).GroundPendingDinner(ctx, db.GroundPendingDinnerParams{
RecipeID: sql.NullString{String: gr.RecipeID, Valid: true}, RecipeID: sql.NullString{String: gr.RecipeID, Valid: true},
PhotoID: sql.NullString{String: gr.PhotoID, Valid: true}, PhotoID: sql.NullString{String: gr.PhotoID, Valid: true},
Method: sql.NullString{String: gr.Method, Valid: true}, Method: sql.NullString{String: gr.Method, Valid: true},
MinutesTotal: nullInt(gr.MinutesTotal), MinutesTotal: nullInt(gr.MinutesTotal),
Difficulty: sql.NullString{String: gr.Difficulty, Valid: true}, Difficulty: sql.NullString{String: gr.Difficulty, Valid: true},
UpdatedAt: store.Now().String(), UpdatedAt: store.Now().String(),
ID: row.ID, ID: row.ID,
NormalizedName: row.NormalizedName, GroundingRevision: p.Revision,
}) })
if err != nil { if err != nil {
return groundResult{}, fmt.Errorf("dinners: ground: record %s: %w", row.ID, err) return groundResult{}, fmt.Errorf("dinners: ground: record %s: %w", row.ID, err)
} }
if n == 0 { if n == 0 {
cur, err := db.New(s.st.Reader).GetDinner(ctx, row.ID) return s.currentGrounding(ctx, row.ID)
if err != nil {
return groundResult{}, fmt.Errorf("dinners: ground: read %s: %w", row.ID, err)
}
return groundResult{DinnerID: cur.ID, GroundingStatus: cur.GroundingStatus}, nil
} }
return groundResult{DinnerID: row.ID, GroundingStatus: GroundingGrounded}, nil return groundResult{DinnerID: row.ID, GroundingStatus: GroundingGrounded}, nil
} }
// FailInterruptedGroundings fails, with ReasonInterrupted, every dinner // currentGrounding is the KindGroundDinner result of a job that did not
// still pending whose KindGroundDinner job is no longer queued or running: // ground dinner id: the grounding status the dinner has now.
// a previous process stopped before enqueuing the job or before its func (s *Service) currentGrounding(ctx context.Context, id string) (groundResult, error) {
// handler recorded a result. Call it once at boot, after the job runner's cur, err := db.New(s.st.Reader).GetDinner(ctx, id)
if err != nil {
return groundResult{}, fmt.Errorf("dinners: ground: read %s: %w", id, err)
}
return groundResult{DinnerID: cur.ID, GroundingStatus: cur.GroundingStatus}, nil
}
// errGroundingStopped is the cause a running KindGroundDinner job's
// grounding is cancelled with when its dinner is archived or renamed.
var errGroundingStopped = errors.New("dinners: grounding stopped: dinner archived or renamed")
// runningGrounding is a KindGroundDinner job running for a dinner at
// revision, with the cancel of its grounding.
type runningGrounding struct {
revision int64
cancel context.CancelCauseFunc
}
// trackGrounding returns a context for grounding dinnerID at revision that
// stopGroundings cancels, and the func to call when the job ends.
func (s *Service) trackGrounding(ctx context.Context, dinnerID string, revision int64) (context.Context, func()) {
groundCtx, cancel := context.WithCancelCause(ctx)
r := &runningGrounding{revision: revision, cancel: cancel}
s.groundingsMu.Lock()
if s.groundings[dinnerID] == nil {
s.groundings[dinnerID] = make(map[*runningGrounding]struct{})
}
s.groundings[dinnerID][r] = struct{}{}
s.groundingsMu.Unlock()
return groundCtx, func() {
s.groundingsMu.Lock()
delete(s.groundings[dinnerID], r)
if len(s.groundings[dinnerID]) == 0 {
delete(s.groundings, dinnerID)
}
s.groundingsMu.Unlock()
cancel(nil)
}
}
// stopGroundings cancels, with errGroundingStopped, the grounding of every
// KindGroundDinner job running for dinnerID at a revision below below. Call
// it once the change that makes those jobs stale is committed.
func (s *Service) stopGroundings(dinnerID string, below int64) {
s.groundingsMu.Lock()
defer s.groundingsMu.Unlock()
for r := range s.groundings[dinnerID] {
if r.revision < below {
r.cancel(errGroundingStopped)
}
}
}
// FailInterruptedGroundings fails, with ReasonInterrupted, every unarchived
// dinner still pending whose KindGroundDinner job is no longer queued or
// running: a previous process stopped before enqueuing the job or before
// its handler recorded a result. An archived dinner is left pending, since
// archiving, not a restart, stopped its grounding. Call it once at boot, after the job runner's
// Start has failed the jobs that process left running, so no dinner stays // Start has failed the jobs that process left running, so no dinner stays
// pending with nothing left to ground it. It returns how many it failed. // pending with nothing left to ground it. It returns how many it failed.
func (s *Service) FailInterruptedGroundings(ctx context.Context) (int64, error) { func (s *Service) FailInterruptedGroundings(ctx context.Context) (int64, error) {
+413 -14
View File
@@ -2,11 +2,15 @@ package dinners_test
import ( import (
"context" "context"
"database/sql"
"encoding/json"
"errors" "errors"
"io" "io"
"io/fs"
"log/slog" "log/slog"
"slices" "slices"
"testing" "testing"
"testing/fstest"
"time" "time"
"code.cubecraftlabs.com/Maniacs_Mansion/Meal-Quest/internal/ai/llm" "code.cubecraftlabs.com/Maniacs_Mansion/Meal-Quest/internal/ai/llm"
@@ -52,6 +56,13 @@ func newEnv(t *testing.T) *env {
func newUnstartedEnv(t *testing.T) *env { func newUnstartedEnv(t *testing.T) *env {
t.Helper() t.Helper()
dataDir := t.TempDir() dataDir := t.TempDir()
return newUnstartedEnvOn(t, openStore(t, dataDir), dataDir)
}
// openStore opens the store in dataDir, closing it when the test ends. It
// is not migrated.
func openStore(t *testing.T, dataDir string) *store.Store {
t.Helper()
st, err := store.Open(t.Context(), dataDir) st, err := store.Open(t.Context(), dataDir)
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
@@ -61,6 +72,13 @@ func newUnstartedEnv(t *testing.T) *env {
t.Error(err) t.Error(err)
} }
}) })
return st
}
// newUnstartedEnvOn is newUnstartedEnv over st, the store in dataDir,
// which it migrates to the latest version.
func newUnstartedEnvOn(t *testing.T, st *store.Store, dataDir string) *env {
t.Helper()
if err := store.Migrate(t.Context(), st.Writer, migrations.FS); err != nil { if err := store.Migrate(t.Context(), st.Writer, migrations.FS); err != nil {
t.Fatal(err) t.Fatal(err)
} }
@@ -288,19 +306,16 @@ func TestManualFixWhilePendingIsKept(t *testing.T) {
// TestRenameWhilePendingGroundsNewName: a Regular renamed while its job is // TestRenameWhilePendingGroundsNewName: a Regular renamed while its job is
// still grounding the old name never gets the old name's result, whether // still grounding the old name never gets the old name's result, whether
// that job succeeds or fails; the job the rename enqueues grounds the new // the old name would be found or not: the rename stops that job where it
// name. // is, and the job the rename enqueues grounds the new name.
func TestRenameWhilePendingGroundsNewName(t *testing.T) { func TestRenameWhilePendingGroundsNewName(t *testing.T) {
t.Parallel() t.Parallel()
tests := []struct { tests := []struct {
name string name string
oldName string oldName string
replies []llm.GenerateResult
}{ }{
{"old name found", "Chicken curry", []llm.GenerateResult{ {"old name found", "Chicken curry"},
recipestest.NormalizedReply("stovetop", 40), recipestest.NormalizedReply("oven", 150), {"old name not found", "Unknown dish"},
}},
{"old name not found", "Unknown dish", []llm.GenerateResult{recipestest.NormalizedReply("oven", 150)}},
} }
for _, tt := range tests { for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) { t.Run(tt.name, func(t *testing.T) {
@@ -308,9 +323,7 @@ func TestRenameWhilePendingGroundsNewName(t *testing.T) {
e := newEnv(t) e := newEnv(t)
e.search.Results["Beef stew"] = []recipes.Candidate{{URL: stew}} e.search.Results["Beef stew"] = []recipes.Candidate{{URL: stew}}
e.search.Gate = make(chan struct{}) e.search.Gate = make(chan struct{})
for _, r := range tt.replies { e.model.QueueGenerate(recipestest.NormalizedReply("oven", 150))
e.model.QueueGenerate(r)
}
c, err := e.svc.CreateRegular(t.Context(), dinners.NewRegular{Name: ptr(tt.oldName)}, member) c, err := e.svc.CreateRegular(t.Context(), dinners.NewRegular{Name: ptr(tt.oldName)}, member)
if err != nil { if err != nil {
@@ -325,12 +338,13 @@ func TestRenameWhilePendingGroundsNewName(t *testing.T) {
if d.GroundingStatus != dinners.GroundingPending { if d.GroundingStatus != dinners.GroundingPending {
t.Fatalf("renamed = %+v, want still pending", d) t.Fatalf("renamed = %+v, want still pending", d)
} }
// The old name's job ends while its search is still held.
first := e.waitJob(t, *c.GroundingJobID)
if first.Status != jobs.StatusSucceeded || string(first.Result) != `{"dinner_id":"`+c.ID+`","grounding_status":"pending"}` {
t.Fatalf("old name's job = %+v (result %s), want succeeded leaving the dinner pending", first, first.Result)
}
close(e.search.Gate) close(e.search.Gate)
first := e.waitJob(t, *c.GroundingJobID)
if string(first.Result) == `{"dinner_id":"`+c.ID+`","grounding_status":"grounded"}` {
t.Fatalf("old name's job reported grounded: %+v (result %s)", first, first.Result)
}
row := e.waitGrounding(t, c.ID) row := e.waitGrounding(t, c.ID)
if row.GroundingStatus != dinners.GroundingGrounded { if row.GroundingStatus != dinners.GroundingGrounded {
t.Fatalf("dinner = %+v, want grounded for the new name", row) t.Fatalf("dinner = %+v, want grounded for the new name", row)
@@ -345,10 +359,222 @@ func TestRenameWhilePendingGroundsNewName(t *testing.T) {
if got := e.search.Queries(); !slices.Equal(got, []string{tt.oldName, "Beef stew"}) { if got := e.search.Queries(); !slices.Equal(got, []string{tt.oldName, "Beef stew"}) {
t.Errorf("searched %q, want the old name then the new", got) t.Errorf("searched %q, want the old name then the new", got)
} }
if got := len(e.model.Requests()); got != 1 {
t.Errorf("model called %d times, want once, for the new name only", got)
}
}) })
} }
} }
// TestRenameLinkWhilePendingGroundsNewName: a Regular added from a link and
// renamed while still pending drops the link and grounds its new name, not
// the old link's recipe and photo.
func TestRenameLinkWhilePendingGroundsNewName(t *testing.T) {
t.Parallel()
e := newUnstartedEnv(t)
e.search.Results["Beef stew"] = []recipes.Candidate{{URL: stew}}
e.model.QueueGenerate(recipestest.NormalizedReply("oven", 150))
c, err := e.svc.CreateRegular(t.Context(), dinners.NewRegular{RecipeURL: ptr(curryA)}, member)
if err != nil {
t.Fatal(err)
}
d, err := e.svc.Update(t.Context(), c.ID, dinners.Patch{Name: ptr("Beef stew")}, member)
if err != nil {
t.Fatal(err)
}
if d.GroundingStatus != dinners.GroundingPending || d.RecipeURL != nil {
t.Fatalf("renamed = %+v, want still pending with no recipe_url", d)
}
e.start(t)
row := e.waitGrounding(t, c.ID)
if row.GroundingStatus != dinners.GroundingGrounded {
t.Fatalf("dinner = %+v, want grounded for the new name", row)
}
rec, err := db.New(e.st.Reader).GetRecipe(t.Context(), row.RecipeID.String)
if err != nil {
t.Fatal(err)
}
if rec.SourceUrl != stew || rec.PhotoID != row.PhotoID || row.Method.String != "oven" || row.MinutesTotal.Int64 != 150 {
t.Errorf("dinner = %+v recipe %s, want grounded in %s, not the old link %s", row, rec.SourceUrl, stew, curryA)
}
if got := e.search.Queries(); !slices.Equal(got, []string{"Beef stew"}) {
t.Errorf("searched %q, want only the new name", got)
}
if row.RecipeUrl.Valid {
t.Errorf("recipe_url = %q, want the old link dropped", row.RecipeUrl.String)
}
if _, err := db.New(e.st.Reader).GetRecipeBySourceURL(t.Context(), curryA); !errors.Is(err, sql.ErrNoRows) {
t.Errorf("old link's recipe lookup err = %v, want it never grounded (sql.ErrNoRows)", err)
}
}
// queuedJob reads the payload of queued KindGroundDinner job id.
func (e *env) queuedJob(t *testing.T, id string) dinners.GroundJob {
t.Helper()
var payload string
if err := e.st.Reader.QueryRowContext(t.Context(), `SELECT payload FROM jobs WHERE id = ?`, id).Scan(&payload); err != nil {
t.Fatal(err)
}
var p dinners.GroundJob
if err := json.Unmarshal([]byte(payload), &p); err != nil {
t.Fatal(err)
}
return p
}
// TestRenameBeforeJobStartsSkipsStaleJob: a Regular renamed before its
// first job starts is grounded only by the job its rename enqueued. The
// first job, though it would read the new name, does no work and writes
// nothing, so its failure cannot leave the dinner "Needs a recipe" while
// the rename's job finds a recipe.
func TestRenameBeforeJobStartsSkipsStaleJob(t *testing.T) {
t.Parallel()
e := newUnstartedEnv(t)
e.search.Results["Beef stew"] = []recipes.Candidate{{URL: stew}}
c, err := e.svc.CreateRegular(t.Context(), dinners.NewRegular{Name: ptr("Chicken curry")}, member)
if err != nil {
t.Fatal(err)
}
first := e.queuedJob(t, *c.GroundingJobID)
if first.DinnerID != c.ID || first.Revision != e.dinner(t, c.ID).GroundingRevision {
t.Fatalf("first job = %+v, want dinner %s at its revision", first, c.ID)
}
if _, err := e.svc.Update(t.Context(), c.ID, dinners.Patch{Name: ptr("Beef stew")}, member); err != nil {
t.Fatal(err)
}
renamed := e.dinner(t, c.ID)
if renamed.GroundingRevision != first.Revision+1 {
t.Fatalf("revision after rename = %d, want %d", renamed.GroundingRevision, first.Revision+1)
}
// The first job would fail if it searched.
e.search.Err = recipes.ErrSearchRateLimited
status, err := e.svc.RunGroundJob(t.Context(), first)
if err != nil || status != dinners.GroundingPending {
t.Fatalf("first job = %q, %v; want pending, no error", status, err)
}
if q := e.search.Queries(); len(q) != 0 {
t.Errorf("first job searched %q, want no work for a stale revision", q)
}
if row := e.dinner(t, c.ID); row != renamed {
t.Fatalf("first job wrote the dinner: %+v, was %+v", row, renamed)
}
e.search.Err = nil
e.model.QueueGenerate(recipestest.NormalizedReply("oven", 150))
status, err = e.svc.RunGroundJob(t.Context(), dinners.GroundJob{DinnerID: c.ID, Revision: renamed.GroundingRevision})
if err != nil || status != dinners.GroundingGrounded {
t.Fatalf("rename's job = %q, %v; want grounded", status, err)
}
row := e.dinner(t, c.ID)
rec, err := db.New(e.st.Reader).GetRecipe(t.Context(), row.RecipeID.String)
if err != nil {
t.Fatal(err)
}
if row.GroundingStatus != dinners.GroundingGrounded || rec.SourceUrl != stew {
t.Errorf("dinner = %+v recipe %s, want grounded in %s", row, rec.SourceUrl, stew)
}
// A stale job finishing last changes nothing either.
if status, err := e.svc.RunGroundJob(t.Context(), first); err != nil || status != dinners.GroundingGrounded {
t.Errorf("first job run last = %q, %v; want grounded reported, no error", status, err)
}
if after := e.dinner(t, c.ID); after != row {
t.Errorf("stale job changed the grounded dinner: %+v, was %+v", after, row)
}
}
// TestEditWhilePendingIsKept: a method or minutes a member saves while a
// Regular is grounding is kept when grounding succeeds; what the member
// did not set comes from the recipe.
func TestEditWhilePendingIsKept(t *testing.T) {
t.Parallel()
e := newEnv(t)
e.search.Gate = make(chan struct{})
e.model.QueueGenerate(recipestest.NormalizedReply("stovetop", 40))
c, err := e.svc.CreateRegular(t.Context(), dinners.NewRegular{Name: ptr("Chicken curry")}, member)
if err != nil {
t.Fatal(err)
}
e.waitQuery(t, "Chicken curry")
if _, err := e.svc.Update(t.Context(), c.ID, dinners.Patch{Method: ptr("grill"), MinutesTotal: ptr(15)}, member); err != nil {
t.Fatal(err)
}
close(e.search.Gate)
row := e.waitGrounding(t, c.ID)
if row.GroundingStatus != dinners.GroundingGrounded || !row.RecipeID.Valid || !row.PhotoID.Valid {
t.Fatalf("dinner = %+v, want grounded", row)
}
if row.Method.String != "grill" || row.MinutesTotal.Int64 != 15 || row.Difficulty.String != "easy" {
t.Errorf("dinner = %s, %d minutes, %s; want the member's grill and 15 minutes, the recipe's easy",
row.Method.String, row.MinutesTotal.Int64, row.Difficulty.String)
}
}
// TestArchiveWhilePendingStopsGrounding: a pending Regular archived before
// its job starts is not searched for, and one archived while its job runs
// is stopped there, with no page fetched or model called; neither ends up
// grounded.
func TestArchiveWhilePendingStopsGrounding(t *testing.T) {
t.Parallel()
t.Run("before the job starts", func(t *testing.T) {
t.Parallel()
e := newUnstartedEnv(t)
e.model.QueueGenerate(recipestest.NormalizedReply("stovetop", 40))
c, err := e.svc.CreateRegular(t.Context(), dinners.NewRegular{Name: ptr("Chicken curry")}, member)
if err != nil {
t.Fatal(err)
}
if _, err := e.svc.Archive(t.Context(), c.ID, member); err != nil {
t.Fatal(err)
}
e.start(t)
if job := e.waitJob(t, *c.GroundingJobID); job.Status != jobs.StatusSucceeded {
t.Fatalf("job = %+v, want succeeded with nothing to do", job)
}
if q := e.search.Queries(); len(q) != 0 {
t.Errorf("searched %q for an archived dinner", q)
}
if row := e.dinner(t, c.ID); row.Status != dinners.StatusArchived || row.GroundingStatus == dinners.GroundingGrounded || row.RecipeID.Valid {
t.Errorf("dinner = %+v, want archived and not grounded", row)
}
})
t.Run("while the job runs", func(t *testing.T) {
t.Parallel()
e := newEnv(t)
e.search.Gate = make(chan struct{})
e.model.QueueGenerate(recipestest.NormalizedReply("stovetop", 40))
c, err := e.svc.CreateRegular(t.Context(), dinners.NewRegular{Name: ptr("Chicken curry")}, member)
if err != nil {
t.Fatal(err)
}
e.waitQuery(t, "Chicken curry")
if _, err := e.svc.Archive(t.Context(), c.ID, member); err != nil {
t.Fatal(err)
}
// The job ends while its search is still held: archiving stopped it.
job := e.waitJob(t, *c.GroundingJobID)
close(e.search.Gate)
if job.Status != jobs.StatusSucceeded || string(job.Result) != `{"dinner_id":"`+c.ID+`","grounding_status":"pending"}` {
t.Fatalf("job = %+v (result %s), want succeeded leaving the dinner pending", job, job.Result)
}
if got := e.pages.Fetched(); len(got) != 0 {
t.Errorf("fetched %q for an archived dinner", got)
}
if got := len(e.model.Requests()); got != 0 {
t.Errorf("model called %d times for an archived dinner", got)
}
if row := e.dinner(t, c.ID); row.Status != dinners.StatusArchived || row.GroundingStatus == dinners.GroundingGrounded || row.RecipeID.Valid {
t.Errorf("dinner = %+v, want archived and not grounded", row)
}
})
}
// TestRenameWhilePendingEnqueueFailure: a pending Regular renamed when the // TestRenameWhilePendingEnqueueFailure: a pending Regular renamed when the
// job for its new name cannot be enqueued is failed, not left pending. // job for its new name cannot be enqueued is failed, not left pending.
func TestRenameWhilePendingEnqueueFailure(t *testing.T) { func TestRenameWhilePendingEnqueueFailure(t *testing.T) {
@@ -492,6 +718,59 @@ func TestFixViaCandidate(t *testing.T) {
} }
} }
// TestFixReplacesPendingLink: fixing a Regular added from a link that held
// no recipe drops that link, so it no longer blocks adding the link again,
// while the link of the recipe chosen does.
func TestFixReplacesPendingLink(t *testing.T) {
t.Parallel()
const lost = "https://gone.example/recipes/lamb-tagine"
tests := []struct {
name string
fix dinners.Fix
// linkTaken is the recipe link the fixed dinner now has, if any.
linkTaken string
}{
{"candidate", dinners.Fix{CandidateID: ptr(curryB)}, curryB},
{"manual", dinners.Fix{Manual: &dinners.ManualRecipe{Ingredients: []string{"1 lamb"}, Minutes: ptr(90), Method: ptr("oven")}}, ""},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
e := newEnv(t)
c, err := e.svc.CreateRegular(t.Context(), dinners.NewRegular{RecipeURL: ptr(lost)}, member)
if err != nil {
t.Fatal(err)
}
if job := e.waitJob(t, *c.GroundingJobID); job.Status != jobs.StatusFailed {
t.Fatalf("job = %+v, want failed", job)
}
// Renamed, so only the link could repeat it.
if _, err := e.svc.Update(t.Context(), c.ID, dinners.Patch{Name: ptr("Weeknight dinner")}, member); err != nil {
t.Fatal(err)
}
e.model.QueueGenerate(recipestest.NormalizedReply("stovetop", 25))
d, err := e.svc.Ground(t.Context(), c.ID, tt.fix, member)
if err != nil {
t.Fatal(err)
}
if d.RecipeURL != nil {
t.Errorf("recipe_url = %q after the fix, want the unusable link dropped", *d.RecipeURL)
}
if _, err := e.svc.CreateRegular(t.Context(), dinners.NewRegular{RecipeURL: ptr(lost)}, member); err != nil {
t.Errorf("adding the dropped link again: %v, want added", err)
}
if tt.linkTaken != "" {
_, err := e.svc.CreateRegular(t.Context(), dinners.NewRegular{RecipeURL: ptr(tt.linkTaken)}, member)
var dup *dinners.DuplicateError
if !errors.As(err, &dup) || dup.Existing.ID != c.ID {
t.Errorf("adding the fixed dinner's recipe link: err = %v, want a duplicate of %s", err, c.ID)
}
}
})
}
}
func TestFixViaRecipeURL(t *testing.T) { func TestFixViaRecipeURL(t *testing.T) {
t.Parallel() t.Parallel()
e := newEnv(t) e := newEnv(t)
@@ -667,6 +946,126 @@ func TestRestartFailsInterruptedGrounding(t *testing.T) {
} }
} }
// TestRestartKeepsArchivedPendingDinner: a pending Regular archived while
// its job runs stays pending, its job succeeded; the boot cleanup leaves it
// as it is rather than failing it as interrupted.
func TestRestartKeepsArchivedPendingDinner(t *testing.T) {
t.Parallel()
e := newEnv(t)
e.search.Gate = make(chan struct{})
c, err := e.svc.CreateRegular(t.Context(), dinners.NewRegular{Name: ptr("Chicken curry")}, member)
if err != nil {
t.Fatal(err)
}
e.waitQuery(t, "Chicken curry")
if _, err := e.svc.Archive(t.Context(), c.ID, member); err != nil {
t.Fatal(err)
}
job := e.waitJob(t, *c.GroundingJobID)
close(e.search.Gate)
if job.Status != jobs.StatusSucceeded {
t.Fatalf("job = %+v, want succeeded", job)
}
// The database is now as the next boot finds it.
n, err := e.svc.FailInterruptedGroundings(t.Context())
if err != nil {
t.Fatal(err)
}
if n != 0 {
t.Errorf("failed %d dinners, want 0", n)
}
if row := e.dinner(t, c.ID); row.Status != dinners.StatusArchived ||
row.GroundingStatus != dinners.GroundingPending || row.GroundingError.Valid {
t.Errorf("dinner = %s, %s (%s), want archived and pending with no error",
row.Status, row.GroundingStatus, row.GroundingError.String)
}
}
// TestUpgradeKeepsLegacyQueuedGrounding: a pending Regular whose
// ground_dinner job was queued before migration 0010, so its payload has
// no revision, is not failed as interrupted at the first boot after the
// upgrade; its job still grounds it. The upgrade starts either from 0009
// or from a database that has already applied 0010.
func TestUpgradeKeepsLegacyQueuedGrounding(t *testing.T) {
t.Parallel()
for name, below := range map[string]string{
"from 0009": "0010",
"from 0010": "0011",
} {
t.Run(name, func(t *testing.T) {
t.Parallel()
upgradeWithLegacyQueuedGrounding(t, below)
})
}
}
// upgradeWithLegacyQueuedGrounding migrates a database to every migration
// named before below, adds a pending Regular with a queued ground_dinner
// job whose payload has no revision, then boots on the latest migrations.
func upgradeWithLegacyQueuedGrounding(t *testing.T, below string) {
ctx := t.Context()
dataDir := t.TempDir()
st := openStore(t, dataDir)
before := fstest.MapFS{}
files, err := fs.Glob(migrations.FS, "*.sql")
if err != nil {
t.Fatal(err)
}
for _, name := range files {
if name >= below {
continue
}
body, err := fs.ReadFile(migrations.FS, name)
if err != nil {
t.Fatal(err)
}
before[name] = &fstest.MapFile{Data: body}
}
if err := store.Migrate(ctx, st.Writer, before); err != nil {
t.Fatal(err)
}
now := store.Now().String()
if _, err := st.Writer.ExecContext(ctx, `INSERT INTO dinners
(id, name, normalized_name, status, origin, is_favorite, grounding_status, updated_by, created_at, updated_at)
VALUES ('dinner-1', 'Chicken curry', 'chicken curry', 'regular', 'regular', 0, 'pending', ?, ?, ?)`,
member, now, now); err != nil {
t.Fatal(err)
}
if _, err := st.Writer.ExecContext(ctx, `INSERT INTO jobs (id, kind, status, payload, created_at, updated_at)
VALUES ('job-1', ?, 'queued', '{"dinner_id":"dinner-1"}', ?, ?)`,
dinners.KindGroundDinner, now, now); err != nil {
t.Fatal(err)
}
// Boot: migrate, start the runner, then fail interrupted groundings. The
// search is held, so the job cannot finish before the cleanup runs.
e := newUnstartedEnvOn(t, st, dataDir)
e.search.Gate = make(chan struct{})
e.model.QueueGenerate(recipestest.NormalizedReply("stovetop", 40))
if p := e.queuedJob(t, "job-1"); p != (dinners.GroundJob{DinnerID: "dinner-1", Revision: 0}) {
t.Errorf("migrated payload = %+v, want dinner-1 at revision 0", p)
}
e.start(t)
n, err := e.svc.FailInterruptedGroundings(ctx)
if err != nil {
t.Fatal(err)
}
if n != 0 {
t.Errorf("failed %d dinners, want 0", n)
}
if row := e.dinner(t, "dinner-1"); row.GroundingStatus != dinners.GroundingPending {
t.Fatalf("dinner = %s (%s), want still pending", row.GroundingStatus, row.GroundingError.String)
}
close(e.search.Gate)
if job := e.waitJob(t, "job-1"); job.Status != jobs.StatusSucceeded {
t.Fatalf("job = %+v", job)
}
if row := e.dinner(t, "dinner-1"); row.GroundingStatus != dinners.GroundingGrounded {
t.Errorf("dinner = %s, want grounded", row.GroundingStatus)
}
}
// TestCreateEnqueueFailureReturnsFailedDinner: a dinner saved when its job // TestCreateEnqueueFailureReturnsFailedDinner: a dinner saved when its job
// cannot be enqueued is returned, with its id, failed and no job id. // cannot be enqueued is returned, with its id, failed and no job id.
func TestCreateEnqueueFailureReturnsFailedDinner(t *testing.T) { func TestCreateEnqueueFailureReturnsFailedDinner(t *testing.T) {
@@ -66,9 +66,15 @@ func (s *Search) Queries() []string {
type Pages struct { type Pages struct {
Bodies map[string][]byte Bodies map[string][]byte
Gate chan struct{} Gate chan struct{}
mu sync.Mutex
fetched []string
} }
func (p *Pages) Fetch(ctx context.Context, rawURL string) (recipes.Page, error) { func (p *Pages) Fetch(ctx context.Context, rawURL string) (recipes.Page, error) {
p.mu.Lock()
p.fetched = append(p.fetched, rawURL)
p.mu.Unlock()
if p.Gate != nil { if p.Gate != nil {
select { select {
case <-p.Gate: case <-p.Gate:
@@ -83,6 +89,13 @@ func (p *Pages) Fetch(ctx context.Context, rawURL string) (recipes.Page, error)
return recipes.Page{URL: rawURL, ContentType: "text/html", Body: body}, nil return recipes.Page{URL: rawURL, ContentType: "text/html", Body: body}, nil
} }
// Fetched returns every URL fetched, in order.
func (p *Pages) Fetched() []string {
p.mu.Lock()
defer p.mu.Unlock()
return append([]string(nil), p.fetched...)
}
// Images is a recipes.ImageFetcher serving JPEGs by URL; any other URL is // Images is a recipes.ImageFetcher serving JPEGs by URL; any other URL is
// a 404. // a 404.
type Images struct { type Images struct {
+4 -4
View File
@@ -175,8 +175,8 @@ func TestMigrateEmbeddedMigrations(t *testing.T) {
if err := Migrate(context.Background(), s.Writer, migrations.FS); err != nil { if err := Migrate(context.Background(), s.Writer, migrations.FS); err != nil {
t.Fatal(err) t.Fatal(err)
} }
if got := versions(migrationRows(t, s)); got != "0000_bootstrap,0001_init,0002_household_settings_updated_by,0003_jobs_status,0004_ai_provider_presets,0005_ai_provider_xai,0006_ai_runs_status,0007_recipe_grounding,0008_dinner_recipe_url,0009_dinner_grounding_error" { if got := versions(migrationRows(t, s)); got != "0000_bootstrap,0001_init,0002_household_settings_updated_by,0003_jobs_status,0004_ai_provider_presets,0005_ai_provider_xai,0006_ai_runs_status,0007_recipe_grounding,0008_dinner_recipe_url,0009_dinner_grounding_error,0010_dinner_grounding_revision,0011_ground_dinner_job_revision" {
t.Errorf("versions = %q, want 0000_bootstrap,0001_init,0002_household_settings_updated_by,0003_jobs_status,0004_ai_provider_presets,0005_ai_provider_xai,0006_ai_runs_status,0007_recipe_grounding,0008_dinner_recipe_url,0009_dinner_grounding_error", got) t.Errorf("versions = %q, want 0000_bootstrap,0001_init,0002_household_settings_updated_by,0003_jobs_status,0004_ai_provider_presets,0005_ai_provider_xai,0006_ai_runs_status,0007_recipe_grounding,0008_dinner_recipe_url,0009_dinner_grounding_error,0010_dinner_grounding_revision,0011_ground_dinner_job_revision", got)
} }
} }
@@ -196,8 +196,8 @@ func TestMigrateBootstrapThenEmbeddedInitApplies(t *testing.T) {
if err := Migrate(ctx, s.Writer, migrations.FS); err != nil { if err := Migrate(ctx, s.Writer, migrations.FS); err != nil {
t.Fatal(err) t.Fatal(err)
} }
if got := versions(migrationRows(t, s)); got != "0000_bootstrap,0001_init,0002_household_settings_updated_by,0003_jobs_status,0004_ai_provider_presets,0005_ai_provider_xai,0006_ai_runs_status,0007_recipe_grounding,0008_dinner_recipe_url,0009_dinner_grounding_error" { if got := versions(migrationRows(t, s)); got != "0000_bootstrap,0001_init,0002_household_settings_updated_by,0003_jobs_status,0004_ai_provider_presets,0005_ai_provider_xai,0006_ai_runs_status,0007_recipe_grounding,0008_dinner_recipe_url,0009_dinner_grounding_error,0010_dinner_grounding_revision,0011_ground_dinner_job_revision" {
t.Errorf("versions = %q, want 0000_bootstrap,0001_init,0002_household_settings_updated_by,0003_jobs_status,0004_ai_provider_presets,0005_ai_provider_xai,0006_ai_runs_status,0007_recipe_grounding,0008_dinner_recipe_url,0009_dinner_grounding_error", got) t.Errorf("versions = %q, want 0000_bootstrap,0001_init,0002_household_settings_updated_by,0003_jobs_status,0004_ai_provider_presets,0005_ai_provider_xai,0006_ai_runs_status,0007_recipe_grounding,0008_dinner_recipe_url,0009_dinner_grounding_error,0010_dinner_grounding_revision,0011_ground_dinner_job_revision", got)
} }
if !tableExists(t, s, "dinners") { if !tableExists(t, s, "dinners") {
t.Error("table from 0001_init was not created") t.Error("table from 0001_init was not created")
@@ -0,0 +1,7 @@
-- 0010_dinner_grounding_revision: each ground_dinner job carries the
-- dinner's grounding_revision when it was enqueued, and records its result
-- only while the dinner still has that revision. Renaming a pending dinner
-- raises the revision before enqueuing the job for the new name, so an
-- earlier job — queued or running — can no longer write the dinner.
ALTER TABLE dinners ADD COLUMN grounding_revision INTEGER NOT NULL DEFAULT 0;
@@ -0,0 +1,11 @@
-- 0011_ground_dinner_job_revision: every dinner starts at grounding
-- revision 0 (0010), so a ground_dinner job enqueued before 0010, whose
-- payload has only dinner_id, is for revision 0. Recording it lets the boot
-- cleanup, which matches a pending dinner to its job by revision, see a job
-- still queued for its dinner. This is a migration of its own, not part of
-- 0010, so it also runs on a database that has already applied 0010.
UPDATE jobs
SET payload = json_set(payload, '$.revision', 0)
WHERE kind = 'ground_dinner'
AND json_type(payload, '$.revision') IS NULL;
+43 -24
View File
@@ -25,10 +25,13 @@ SELECT COUNT(*) FROM dinners
WHERE status = 'regular'; WHERE status = 'regular';
-- name: ListUnarchivedDinners :many -- name: ListUnarchivedDinners :many
-- Every dinner a new or renamed dinner must not repeat. -- Every dinner a new or renamed dinner must not repeat, with the
SELECT * FROM dinners -- source_url of the recipe it is grounded in (NULL while it has none).
WHERE status <> 'archived' SELECT sqlc.embed(dinners), recipes.source_url AS recipe_source_url
ORDER BY id; FROM dinners
LEFT JOIN recipes ON recipes.id = dinners.recipe_id
WHERE dinners.status <> 'archived'
ORDER BY dinners.id;
-- name: UpdateDinner :one -- name: UpdateDinner :one
-- An unknown id returns sql.ErrNoRows. -- An unknown id returns sql.ErrNoRows.
@@ -42,8 +45,10 @@ SET name = ?,
difficulty = ?, difficulty = ?,
cuisine = ?, cuisine = ?,
recipe_id = ?, recipe_id = ?,
recipe_url = ?,
photo_id = ?, photo_id = ?,
grounding_status = ?, grounding_status = ?,
grounding_revision = ?,
updated_by = ?, updated_by = ?,
updated_at = ? updated_at = ?
WHERE id = ? WHERE id = ?
@@ -77,10 +82,12 @@ WHERE id = ?
RETURNING *; RETURNING *;
-- name: GroundDinner :one -- name: GroundDinner :one
-- Records a dinner's grounding (grounded or manual) and clears any -- Records a dinner's grounding by hand (grounded or manual) and clears any
-- grounding_error. An unknown id returns sql.ErrNoRows. -- grounding_error and pasted recipe_url: the recipe chosen replaces the
-- link. An unknown id returns sql.ErrNoRows.
UPDATE dinners UPDATE dinners
SET recipe_id = ?, SET recipe_id = ?,
recipe_url = NULL,
photo_id = ?, photo_id = ?,
method = ?, method = ?,
minutes_total = ?, minutes_total = ?,
@@ -93,43 +100,55 @@ WHERE id = ?
RETURNING *; RETURNING *;
-- name: GroundPendingDinner :execrows -- name: GroundPendingDinner :execrows
-- Records the background grounding of a dinner still pending under the -- Records the background grounding of an unarchived dinner still pending at
-- normalized name it was grounded for; a dinner grounded some other way or -- the grounding_revision it was grounded for; a dinner grounded some other
-- renamed meanwhile is left as it is (0 rows). -- way, renamed or archived meanwhile is left as it is (0 rows). A method,
-- minutes_total or difficulty already set is kept: a pending dinner has
-- them only when a member edited them while it was grounding.
UPDATE dinners UPDATE dinners
SET recipe_id = ?, SET recipe_id = sqlc.arg(recipe_id),
photo_id = ?, photo_id = sqlc.arg(photo_id),
method = ?, method = COALESCE(method, sqlc.arg(method)),
minutes_total = ?, minutes_total = COALESCE(minutes_total, sqlc.arg(minutes_total)),
difficulty = ?, difficulty = COALESCE(difficulty, sqlc.arg(difficulty)),
grounding_status = 'grounded', grounding_status = 'grounded',
grounding_error = NULL, grounding_error = NULL,
updated_at = ? updated_at = sqlc.arg(updated_at)
WHERE id = ? AND grounding_status = 'pending' AND normalized_name = ?; WHERE id = sqlc.arg(id)
AND status <> 'archived'
AND grounding_status = 'pending'
AND grounding_revision = sqlc.arg(grounding_revision);
-- name: FailPendingDinnerGrounding :execrows -- name: FailPendingDinnerGrounding :execrows
-- Records why the background grounding of a dinner still pending under the -- Records why the background grounding of an unarchived dinner still
-- normalized name it was grounded for failed; a dinner grounded some other -- pending at the grounding_revision it was grounded for failed; a dinner
-- way or renamed meanwhile is left as it is (0 rows). -- grounded some other way, renamed or archived meanwhile is left as it is
-- (0 rows).
UPDATE dinners UPDATE dinners
SET grounding_status = 'failed', SET grounding_status = 'failed',
grounding_error = ?, grounding_error = ?,
updated_at = ? updated_at = ?
WHERE id = ? AND grounding_status = 'pending' AND normalized_name = ?; WHERE id = ?
AND status <> 'archived'
AND grounding_status = 'pending'
AND grounding_revision = ?;
-- name: FailOrphanedPendingDinnerGroundings :execrows -- name: FailOrphanedPendingDinnerGroundings :execrows
-- Fails every dinner still pending whose grounding job is no longer queued -- Fails every unarchived dinner still pending whose grounding job for its
-- or running: the process stopped before the job, or before its handler -- current grounding_revision is no longer queued or running: the process
-- recorded a result. Run at boot, after the job runner fails the jobs a -- stopped before the job, or before its handler recorded a result. An
-- previous process left running. -- archived dinner is left as it is; archiving stopped its grounding. Run at
-- boot, after the job runner fails the jobs a previous process left running.
UPDATE dinners UPDATE dinners
SET grounding_status = 'failed', SET grounding_status = 'failed',
grounding_error = sqlc.arg(grounding_error), grounding_error = sqlc.arg(grounding_error),
updated_at = sqlc.arg(updated_at) updated_at = sqlc.arg(updated_at)
WHERE grounding_status = 'pending' WHERE grounding_status = 'pending'
AND status <> 'archived'
AND NOT EXISTS ( AND NOT EXISTS (
SELECT 1 FROM jobs SELECT 1 FROM jobs
WHERE jobs.kind = sqlc.arg(kind) WHERE jobs.kind = sqlc.arg(kind)
AND jobs.status IN ('queued', 'running') AND jobs.status IN ('queued', 'running')
AND json_extract(jobs.payload, '$.dinner_id') = dinners.id AND json_extract(jobs.payload, '$.dinner_id') = dinners.id
AND json_extract(jobs.payload, '$.revision') = dinners.grounding_revision
); );
File diff suppressed because it is too large Load Diff
+20
View File
@@ -0,0 +1,20 @@
import { describe, expect, it } from 'vitest'
import cases from '../../../internal/ai/harness/testdata/dedupe_cases.json'
import { normalizeName, similarNames } from './names.ts'
// The cases internal/ai/harness's tests run against Normalize and Similar.
describe('dinner names, as the server compares them', () => {
it('has cases to run', () => {
expect(cases.normalize.length).toBeGreaterThan(0)
expect(cases.similar.length).toBeGreaterThan(0)
})
it.each(cases.normalize)('normalizes "$name" to "$want"', ({ name, want }) => {
expect(normalizeName(name)).toBe(want)
})
it.each(cases.similar)('"$a" and "$b" similar: $want', ({ a, b, want }) => {
expect(similarNames(a, b)).toBe(want)
expect(similarNames(b, a)).toBe(want)
})
})
+58
View File
@@ -0,0 +1,58 @@
// The PWA's copy of the server's dinner-name rules (internal/ai/harness
// Normalize and Similar), so a screen can tell beforehand that the dinners
// API would refuse a name as a duplicate. The server still decides.
// internal/ai/harness/testdata/dedupe_cases.json is run against both.
import folds from './letterFolds.json'
// SIMILAR_THRESHOLD is harness.SimilarThreshold.
const SIMILAR_THRESHOLD = 0.85
// fillerWords is harness's fillerWords.
const fillerWords = new Set(['easy', 'best', 'homemade', 'quick', 'simple', 'the', 'a', 'recipe', 'classic'])
// letterFolds is harness's letterFolds: gen_folds.py writes both tables,
// and harness's TestLetterFoldsMatchPWA fails when they differ.
const letterFolds: Record<string, string> = folds
const combiningMark = /\p{Mn}/u
const wordChar = /[\p{L}\p{Nd}]/u
// normalizeName is harness.Normalize: lowercased, accents dropped, letters
// folded, apostrophes removed, other punctuation turned into spaces and
// filler words dropped. "Easy Chicken Tikka-Masala!" is
// "chicken tikka masala".
export function normalizeName(name: string): string {
let out = ''
// Lowercased letter by letter, as Go's strings.ToLower does, so no
// letter is lowered by its context (a final Σ is σ, not ς).
for (const letter of name) {
for (const ch of letter.toLowerCase()) {
if (combiningMark.test(ch) || ch === "'" || ch === '’') continue
const fold = letterFolds[ch]
if (fold !== undefined) out += fold
else if (wordChar.test(ch)) out += ch
else out += ' '
}
}
return out
.split(' ')
.filter((word) => word !== '' && !fillerWords.has(word))
.join(' ')
}
function words(name: string): Set<string> {
const normalized = normalizeName(name)
return new Set(normalized === '' ? [] : normalized.split(' '))
}
// similarNames is harness.Similar: whether a and b name the same dinner, by
// the share of the words of their normalized forms they have in common.
export function similarNames(a: string, b: string): boolean {
const left = words(a)
const right = words(b)
if (left.size === 0 || right.size === 0) return false
let shared = 0
for (const word of left) if (right.has(word)) shared++
return shared / (left.size + right.size - shared) >= SIMILAR_THRESHOLD
}
@@ -100,6 +100,7 @@ async function renderStep(enableNotifications?: EnableNotifications) {
element={<NotificationsStep enableNotifications={enableNotifications} />} element={<NotificationsStep enableNotifications={enableNotifications} />}
/> />
<Route path="/" element={<h1>Home</h1>} /> <Route path="/" element={<h1>Home</h1>} />
<Route path="/onboarding/regulars" element={<h1>Regulars step</h1>} />
</Routes> </Routes>
</AuthBootstrap> </AuthBootstrap>
</ApiProviders> </ApiProviders>
@@ -257,6 +258,41 @@ describe('onboarding step 3', () => {
}) })
}) })
describe('onboarding step 3 with too few Regulars', () => {
it('says why and offers a way back to add Regulars', async () => {
const server = await renderStep()
fireEvent.click(later())
await vi.waitFor(() => expect(server.held).toHaveLength(1))
server.held[0]!.answer(
jsonResponse(400, {
error: {
code: 'invalid_settings',
message: 'invalid household settings: onboarding_completed: add at least 3 Regulars first (have 2)',
retryable: false,
},
}),
)
expect(await screen.findByRole('alert')).toHaveTextContent(
'Could not finish setting up: invalid household settings: onboarding_completed: add at least 3 Regulars first (have 2)',
)
const addRegulars = screen.getByRole('button', { name: 'Add Regulars' })
expectMinHitArea(addRegulars)
fireEvent.click(addRegulars)
expect(await screen.findByRole('heading', { level: 1, name: 'Regulars step' })).toBeInTheDocument()
})
it('offers no way back for any other refusal', async () => {
const server = await renderStep()
fireEvent.click(later())
await vi.waitFor(() => expect(server.held).toHaveLength(1))
server.held[0]!.answer(errorResponse(500, 'internal'))
expect(await screen.findByRole('alert')).toHaveTextContent('Could not finish setting up: internal message')
expect(screen.queryByRole('button', { name: 'Add Regulars' })).not.toBeInTheDocument()
})
})
describe('notificationsUnavailable', () => { describe('notificationsUnavailable', () => {
it('answers "unavailable" without asking for permission', async () => { it('answers "unavailable" without asking for permission', async () => {
await expect(notificationsUnavailable()).resolves.toBe('unavailable') await expect(notificationsUnavailable()).resolves.toBe('unavailable')
+18 -4
View File
@@ -2,7 +2,7 @@ import { useState } from 'react'
import { CalendarDays, ShoppingBasket, Sparkles, UtensilsCrossed, type LucideIcon } from 'lucide-react' import { CalendarDays, ShoppingBasket, Sparkles, UtensilsCrossed, type LucideIcon } from 'lucide-react'
import { useNavigate } from 'react-router' import { useNavigate } from 'react-router'
import { InstallHint } from '../pwa/InstallHint.tsx' import { InstallHint } from '../pwa/InstallHint.tsx'
import { useCompleteOnboarding } from '../settings/api.ts' import { needsMoreRegulars, useCompleteOnboarding } from '../settings/api.ts'
import { Button } from '../ui/Button.tsx' import { Button } from '../ui/Button.tsx'
import { Icon } from '../ui/Icon.tsx' import { Icon } from '../ui/Icon.tsx'
import { OnboardingStep } from './OnboardingStep.tsx' import { OnboardingStep } from './OnboardingStep.tsx'
@@ -50,12 +50,15 @@ export type NotificationsStepProps = {
// NotificationsStep is onboarding step 3: what MealQuest notifies about, the // NotificationsStep is onboarding step 3: what MealQuest notifies about, the
// iOS install hint, and finishing onboarding. Either button records that // iOS install hint, and finishing onboarding. Either button records that
// onboarding is finished and goes Home. // onboarding is finished and goes Home. When the server refuses because the
// household has fewer than three Regulars, the reason shows with a way back
// to step 2 to add them.
export function NotificationsStep({ enableNotifications = notificationsUnavailable }: NotificationsStepProps) { export function NotificationsStep({ enableNotifications = notificationsUnavailable }: NotificationsStepProps) {
const navigate = useNavigate() const navigate = useNavigate()
const complete = useCompleteOnboarding() const complete = useCompleteOnboarding()
const [enabling, setEnabling] = useState(false) const [enabling, setEnabling] = useState(false)
const [problem, setProblem] = useState<string | null>(null) const [problem, setProblem] = useState<string | null>(null)
const [fewRegulars, setFewRegulars] = useState(false)
const busy = enabling || complete.isPending const busy = enabling || complete.isPending
async function finish() { async function finish() {
@@ -63,13 +66,19 @@ export function NotificationsStep({ enableNotifications = notificationsUnavailab
await complete.mutateAsync() await complete.mutateAsync()
} catch (error) { } catch (error) {
setProblem(`Could not finish setting up: ${(error as Error).message}`) setProblem(`Could not finish setting up: ${(error as Error).message}`)
setFewRegulars(needsMoreRegulars(error as Error))
return return
} }
void navigate('/', { replace: true }) void navigate('/', { replace: true })
} }
async function turnOn() { function clearProblem() {
setProblem(null) setProblem(null)
setFewRegulars(false)
}
async function turnOn() {
clearProblem()
setEnabling(true) setEnabling(true)
try { try {
await enableNotifications() await enableNotifications()
@@ -83,7 +92,7 @@ export function NotificationsStep({ enableNotifications = notificationsUnavailab
} }
function later() { function later() {
setProblem(null) clearProblem()
void finish() void finish()
} }
@@ -124,6 +133,11 @@ export function NotificationsStep({ enableNotifications = notificationsUnavailab
{problem} {problem}
</p> </p>
)} )}
{fewRegulars && (
<Button variant="outline" className={styles.recover} onClick={() => void navigate('/onboarding/regulars')}>
Add Regulars
</Button>
)}
</OnboardingStep> </OnboardingStep>
) )
} }
+5
View File
@@ -229,6 +229,11 @@
color: var(--color-accent-700); color: var(--color-accent-700);
} }
/* Step 3: back to step 2 when the server wants more Regulars. */
.recover {
margin-top: var(--space-3);
}
.status { .status {
margin-top: 10px; margin-top: 10px;
} }
@@ -146,6 +146,14 @@ describe('onboarding step 2 (mock 1h)', () => {
expect(declaredStyle(pills[0]!).height).toBe('6px') expect(declaredStyle(pills[0]!).height).toBe('6px')
}) })
it('does not offer a common pick the server would refuse as a duplicate of a Regular', async () => {
await regularsStep([dinnerBody('d1', 'Homemade chili'), dinnerBody('d2', 'Easy Chicken Stir Fry')])
expect(screen.queryByRole('button', { name: 'Add Chili' })).not.toBeInTheDocument()
expect(screen.queryByRole('button', { name: 'Add Chicken stir-fry' })).not.toBeInTheDocument()
expect(screen.getByRole('button', { name: 'Add Beef tacos' })).toBeInTheDocument()
})
it('sizes the input pill, "+" button, picks and Next as the mock does', async () => { it('sizes the input pill, "+" button, picks and Next as the mock does', async () => {
await regularsStep() await regularsStep()
expect(hitArea(input()).height).toBe(48) expect(hitArea(input()).height).toBe(48)
+4 -6
View File
@@ -2,6 +2,7 @@ import { useState, type FormEvent } from 'react'
import { Plus } from 'lucide-react' import { Plus } from 'lucide-react'
import { useNavigate } from 'react-router' import { useNavigate } from 'react-router'
import { useAddRegular, useRegulars, useRemoveRegular, type Regular } from '../dinners/api.ts' import { useAddRegular, useRegulars, useRemoveRegular, type Regular } from '../dinners/api.ts'
import { similarNames } from '../dinners/names.ts'
import { useRefusals } from '../dinners/useRefusals.ts' import { useRefusals } from '../dinners/useRefusals.ts'
import { Button } from '../ui/Button.tsx' import { Button } from '../ui/Button.tsx'
import { Chip } from '../ui/Chip.tsx' import { Chip } from '../ui/Chip.tsx'
@@ -16,7 +17,8 @@ import styles from './Onboarding.module.css'
// household moves on. // household moves on.
export const MIN_REGULARS = 3 export const MIN_REGULARS = 3
// commonPicks are offered as one-tap Regulars until the household has them. // commonPicks are offered as one-tap Regulars until the household has them,
// or a Regular the server would count as the same dinner.
export const commonPicks = [ export const commonPicks = [
'Beef tacos', 'Beef tacos',
'Spaghetti bolognese', 'Spaghetti bolognese',
@@ -28,10 +30,6 @@ export const commonPicks = [
'Fried rice', 'Fried rice',
] ]
function sameName(a: string, b: string): boolean {
return a.trim().toLowerCase() === b.trim().toLowerCase()
}
// RegularsStep is onboarding step 2, handoff mock 1h: the household adds // RegularsStep is onboarding step 2, handoff mock 1h: the household adds
// the dinners it makes on repeat. Each add and remove shows at once and is // the dinners it makes on repeat. Each add and remove shows at once and is
// undone if the server refuses it. // undone if the server refuses it.
@@ -43,7 +41,7 @@ export function RegularsStep() {
const [draft, setDraft] = useState('') const [draft, setDraft] = useState('')
const { refusals, track } = useRefusals() const { refusals, track } = useRefusals()
const list = regulars.data ?? [] const list = regulars.data ?? []
const picks = commonPicks.filter((pick) => !list.some((regular) => sameName(regular.name, pick))) const picks = commonPicks.filter((pick) => !list.some((regular) => similarNames(regular.name, pick)))
// Next waits for every add to be answered, so a refused add's error shows // Next waits for every add to be answered, so a refused add's error shows
// here and an add the server takes back never counts towards the three. // here and an add the server takes back never counts towards the three.
const adding = list.some((regular) => regular.pending) const adding = list.some((regular) => regular.pending)
+12
View File
@@ -441,6 +441,18 @@ export function householdSettingsProblems(error: Error): HouseholdSettingsProble
return problems return problems
} }
// needsMoreRegulars reports whether error is the server refusing to finish
// onboarding because the household has fewer than three Regulars
// (internal/household MinRegulars) — the only refusal of
// {"onboarding_completed": true}.
export function needsMoreRegulars(error: Error): boolean {
return (
error instanceof ApiError &&
error.code === 'invalid_settings' &&
error.message.startsWith(`${invalidHouseholdPrefix}onboarding_completed: `)
)
}
// aiStatus is the AI assistant row's status line: the provider and the // aiStatus is the AI assistant row's status line: the provider and the
// model's display name, or that no model is chosen yet (the seed). // model's display name, or that no model is chosen yet (the seed).
export function aiStatus(settings: Pick<AISettings, 'provider' | 'model'>): string { export function aiStatus(settings: Pick<AISettings, 'provider' | 'model'>): string {