MealQuest/docs/ARCHITECTURE.md

# MealQuest architecture

This mirrors the architecture section of `CONTEXT.MD` in the CTXD project
`mealquest`, which is the source of record; if the two disagree, CTXD wins and
this file should be updated. It describes the **planned** application. The
repository today is the Feature 01 scaffold: `cmd/server`, `internal/config`,
`internal/server` (`/health`), `internal/frontend` (embedded PWA with SPA
fallback), the `web/` skeleton, `Makefile`, `Dockerfile` and CI.

Decision numbers (D-NNN) refer to `DECISIONS.MD` in CTXD.

## Topology

```
PWA (phone / tablet / desktop)
   │ HTTPS
   ▼
Traefik v3.6 — 10.60.1.33
   file route: mealquest-ccl.yml
   │
   ▼
docker compose project "mealquest" — 10.60.1.14
   ├── mealquest   Go service + embedded PWA, host 8090 → container 8080
   ├── searxng     sidecar for recipe search; internal compose network
   │               only, no published port
   └── Litestream  SQLite replication → Synology NAS 10.60.1.50
```

One Go service serves the API, the auth flow, media and the PWA. Its SQLite
database is replicated by Litestream to the NAS.

Traefik terminates TLS. The service honours `X-Forwarded-For` and
`X-Forwarded-Proto` only from peers listed in `MEALQUEST_TRUSTED_PROXY`
(comma-separated IPs/CIDRs; production lists `10.60.1.33`); from any other
peer they are ignored. The client is the rightmost `X-Forwarded-For` address
that is not a trusted proxy. The resolved client IP is added as `client_ip`
to every log line written while handling a request. The OIDC redirect URI
and the Secure cookies are fixed by `MEALQUEST_PUBLIC_URL`, never by request
headers, so sign-in works whichever peer the request arrives from.

## Repository layout

| Path | Purpose |
| --- | --- |
| `cmd/server` | Entry point; wiring only. |
| `internal/config` | Env-only `MEALQUEST_*` config; secrets from `*_FILE`. |
| `internal/server` | HTTP server, router, middleware. |
| `internal/frontend` | Embedded PWA, SPA fallback. |
| `internal/auth` | Authentik OIDC, sessions, CSRF. |
| `internal/db`, `internal/store` | SQLite connections, sqlc queries. |
| `internal/secretbox` | AES-256-GCM encryption for `_ct` columns. |
| `internal/household`, `internal/dinners`, `internal/recipes` | Domain. |
| `internal/ai/llm`, `internal/ai/harness`, `internal/ai/skills` | AI provider adapters, skill harness, skills. |
| `internal/suggest`, `internal/plan`, `internal/shopping`, `internal/ideas` | Feature services. |
| `internal/gcal`, `internal/notify`, `internal/push`, `internal/chat` | Integrations. |
| `internal/jobs`, `internal/scheduler` | Background jobs and schedules. |
| `migrations/` | SQL migrations (no goose or golang-migrate). |
| `web/` | React PWA. |
| `deploy/` | Deployment files. |
| `docs/` | Documentation. |
| `design/` | Requirements and UI handoff (read-only reference). |

## Backend stack and conventions

- Go 1.25, `CGO_ENABLED=0`, a single static binary.
- Router: `go-chi/chi/v5`. Logging: `log/slog` JSON.
- SQLite via `modernc.org/sqlite`, with `journal_mode=WAL`,
  `foreign_keys=ON` and `busy_timeout`. **One writer connection plus a read
  pool.** Queries generated by `sqlc`.
- IDs are ULIDs (`oklog/ulid/v2`).
- Timestamps are stored in UTC; local calendar dates are `YYYY-MM-DD` strings
  in the household timezone.
- Configuration is environment-only (`MEALQUEST_*`). Secrets come only from
  files named by `*_FILE` variables. Missing or invalid config stops startup.
- Error responses always have the body
  `{"error":{"code":"…","message":"…","retryable":false}}`.

## Authentication

- Authentik OIDC; the backend is a **confidential client**
  (`coreos/go-oidc/v3`, `golang.org/x/oauth2`).
- Session cookie `__Host-mealquest_session` (HttpOnly, Secure,
  SameSite=Lax). Sessions expire after 30 days idle or 180 days absolute.
- Non-GET requests require the CSRF header `X-MealQuest-CSRF`.
- Access requires membership of the Authentik group `mealquest-admin`.
  There is a **single role**: one shared household, no per-person profiles.

## Data model

Columns ending in `_ct` hold AES-256-GCM ciphertext (via `secretbox`).

| Table | Holds |
| --- | --- |
| `household_settings` | One row: timezone, dinner_time (17:30), reminder_time (16:30), review_time (09:00), plan_draft day/time (Fri 09:00), suggestions day/time (Sun 08:00), chat_autosend day/time (Sat 08:00), onboarding_completed_at, updates_seen_at. |
| `recipes` | A real recipe page: source_url (unique), site_name, title, total/prep/cook minutes (the page's JSON-LD), yield, raw_jsonld, fetched_at; and its grounding: photo_id, method, difficulty, minutes_total (normalized prep to table). |
| `recipe_ingredients` | recipe_id, position, raw_text, name, qty, unit, note. |
| `recipe_steps` | recipe_id, position, text. |
| `photos` | source_url, local_path, width, height, sha256, fetched_at. |
| `dinners` | The meal record and its lifecycle: name, normalized_name, status (`suggested` / `quest_log` / `tried` / `regular` / `archived`), origin, method, minutes_total, difficulty, cuisine, is_favorite, recipe_id, photo_id, grounding_status, updated_by. |
| `suggestion_batches` | trigger, status, ai_run_id. |
| `suggestions` | dinner_id, batch_id, reason, cited_regular_ids (JSON), status, decided_at. |
| `week_plans` | week_start (unique), status (`draft` / `synced` / `changed`), ai_note, ai_run_id, calendar_synced_at. |
| `plan_days` | plan_id, date (unique per plan), dinner_id, is_new, calendar_event_id. |
| `rotation_reviews` | dinner_id + plan_date (unique), status, decision (`keep` / `drop` / `not_made`), decided_at, updated_by. |
| `shopping_lists` | week_start (unique), plan_id, title, status (`current` / `archived`), sent_to_chat_at, frozen_at. |
| `shopping_groups` / `shopping_items` | One group per dinner/day; item name, qty, needed (bool), position. |
| `ai_settings` | One row: provider, base_url, model, temperature, system_prompt, api_key_ct, api_key_last4, key_verified_at. |
| `recipe_search_settings` | One row: provider (`searxng` / `brave`), base_url (the SearXNG instance), api_key_ct and api_key_last4 (Brave). |
| `integrations_google` | One row: refresh_token_ct, calendar_id, calendar_name, connected_at, last_sync_error. |
| `chat_settings` | One row: webhook_url_ct, space_label, last_test_at, last_test_status. |
| `notification_prefs` | event (`plan_ready` / `list_ready` / `day_of_reminder` / `new_suggestions` / `rotation_review`), push, chat. |
| `push_subscriptions` | endpoint (unique), p256dh, auth, user_agent, failure_count, last_success_at. |
| `notification_outbox` | event_key + channel (unique), payload, status, attempts, next_attempt_at, last_error. |
| `jobs` | kind, status, payload, result (progressive), error, timestamps. |
| `schedule_runs` | schedule, period_key (unique). |
| `ai_runs` | skill, skill_version, provider, model, temperature, status, latency_ms, tokens_in/out, error, redacted request/response (30-day retention). |
| `sessions` | id_hash, subject, email, name, groups, csrf_token, created/last_seen/expires. |

`recipes` holds a fetched recipe page; `dinners` holds the household's meal
record and its lifecycle, optionally linked to a recipe and a photo.

## Dinner lifecycle

```
suggested (deck) ──Try it──▶ quest_log ──planned date passes──▶ tried
    │                            ▲                                │
    │                            └──────── explicit decision ─────┤
    │                                                             ├──▶ regular
    │                                                             └──▶ archived
    └──Pass──▶ archived (name kept for dedupe)
```

From `tried`, an explicit decision moves the dinner to `regular`, to
`archived`, or back to `quest_log`. Promotion to `regular` is **never
automatic** (D-021).

## API surface

- JSON API under `/api/v1`, using the error body above for every failure.
  It requires the session cookie and, for non-GET requests, the
  `X-MealQuest-CSRF` header.
- `/auth/*` — the OIDC flow.
- `/media/*` — cached photos, behind the session cookie.
  `GET /media/photos/{id}-{400|1200}.jpg` serves a stored size of a photo
  with a `photos` row, with
  `Cache-Control: private, max-age=31536000, immutable`; anything else is
  404.
- `GET`/`PATCH /api/v1/settings/recipe-search` — provider, base_url and
  the Brave key (`api_key`, write-only: responses carry only
  `api_key_set` and `api_key_last4`; an empty string removes it). The
  result must be usable: SearXNG needs a base_url, Brave a key.
  `POST /api/v1/settings/recipe-search/test` searches "chicken tikka
  masala" and returns `{query, count}`, 409 `not_configured`, or 502
  `search_failed` with the provider's error.
- `/health` — liveness.
- Everything else serves the embedded PWA with SPA fallback.

## AI layer

- **Provider interface** (`ai/llm`) with adapters for OpenAI-compatible,
  Anthropic and Ollama endpoints, called over plain HTTP — no vendor SDKs.
- **Skill harness** (`ai/harness`): prompt layering, validation of the
  structured output, and **one repair retry**; if the repair also fails the
  run fails — no silent fallback. Every run writes one `ai_runs` row when it
  ends, covering the repair turn too, with status `ok`, `repaired`,
  `bad_output` or `provider_error`; a prompt that cannot be rendered is
  `bad_output` and a provider that cannot be built is `provider_error`, both
  with no turns. The harness builds the provider itself from the settings
  and the decrypted API key (`presets.FromSettings`), so the key it redacts
  is always the key the provider sends. Request and response
  bodies are redacted before they are stored: the decrypted API key,
  Authorization and API-key header values, Google Chat webhook URLs and URLs
  with key-like query parameters are removed, inside JSON strings too; the
  skill, provider and model columns are redacted the same way. Writing the
  row is bounded: a run whose work ended in time must write its row before
  the skill's deadline or it fails with `llm.ErrTimeout` and its row records
  the timeout; a failed run's row has a further fixed bound, and if even
  that write fails the run returns the write error with its own and no row
  is stored. A stored row's status therefore always matches what the caller
  received.
  `harness.Purge` deletes rows older than `harness.RunRetention` (30 days),
  clearing the `suggestion_batches` and `week_plans` references first. The
  harness does not call it: the scheduler's daily `ai_runs` purge
  (Feature 20) must call `harness.Purge` with `now - RunRetention`.
- **Concurrency guards** (`harness.Guard`, one per process): `harness.Run`
  takes a scope key, and concurrent runs of one skill with one scope share a
  single in-flight run and its result, error included (single-flight); runs
  with different scopes are independent. A semaphore lets at most
  `harness.DefaultMaxCalls` (2, set by `NewGuard`) LLM calls run at once;
  waiting for a slot honours the context. Each skill declares a timeout
  (`harness.DefaultTimeout`, 60 s, unless it needs another) bounding the
  whole run, slot waits and repair turn included; a run past it fails with
  `llm.ErrTimeout`. A caller whose context ends gets the context error; the
  shared run is cancelled, freeing its slot, once no caller waits for it.
- **Prompt layering** (D-007): each skill version's prompt is a
  `text/template` file embedded from `ai/skills/prompts/`
  (`<skill>.v<version>.tmpl`). The system prompt is always, in order:
  [1] the admin persona (`ai_settings.system_prompt`), [2] the skill's
  instructions, [3] the household context block, [4] the output contract
  with the JSON Schema — the contract is always last. The user message comes
  from the skill's input. `checksums.txt` pins every template; changing one
  without a version bump fails the tests.
- **Household context** (`harness.BuildContext`): layer 3 holds today's date
  and weekday in the household timezone, the Regulars, the Quest log, the
  last 60 passed Side quests and the plans of the last four ISO weeks. A
  character budget trims the oldest plans first, then the oldest passes;
  Regulars and the Quest log are never trimmed.
- **Name dedupe** (`harness.Normalize`, `harness.Similar`, `harness.Deduper`):
  a dinner name is normalized by lowercasing, dropping combining marks and
  folding precomposed letters to their base letters through a table generated
  from Unicode data (`gen_folds.py`), so composed and decomposed spellings
  match, stripping punctuation, dropping filler words (easy, best, homemade, quick, simple,
  the, a, recipe, classic) and collapsing whitespace. Two names are the same
  dinner when the token-set (Jaccard) similarity of their words is at least
  0.85. A `Deduper` over the Regulars, Quest log, pending suggestions and
  passed names rejects a suggestion that repeats any of them.

## Recipe grounding

- Dinners are grounded in real recipe pages found through recipe search
  (`searxng` or `brave`, per `recipe_search_settings`).
- Fetching is **SSRF-safe** (`recipes.Fetcher`): https:// only, at every
  redirect hop. Its dialer resolves the host once and refuses unless every
  answer is public — every non-globally-reachable IPv4 special-purpose
  range (loopback, RFC 1918, link-local, carrier-grade NAT, documentation
  TEST-NETs, benchmarking 198.18/15, multicast, reserved and more) is
  refused, IPv4-mapped forms included, and IPv6 must be global unicast
  (2000::/3) outside documentation, 6to4, Teredo and other special ranges —
  then dials the address it checked, so a
  rebinding DNS answer cannot swap in a private one. Keep-alives and
  environment proxies are off, so every request dials and checks afresh. At
  most 3 redirects; a body over 2 MB is an error, never truncated; 10 s for
  the whole fetch; a descriptive `User-Agent`; requests to one host at least
  1 s apart, counted from requests actually sent (an abandoned wait holds no
  slot); 2xx pages cached by URL for an hour (64 pages, oldest evicted), and
  callers get their own copy of a cached body.
- Recipe data is parsed from schema.org `Recipe` JSON-LD
  (`recipes.Extract`): a small targeted scan of the page's
  `application/ld+json` scripts and `og:site_name` (no HTML parser
  dependency), finding the Recipe node at a script's top level, in a
  top-level array or in an `@graph`, with `@type` a string or an array of
  terms or absolute schema.org IRIs (`https://schema.org/Recipe`). An image
  or publisher given as an `{"@id": ...}` reference resolves to the node
  with that `@id` anywhere in the page's JSON-LD. The largest image (by ImageObject width × height; the first of equals) is
  resolved to an absolute URL; instructions — one string of lines, or a
  list of strings, HowToSteps and HowToSections — are flattened in order;
  ISO-8601 prep/cook/total times become minutes, total derived as prep +
  cook when only those are given; the site name is the publisher's, else
  `og:site_name`, else the host. A page with no Recipe is `ErrNoRecipe`,
  with two is `ErrManyRecipes`; a shape schema.org does not allow — such as
  a non-ISO-8601 duration like "20 minutes" or an empty one, or an image
  `@id` no node has — is `ErrMalformedRecipe`, never guessed at.
- Recipe search (`recipes.SearchProvider`: `Search(ctx, query, n)` →
  `[]Candidate{URL, Title, Snippet}`, n 1–20) appends " recipe" to a query
  without that word. SearXNG is called at
  `{recipe_search_settings.base_url}/search?q=…&format=json&categories=general` —
  directly, not through the Fetcher, since this configured endpoint may be
  on the internal network; Brave calls the Brave Web Search API with the
  key decrypted from `recipe_search_settings.api_key_ct`
  (`recipes.NewSearchProvider`). Redirects are not followed; 401/403 is
  `ErrSearchRefused`, 429 `ErrSearchRateLimited`, other non-2xx a
  `SearchStatusError`, and a body that is not the provider's JSON (SearXNG
  without a `results` array; Brave without `"type": "search"` or with a
  `web` object lacking `results` — Brave omits `web` when nothing matched),
  over 1 MB or with a result lacking an absolute http(s) URL
  `ErrSearchResponse`.
- Ranking (`recipes.Rank`) fetches and extracts the top 5 candidates
  concurrently, rejects any whose recipe lacks ingredients or an image, and
  orders the rest by the token similarity of the recipe's title to the dish
  name (`harness.Similarity`, 0–1) plus a quarter for each of ingredients,
  image, steps and total minutes; ties keep search order. No unranked
  candidate is dropped silently: when some are ranked, they are returned
  with `ErrCandidatesUnranked`; when none are, the error is
  `ErrNoUsableRecipe`. Either is joined with each unranked candidate's URL
  and reason (fetch failure, no valid recipe, missing ingredients or
  image), so the caller decides per item what a failure means.
- Ingredient lines are normalized by the `normalize_recipe` v1 skill
  (`skills.NormalizeRecipe`), always at temperature 0 whatever the
  household's setting. From the title, raw ingredient lines, steps and
  JSON-LD times it returns `{ingredients: [{name, qty, unit, note}], method,
  difficulty, minutes_total}`. Validation requires exactly one ingredient
  per input line, in order (a name matching another line but not its own is
  out of order), each with a name; a mismatch gets the harness's repair
  turn. Code-derived values win: a positive JSON-LD total is
  `minutes_total`; "slow cooker" or "crock pot" in the title or a step makes
  the method `crockpot`; a difficulty the model leaves null is derived —
  hard over 10 steps or 90 minutes, easy at 5 steps or fewer and 45 minutes
  or less, else medium, a crockpot's minutes not counting.
- Photos are cached locally (`photos`) and processed with
  `golang.org/x/image`. `recipes.PhotoCache.DownloadPhoto` downloads through
  the Fetcher's `FetchImage` (JPEG, PNG, WebP or GIF content types, 8 MB
  cap, not cached), decodes the image (a GIF's first frame) and refuses a
  decoded image under 300 px on its short side, recording nothing.
  `CachePhoto` then records it in the caller's transaction: it scales the
  image so its long side is 1200 px and 400 px, and stores each as JPEG
  quality 82 at `$MEALQUEST_DATA_DIR/media/photos/{id}-{size}.jpg`. Images
  are deduplicated by the SHA-256 of the downloaded bytes; the writer's
  single connection serialises transactions, so one image is never
  recorded twice. Files written for a transaction that rolls back are
  removed by the caller (`RemovePhotoFiles`).
- Grounding (`recipes.Grounder`) ties these together.
  `Ground(ctx, dishName, query)` searches for the top 5 candidates and ranks
  them all, stored and new alike: a candidate whose `source_url` is already
  stored is scored on its stored recipe (title, ingredients, steps, times,
  photo source) and its page is never refetched; every other candidate is
  fetched and extracted. Ranking alone decides between them. It then tries
  each ranked candidate best first. A stored one is returned as it is, so a
  stored recipe is reused only when it ranks above every new candidate that
  grounds. A new one is grounded in this order: (1) `DownloadPhoto` —
  the download and validation step of photo caching — (2)
  `normalize_recipe`, then (3) one **transaction** that runs `CachePhoto`
  (scales and writes the photo files and records the `photos` row) and
  stores the recipe, ingredients and steps (an existing `source_url` is
  updated in place); files written for a rolled-back transaction are
  removed. It returns `Grounded{recipe_id,
  photo_id, method, minutes_total, difficulty, source_url, site_name}`. A
  candidate whose photo fails or whose model output is unusable
  (`llm.ErrBadOutput`) is passed over; when none grounds, the error is a
  typed `*recipes.ErrUngroundable` listing every reason (no candidates,
  Rank's per-candidate reasons, photo or normalize failures). Failures
  that are not the candidates' — search, model provider, database — are
  returned as they are. `GroundURL(ctx, url)` does the same for one pasted
  URL. `GroundMany` grounds items 3 at a time and returns one result per
  item, so one failure never affects its siblings. A recipe is usable only
  with ingredients, an image and, if given, a positive total time.

## Scheduler

Times are in the household timezone. The times of `day_of_reminder`
(`reminder_time`) and `rotation_check` (`review_time`), and the day and time
of `suggestions_weekly`, `plan_draft` and `shopping_chat_autosend`, are
household settings in `household_settings` (`dinner_time` is also stored
there); the table shows their defaults. `week_rollover` and the `ai_runs`
purge are fixed and not configurable.

| Schedule | When | Does |
| --- | --- | --- |
| `suggestions_weekly` | Sunday 08:00 | New batch of Side quests. |
| `plan_draft` | Friday 09:00 | AI drafts next week's plan. |
| `shopping_chat_autosend` | Saturday 08:00 | Sends the Supply run to Google Chat. |
| `day_of_reminder` | Daily 16:30 | Tonight's dinner reminder. |
| `rotation_check` | Daily 09:00 | Asks "Keep it in rotation?" for tried dinners. |
| `week_rollover` | Monday 00:05 (fixed) | Freezes last week's Supply run into the Supply archive. |
| `ai_runs` purge | Daily (fixed) | Calls `harness.Purge` to delete `ai_runs` rows past 30-day retention (Feature 20). |

Runs are recorded in `schedule_runs` and executed through `jobs`.

## Notifications

- Events: `plan_ready`, `list_ready`, `day_of_reminder`, `new_suggestions`,
  `rotation_review`; each can go to Web Push and/or Google Chat per
  `notification_prefs`.
- Delivery goes through `notification_outbox`.
- Web Push uses VAPID (`webpush-go`). Google Chat uses the stored incoming
  webhook.

## Google Calendar

- One-way push: the app is the source of truth; edits in Google Calendar do
  not come back.
- Connected once with Google OAuth in Settings; one event per planned
  dinner.
- **Deterministic event ids**; each plan day records its
  `calendar_event_id`.

## Frontend

- React 19, TypeScript, Vite, React Router 7, TanStack Query 5,
  `vite-plugin-pwa`, `lucide-react` (stroke 2.75).
- CSS Modules over `tokens.css`; no Tailwind, no Framer Motion, no
  hard-coded hex. Self-hosted Bagel Fat One and Figtree.
- Tests: Vitest, Testing Library, jsdom.
- Offline cache for the current plan, shopping list and recipes.

| Route | Screen |
| --- | --- |
| `/` | Home |
| `/onboarding` | Add regular dinners |
| `/plan` | Weekly plan (Draft board) |
| `/plan/shopping` | Shopping list (Supply run) |
| `/plan/shopping/archive` | Supply archive |
| `/discover` | Swipe suggestions (Side quests) |
| `/quick` | Quick dinners |
| `/cook-from-pantry` | Cook from what I have |
| `/dinner/:id` | Recipe / dinner detail (Kitchen reading mode on tablet) |
| `/settings` | Settings home |
| `/settings/household` | Household settings |
| `/settings/regulars` | Regular dinners |
| `/settings/calendar` | Google Calendar connection |
| `/settings/notifications` | Notifications and Google Chat |
| `/settings/ai` | Admin · AI assistant |

`/settings/household`, `/settings/regulars` and `/settings/calendar` come from
CTXD; the other routes are defined in the UI handoff.

## Boundaries and gotchas

- **Secrets never reach the client** — API keys, OAuth tokens and the Chat
  webhook stay server-side, encrypted at rest.
- **The app is the source of truth for the plan**; Google Calendar is a
  one-way mirror.
- **Never auto-promote to Regular** (D-021).
- **Quick** means cooked in a crock pot **or** ≤ 30 minutes prep to table.
- **Weeks run Monday–Sunday** in the household timezone.
- **No silent fallbacks** — fail fast and report.
- **iOS Web Push requires the PWA to be installed to the Home Screen**; show
  an install hint.