MealQuest/docs/ARCHITECTURE.md

MealQuest architecture

This mirrors the architecture section of CONTEXT.MD in the CTXD project mealquest, which is the source of record; if the two disagree, CTXD wins and this file should be updated. It describes the planned application. The repository today is the Feature 01 scaffold: cmd/server, internal/config, internal/server (/health), internal/frontend (embedded PWA with SPA fallback), the web/ skeleton, Makefile, Dockerfile and CI.

Decision numbers (D-NNN) refer to DECISIONS.MD in CTXD.

Topology

PWA (phone / tablet / desktop)
   │ HTTPS
   ▼
Traefik v3.6 — 10.60.1.33
   file route: mealquest-ccl.yml
   │
   ▼
docker compose project "mealquest" — 10.60.1.14
   ├── mealquest   Go service + embedded PWA, host 8090 → container 8080
   ├── searxng     sidecar for recipe search; internal compose network
   │               only, no published port
   └── Litestream  SQLite replication → Synology NAS 10.60.1.50

One Go service serves the API, the auth flow, media and the PWA. Its SQLite database is replicated by Litestream to the NAS.

Traefik terminates TLS. The service honours X-Forwarded-For and X-Forwarded-Proto only from peers listed in MEALQUEST_TRUSTED_PROXY (comma-separated IPs/CIDRs; production lists 10.60.1.33); from any other peer they are ignored. The client is the rightmost X-Forwarded-For address that is not a trusted proxy. The resolved client IP is added as client_ip to every log line written while handling a request. The OIDC redirect URI and the Secure cookies are fixed by MEALQUEST_PUBLIC_URL, never by request headers, so sign-in works whichever peer the request arrives from.

Repository layout

Path	Purpose
cmd/server	Entry point; wiring only.
internal/config	Env-only MEALQUEST_* config; secrets from *_FILE.
internal/server	HTTP server, router, middleware.
internal/frontend	Embedded PWA, SPA fallback.
internal/auth	Authentik OIDC, sessions, CSRF.
internal/db, internal/store	SQLite connections, sqlc queries.
internal/secretbox	AES-256-GCM encryption for _ct columns.
internal/household, internal/dinners, internal/recipes	Domain.
internal/ai/llm, internal/ai/harness, internal/ai/skills	AI provider adapters, skill harness, skills.
internal/suggest, internal/plan, internal/shopping, internal/ideas	Feature services.
internal/gcal, internal/notify, internal/push, internal/chat	Integrations.
internal/jobs, internal/scheduler	Background jobs and schedules.
migrations/	SQL migrations (no goose or golang-migrate).
web/	React PWA.
deploy/	Deployment files.
docs/	Documentation.
design/	Requirements and UI handoff (read-only reference).

Backend stack and conventions

• Go 1.25, CGO_ENABLED=0, a single static binary.
• Router: go-chi/chi/v5. Logging: log/slog JSON.
• SQLite via modernc.org/sqlite, with journal_mode=WAL, foreign_keys=ON and busy_timeout. One writer connection plus a read pool. Queries generated by sqlc.
• IDs are ULIDs (oklog/ulid/v2).
• Timestamps are stored in UTC; local calendar dates are YYYY-MM-DD strings in the household timezone.
• Configuration is environment-only (MEALQUEST_*). Secrets come only from files named by *_FILE variables. Missing or invalid config stops startup.
• Error responses always have the body {"error":{"code":"…","message":"…","retryable":false}}.

Authentication

• Authentik OIDC; the backend is a confidential client (coreos/go-oidc/v3, golang.org/x/oauth2).
• Session cookie __Host-mealquest_session (HttpOnly, Secure, SameSite=Lax). Sessions expire after 30 days idle or 180 days absolute.
• Non-GET requests require the CSRF header X-MealQuest-CSRF.
• Access requires membership of the Authentik group mealquest-admin. There is a single role: one shared household, no per-person profiles.

Data model

Columns ending in _ct hold AES-256-GCM ciphertext (via secretbox).

Table	Holds
household_settings	One row: timezone, dinner_time (17:30), reminder_time (16:30), review_time (09:00), plan_draft day/time (Fri 09:00), suggestions day/time (Sun 08:00), chat_autosend day/time (Sat 08:00), onboarding_completed_at, updates_seen_at.
recipes	A real recipe page: source_url (unique), site_name, title, total/prep/cook minutes (the page's JSON-LD), yield, raw_jsonld, fetched_at; and its grounding: photo_id, method, difficulty, minutes_total (normalized prep to table).
recipe_ingredients	recipe_id, position, raw_text, name, qty, unit, note.
recipe_steps	recipe_id, position, text.
photos	source_url, local_path, width, height, sha256, fetched_at.
dinners	The meal record and its lifecycle: name, normalized_name, status (suggested / quest_log / tried / regular / archived), origin, method, minutes_total, difficulty, cuisine, is_favorite, recipe_id, photo_id, grounding_status, updated_by.
suggestion_batches	trigger, status, ai_run_id.
suggestions	dinner_id, batch_id, reason, cited_regular_ids (JSON), status, decided_at.
week_plans	week_start (unique), status (draft / synced / changed), ai_note, ai_run_id, calendar_synced_at.
plan_days	plan_id, date (unique per plan), dinner_id, is_new, calendar_event_id.
rotation_reviews	dinner_id + plan_date (unique), status, decision (keep / drop / not_made), decided_at, updated_by.
shopping_lists	week_start (unique), plan_id, title, status (current / archived), sent_to_chat_at, frozen_at.
shopping_groups / shopping_items	One group per dinner/day; item name, qty, needed (bool), position.
ai_settings	One row: provider, base_url, model, temperature, system_prompt, api_key_ct, api_key_last4, key_verified_at.
recipe_search_settings	One row: provider (searxng / brave), base_url (the SearXNG instance), api_key_ct and api_key_last4 (Brave).
integrations_google	One row: refresh_token_ct, calendar_id, calendar_name, connected_at, last_sync_error.
chat_settings	One row: webhook_url_ct, space_label, last_test_at, last_test_status.
notification_prefs	event (plan_ready / list_ready / day_of_reminder / new_suggestions / rotation_review), push, chat.
push_subscriptions	endpoint (unique), p256dh, auth, user_agent, failure_count, last_success_at.
notification_outbox	event_key + channel (unique), payload, status, attempts, next_attempt_at, last_error.
jobs	kind, status, payload, result (progressive), error, timestamps.
schedule_runs	schedule, period_key (unique).
ai_runs	skill, skill_version, provider, model, temperature, status, latency_ms, tokens_in/out, error, redacted request/response (30-day retention).
sessions	id_hash, subject, email, name, groups, csrf_token, created/last_seen/expires.

recipes holds a fetched recipe page; dinners holds the household's meal record and its lifecycle, optionally linked to a recipe and a photo.

Dinner lifecycle

suggested (deck) ──Try it──▶ quest_log ──planned date passes──▶ tried
    │                            ▲                                │
    │                            └──────── explicit decision ─────┤
    │                                                             ├──▶ regular
    │                                                             └──▶ archived
    └──Pass──▶ archived (name kept for dedupe)

From tried, an explicit decision moves the dinner to regular, to archived, or back to quest_log. Promotion to regular is never automatic (D-021).

API surface

• JSON API under /api/v1, using the error body above for every failure. It requires the session cookie and, for non-GET requests, the X-MealQuest-CSRF header.
• /auth/* — the OIDC flow.
• /media/* — cached photos, behind the session cookie. GET /media/photos/{id}-{400|1200}.jpg serves a stored size of a photo with a photos row, with Cache-Control: private, max-age=31536000, immutable; anything else is 404.
• GET/PATCH /api/v1/settings/recipe-search — provider, base_url and the Brave key (api_key, write-only: responses carry only api_key_set and api_key_last4; an empty string removes it). The result must be usable: SearXNG needs a base_url, Brave a key. Responses also carry the read-only default_base_url, the deployment's MEALQUEST_SEARXNG_URL (or null), which the client pre-fills while no base_url is saved; only a PATCH of base_url stores a URL. POST /api/v1/settings/recipe-search/test searches "chicken tikka masala" and returns {query, count}, 409 not_configured, or 502 search_failed with the provider's error.
• GET/PATCH /api/v1/settings/ai (aisettings.Service) — provider, base_url, model, temperature, system_prompt, plus the read-only api_key_last4 and key_verified_at. PATCH requires a preset name as provider, temperature 0–1 in 0.1 steps, a non-empty model and a system_prompt of at most 8000 characters; a fixed-URL preset's base_url must be empty or its own. Provider and base_url cannot change while a key is stored (remove the key first), so a stored key is never sent to an endpoint it was not stored for. PUT /api/v1/settings/ai/api-key takes {api_key} (write-only, at least 8 characters, sealed with the secret box; an empty string removes it) and clears key_verified_at; a preset without a fixed base_url takes a key only when its base_url passes secureurl.Check. POST /api/v1/settings/ai/verify calls Provider.Verify and records key_verified_at; GET /api/v1/settings/ai/models returns {models: [{id, display_name}]} live from the provider, with the key redacted from every id and display_name. Both work before a model is chosen (presets.ForAccount), answer 409 not_configured when the settings cannot build a provider, and 502 with the provider's typed error (auth_failed, rate_limited, bad_output, provider_error; 504 timeout) when the call fails. POST /api/v1/settings/ai/reset-prompt restores household.DefaultPersonaPrompt (CTXD PROMPTS.MD §4). No response holds the key. Service.HarnessConfig reads the settings for each skill run, so a saved change applies to the next run.
• /health — liveness.
• Everything else serves the embedded PWA with SPA fallback.

AI layer

• Provider interface (ai/llm) with adapters for OpenAI-compatible, Anthropic and Ollama endpoints, called over plain HTTP — no vendor SDKs.
• Skill harness (ai/harness): prompt layering, validation of the structured output, and one repair retry; if the repair also fails the run fails — no silent fallback. Every run writes one ai_runs row when it ends, covering the repair turn too, with status ok, repaired, bad_output or provider_error; a prompt that cannot be rendered is bad_output and a provider that cannot be built is provider_error, both with no turns. The harness builds the provider itself from the settings and the decrypted API key (presets.FromSettings), so the key it redacts is always the key the provider sends. Request and response bodies are redacted before they are stored: the decrypted API key, Authorization and API-key header values, Google Chat webhook URLs and URLs with key-like query parameters are removed, inside JSON strings too; the skill, provider and model columns are redacted the same way. Writing the row is bounded: a run whose work ended in time must write its row before the skill's deadline or it fails with llm.ErrTimeout and its row records the timeout; a failed run's row has a further fixed bound, and if even that write fails the run returns the write error with its own and no row is stored. A stored row's status therefore always matches what the caller received. harness.Purge deletes rows older than harness.RunRetention (30 days), clearing the suggestion_batches and week_plans references first. The harness does not call it: the scheduler's daily ai_runs purge (Feature 20) must call harness.Purge with now - RunRetention.
• Concurrency guards (harness.Guard, one per process): harness.Run takes a scope key, and concurrent runs of one skill with one scope share a single in-flight run and its result, error included (single-flight); runs with different scopes are independent. A semaphore lets at most harness.DefaultMaxCalls (2, set by NewGuard) LLM calls run at once; waiting for a slot honours the context. Each skill declares a timeout (harness.DefaultTimeout, 60 s, unless it needs another) bounding the whole run, slot waits and repair turn included; a run past it fails with llm.ErrTimeout. A caller whose context ends gets the context error; the shared run is cancelled, freeing its slot, once no caller waits for it.
• Prompt layering (D-007): each skill version's prompt is a text/template file embedded from ai/skills/prompts/ (<skill>.v<version>.tmpl). The system prompt is always, in order: [1] the admin persona (ai_settings.system_prompt), [2] the skill's instructions, [3] the household context block, [4] the output contract with the JSON Schema — the contract is always last. The user message comes from the skill's input. checksums.txt pins every template; changing one without a version bump fails the tests.
• Household context (harness.BuildContext): layer 3 holds today's date and weekday in the household timezone, the Regulars, the Quest log, the last 60 passed Side quests and the plans of the last four ISO weeks. A character budget trims the oldest plans first, then the oldest passes; Regulars and the Quest log are never trimmed.
• Name dedupe (harness.Normalize, harness.Similar, harness.Deduper): a dinner name is normalized by lowercasing, dropping combining marks and folding precomposed letters to their base letters through a table generated from Unicode data (gen_folds.py), so composed and decomposed spellings match, stripping punctuation, dropping filler words (easy, best, homemade, quick, simple, the, a, recipe, classic) and collapsing whitespace. Two names are the same dinner when the token-set (Jaccard) similarity of their words is at least 0.85. A Deduper over the Regulars, Quest log, pending suggestions and passed names rejects a suggestion that repeats any of them.

Recipe grounding

• Dinners are grounded in real recipe pages found through recipe search (searxng or brave, per recipe_search_settings).
• Fetching is SSRF-safe (recipes.Fetcher): https:// only, at every redirect hop. Its dialer resolves the host once and refuses unless every answer is public — every non-globally-reachable IPv4 special-purpose range (loopback, RFC 1918, link-local, carrier-grade NAT, documentation TEST-NETs, benchmarking 198.18/15, multicast, reserved and more) is refused, IPv4-mapped forms included, and IPv6 must be global unicast (2000::/3) outside documentation, 6to4, Teredo and other special ranges — then dials the address it checked, so a rebinding DNS answer cannot swap in a private one. Keep-alives and environment proxies are off, so every request dials and checks afresh. At most 3 redirects; a body over 2 MB is an error, never truncated; 10 s for the whole fetch; a descriptive User-Agent; requests to one host at least 1 s apart, counted from requests actually sent (an abandoned wait holds no slot); 2xx pages cached by URL for an hour (64 pages, oldest evicted), and callers get their own copy of a cached body.
• Recipe data is parsed from schema.org Recipe JSON-LD (recipes.Extract): a small targeted scan of the page's application/ld+json scripts and og:site_name (no HTML parser dependency), finding the Recipe node at a script's top level, in a top-level array or in an @graph, with @type a string or an array of terms or absolute schema.org IRIs (https://schema.org/Recipe). An image or publisher given as an {"@id": ...} reference resolves to the node with that @id anywhere in the page's JSON-LD. The largest image (by ImageObject width × height; the first of equals) is resolved to an absolute URL; instructions — one string of lines, or a list of strings, HowToSteps and HowToSections — are flattened in order; ISO-8601 prep/cook/total times become minutes, total derived as prep + cook when only those are given; the site name is the publisher's, else og:site_name, else the host. A page with no Recipe is ErrNoRecipe, with two is ErrManyRecipes; a shape schema.org does not allow — such as a non-ISO-8601 duration like "20 minutes" or an empty one, or an image @id no node has — is ErrMalformedRecipe, never guessed at.
• Recipe search (recipes.SearchProvider: Search(ctx, query, n) → []Candidate{URL, Title, Snippet}, n 1–20) appends " recipe" to a query without that word. SearXNG is called at {recipe_search_settings.base_url}/search?q=…&format=json&categories=general — directly, not through the Fetcher, since this configured endpoint may be on the internal network; Brave calls the Brave Web Search API with the key decrypted from recipe_search_settings.api_key_ct (recipes.NewSearchProvider). Redirects are not followed; 401/403 is ErrSearchRefused, 429 ErrSearchRateLimited, other non-2xx a SearchStatusError, and a body that is not the provider's JSON (SearXNG without a results array; Brave without "type": "search" or with a web object lacking results — Brave omits web when nothing matched), over 1 MB or with a result lacking an absolute http(s) URL ErrSearchResponse.
• Ranking (recipes.Rank) fetches and extracts the top 5 candidates concurrently, rejects any whose recipe lacks ingredients or an image, and orders the rest by the token similarity of the recipe's title to the dish name (harness.Similarity, 0–1) plus a quarter for each of ingredients, image, steps and total minutes; ties keep search order. No unranked candidate is dropped silently: when some are ranked, they are returned with ErrCandidatesUnranked; when none are, the error is ErrNoUsableRecipe. Either is joined with each unranked candidate's URL and reason (fetch failure, no valid recipe, missing ingredients or image), so the caller decides per item what a failure means.
• Ingredient lines are normalized by the normalize_recipe v1 skill (skills.NormalizeRecipe), always at temperature 0 whatever the household's setting. From the title, raw ingredient lines, steps and JSON-LD times it returns {ingredients: [{name, qty, unit, note}], method, difficulty, minutes_total}. Validation requires exactly one ingredient per input line, in order (a name matching another line but not its own is out of order), each with a name; a mismatch gets the harness's repair turn. Code-derived values win: a positive JSON-LD total is minutes_total; "slow cooker" or "crock pot" in the title or a step makes the method crockpot; a difficulty the model leaves null is derived — hard over 10 steps or 90 minutes, easy at 5 steps or fewer and 45 minutes or less, else medium, a crockpot's minutes not counting.
• Photos are cached locally (photos) and processed with golang.org/x/image. recipes.PhotoCache.DownloadPhoto downloads through the Fetcher's FetchImage (JPEG, PNG, WebP or GIF content types, 8 MB cap, not cached), decodes the image (a GIF's first frame) and refuses a decoded image under 300 px on its short side, recording nothing. CachePhoto then records it in the caller's transaction: it scales the image so its long side is 1200 px and 400 px, and stores each as JPEG quality 82 at $MEALQUEST_DATA_DIR/media/photos/{id}-{size}.jpg. Images are deduplicated by the SHA-256 of the downloaded bytes; the writer's single connection serialises transactions, so one image is never recorded twice. Files written for a transaction that rolls back are removed by the caller (RemovePhotoFiles).
• Grounding (recipes.Grounder) ties these together. Ground(ctx, dishName, query) searches for the top 5 candidates and ranks them all, stored and new alike: a candidate whose source_url is already stored is scored on its stored recipe (title, ingredients, steps, times, photo source) and its page is never refetched; every other candidate is fetched and extracted. Ranking alone decides between them. It then tries each ranked candidate best first. A stored one is returned as it is, so a stored recipe is reused only when it ranks above every new candidate that grounds. A new one is grounded in this order: (1) DownloadPhoto — the download and validation step of photo caching — (2) normalize_recipe, then (3) one transaction that runs CachePhoto (scales and writes the photo files and records the photos row) and stores the recipe, ingredients and steps (an existing source_url is updated in place); files written for a rolled-back transaction are removed. It returns Grounded{recipe_id, photo_id, method, minutes_total, difficulty, source_url, site_name}. A candidate whose photo fails or whose model output is unusable (llm.ErrBadOutput) is passed over; when none grounds, the error is a typed *recipes.ErrUngroundable listing every reason (no candidates, Rank's per-candidate reasons, photo or normalize failures). Failures that are not the candidates' — search, model provider, database — are returned as they are. GroundURL(ctx, url) does the same for one pasted URL. GroundMany grounds items 3 at a time and returns one result per item, so one failure never affects its siblings. A recipe is usable only with ingredients, an image and, if given, a positive total time.

Scheduler

Times are in the household timezone. The times of day_of_reminder (reminder_time) and rotation_check (review_time), and the day and time of suggestions_weekly, plan_draft and shopping_chat_autosend, are household settings in household_settings (dinner_time is also stored there); the table shows their defaults. week_rollover and the ai_runs purge are fixed and not configurable.

Schedule	When	Does
suggestions_weekly	Sunday 08:00	New batch of Side quests.
plan_draft	Friday 09:00	AI drafts next week's plan.
shopping_chat_autosend	Saturday 08:00	Sends the Supply run to Google Chat.
day_of_reminder	Daily 16:30	Tonight's dinner reminder.
rotation_check	Daily 09:00	Asks "Keep it in rotation?" for tried dinners.
week_rollover	Monday 00:05 (fixed)	Freezes last week's Supply run into the Supply archive.
ai_runs purge	Daily (fixed)	Calls harness.Purge to delete ai_runs rows past 30-day retention (Feature 20).

Runs are recorded in schedule_runs and executed through jobs.

Notifications

• Events: plan_ready, list_ready, day_of_reminder, new_suggestions, rotation_review; each can go to Web Push and/or Google Chat per notification_prefs.
• Delivery goes through notification_outbox.
• Web Push uses VAPID (webpush-go). Google Chat uses the stored incoming webhook.

Google Calendar

• One-way push: the app is the source of truth; edits in Google Calendar do not come back.
• Connected once with Google OAuth in Settings; one event per planned dinner.
• Deterministic event ids; each plan day records its calendar_event_id.

Frontend

• React 19, TypeScript, Vite, React Router 7, TanStack Query 5, vite-plugin-pwa, lucide-react (stroke 2.75).
• CSS Modules over tokens.css; no Tailwind, no Framer Motion, no hard-coded hex. Self-hosted Bagel Fat One and Figtree.
• Tests: Vitest, Testing Library, jsdom.
• Offline cache for the current plan, shopping list and recipes.
• Logout (POST /auth/logout) answers with Clear-Site-Data: "storage", "cache", wiping the offline marker, the service worker and its caches; the worker leaves that request to the browser, as it does every network-only one.

Route	Screen
/	Home
/onboarding	Add regular dinners
/plan	Weekly plan (Draft board)
/plan/shopping	Shopping list (Supply run)
/plan/shopping/archive	Supply archive
/discover	Swipe suggestions (Side quests)
/quick	Quick dinners
/cook-from-pantry	Cook from what I have
/dinner/:id	Recipe / dinner detail (Kitchen reading mode on tablet)
/settings	Settings home
/settings/household	Household settings
/settings/regulars	Regular dinners
/settings/calendar	Google Calendar connection
/settings/notifications	Notifications and Google Chat
/settings/ai	Admin · AI assistant

/settings/household, /settings/regulars and /settings/calendar come from CTXD; the other routes are defined in the UI handoff.

Boundaries and gotchas

• Secrets never reach the client — API keys, OAuth tokens and the Chat webhook stay server-side, encrypted at rest.
• The app is the source of truth for the plan; Google Calendar is a one-way mirror.
• Never auto-promote to Regular (D-021).
• Quick means cooked in a crock pot or ≤ 30 minutes prep to table.
• Weeks run Monday–Sunday in the household timezone.
• No silent fallbacks — fail fast and report.
• iOS Web Push requires the PWA to be installed to the Home Screen; show an install hint.