Compare commits

..
Author SHA1 Message Date
dev_agentandClaude Opus 5.5 32453d552b Test photo backfill single-flight under simultaneous triggers
CI / ci (push) Successful in 9m54s
CI / ci (pull_request) Successful in 9m55s
TestStartPhotoBackfillIsSingleFlight now releases eight
StartPhotoBackfill calls together behind a start gate, both while the
pass is queued and while it runs, and asserts they all return the one
job id with a single backfill job stored. The restart case closes the
store and reopens it from the data directory before triggering again.

Maestro task 36c40130-a69a-46ed-a047-2cdc5ee2d2e0 (review finding F23-1-T).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 21:34:09 -04:00
dev_agentandClaude Opus 5.5 7c0eb5cc3a Make photo backfill single-flight and fail on unrecorded writes
CI / ci (pull_request) Successful in 10m6s
CI / ci (push) Successful in 10m10s
Task 36c40130-a69a-46ed-a047-2cdc5ee2d2e0, remediation of review
findings F23-1 and F23-2 (PR #205).

F23-1: StartPhotoBackfill now checks, in the same write transaction
that enqueues, for a full backfill_dinner_photos job already queued or
running (new GetActivePhotoBackfillJob query) and returns its id instead
of enqueuing another, as plan.StartDraft does with GetActiveDraftJob.
The claim is the stored job row, so it holds across restarts; a job left
running by a stopped process is failed at boot and no longer blocks.

F23-2: a dinner whose outcome write fails is still reported failed with
grounding_error and the remaining dinners still run, but the job then
fails (internal_error, as a ground_dinner job's failed write does) with
the per-dinner outcomes published as its result, so it never reports
success for state it did not store.

Tests: overlapping triggers while queued, after a restart and while
running return the one job; a refused dinner write fails the job.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 21:16:13 -04:00
dev_agentandClaude Opus 5.5 ec718024a1 Add photo backfill for failed dinners without a photo
CI / ci (pull_request) Failing after 10m12s
CI / ci (push) Successful in 11m35s
Task 36c40130-a69a-46ed-a047-2cdc5ee2d2e0 (Feature 23, task 1 of 2).

A backfill_dinner_photos job grounds, by name with the existing
recipes.Grounder.Ground, every unarchived dinner whose grounding failed
and that has no photo and no pasted recipe_url. Ground already tries the
ranked candidates best first and keeps the first whose photo caches and
whose ingredients normalize, reusing stored recipes and photos. Success
records the recipe and photo as grounded; a failure leaves the dinner
failed with the new grounding_error for the fix-recipe flow, without
stopping its siblings. Writes are guarded on status, photo and
grounding_revision, so grounded, manual, pending, renamed and archived
dinners are never touched.

POST /api/v1/settings/photo-backfill starts a full pass and answers 202
with the job id; the kind is service-only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 20:53:58 -04:00
18 changed files with 14 additions and 1287 deletions
+1 -1
View File
@@ -84,7 +84,7 @@ behaviour, exercise it (hit the endpoint, render the page).
| `internal/*` (planned) | `auth`, `db`, `store`, `secretbox`, `household`, `dinners`, `recipes`, `ai/llm`, `ai/harness`, `ai/skills`, `suggest`, `plan`, `shopping`, `ideas`, `gcal`, `notify`, `push`, `chat`, `jobs`, `scheduler`. |
| `migrations/` | SQL migrations (planned). |
| `web/` | React PWA source. |
| `deploy/` | Deployment files: compose stack, systemd units, [deploy runbook](deploy/RUNBOOK.md) and the automated deploy (`auto-deploy.sh` with `systemd/mealquest-auto-deploy.*`). |
| `deploy/` | Deployment files (planned). |
| `docs/` | Project documentation, including [ARCHITECTURE.md](docs/ARCHITECTURE.md). |
| `design/` | Requirements and UI handoff. **Read-only reference** — do not edit. |
| `Dockerfile` | Multi-stage image build, at the repository root. |
-357
View File
@@ -1,357 +0,0 @@
# MealQuest deploy runbook
The one way to deploy MealQuest to production: the docker compose project
`mealquest` on office-srvr `10.60.1.14` (see
[ARCHITECTURE.md](../docs/ARCHITECTURE.md#topology)). Production runs `main`
or a release tag; only `overseer` merges `dev` → `main`. Every deploy builds
on the host, from the checked-out commit, with `make docker`.
Host setup (clone, firewall, the systemd unit, Traefik, DNS, Litestream) and
the database restore are in CTXD project `mealquest`, `RUNBOOKS.MD`. This
file is the build, restart, verification and rollback procedure; where it
uses a CTXD section, it names it.
All commands run on `10.60.1.14` as `overseer`, in bash, from the checkout
`/home/overseer/mealquest`. A block that ends with `FAILED` or a non-zero
`$?` stops the deploy: do not go on to the next step.
## How the image carries its version
`make docker` runs `docker build --build-arg VERSION=<version>` and tags the
image `mealquest:<version>`, where `<version>` is
`git describe --tags --always` of the checkout (a release tag such as
`v1.0.0`, otherwise the short SHA), with `-dirty` added when a tracked file
is modified. The [Dockerfile](../Dockerfile) links it into the binary, and
`/health` reports it:
```json
{"status":"ok","version":"v1.0.0"}
```
Never pass `--build-arg VERSION` by hand, and never build with
`docker compose build` or `docker compose up --build`: compose passes no
version, so that image reports `dev`.
The compose service `mealquest` runs the image named `mealquest-mealquest`
(compose's name for the image it would build; check with
`docker compose -f deploy/compose.yaml config --images mealquest`). It
already exists, so `docker compose up` uses it and never builds. A deploy
points that name at the `make docker` image with `docker tag`. Every
`mealquest:<version>` image is kept for rollback; do not prune them.
## Prerequisites
- `make`, `git`, `curl`, `python3` and Docker Engine with Compose on the
host (`make` is not in the base install: `sudo apt-get install make`).
- `deploy/.env` (git-ignored, owner `overseer`, mode 0600) sets every
variable [`deploy/.env.example`](.env.example) sets, with
`MEALQUEST_BIND_ADDRESS=10.60.1.14`. It must not set the variables
`deploy/compose.yaml` fixes (`MEALQUEST_HTTP_ADDR`, `MEALQUEST_DATA_DIR`,
`MEALQUEST_SEARXNG_URL` and the `*_FILE` variables). Creating it: CTXD
`RUNBOOKS.MD` "Stack and first deploy" > "3. Environment file".
- `deploy/secrets/` (git-ignored, mode 0700) holds `secret_key`,
`oidc_client_secret`, `google_client_secret`, `vapid_private_key` and
`litestream_key`, each owned by `10001:10001` with mode 0600, and
`searxng.env`, owned by `overseer` with mode 0600. Creating them: CTXD
`RUNBOOKS.MD` "Stack and first deploy" > "4. Secrets".
- The compose file is the checkout's [`deploy/compose.yaml`](compose.yaml);
the systemd unit [`mealquest.service`](systemd/mealquest.service) starts
and stops it and is the only way the stack starts.
- DNS and Traefik already route `https://mealquest.cubecraftlabs.com` to
`10.60.1.14:8090` (CTXD `RUNBOOKS.MD` "Reverse proxy (Traefik)", "DNS").
Check `deploy/.env` and the secrets before every deploy (a new release may
add a variable; the service refuses to boot without it). Both commands must
print nothing:
```sh
cd /home/overseer/mealquest
comm -23 <(grep -oE '^MEALQUEST_[A-Z_]+' deploy/.env.example | sort -u) \
<(grep -oE '^MEALQUEST_[A-Z_]+' deploy/.env | sort -u) # variables missing from deploy/.env
sudo find deploy/secrets -type f ! -perm 600 # secrets not mode 0600
```
## Deploy
### 1. Build
Record the release production is running now, check out the new release
and build its image. Nothing restarts.
The rollback target is the version the running service reports on
`/health`, read before the checkout changes, never the checkout's `HEAD`:
after a failed or deferred deploy the checkout can be ahead of production.
The block resolves that version to a commit and prints it as the rollback
SHA. A running version of `dev` or `*-dirty` names no commit: the block
fails, and the rollback target must be established by hand before going on.
The checkout must be clean (the same test as the Makefile's `GIT_DIRTY`:
no modified tracked file outside the generated `internal/frontend/dist`).
The block computes the image tag exactly as `make docker` does, `-dirty`
included, and checks that image exists after the build.
```sh
cd /home/overseer/mealquest && (
set -euo pipefail
fail() { echo "$*" >&2; exit 1; }
built=
trap 'st=$?
if [ "$st" -eq 0 ] && [ -n "$built" ]; then echo "built: $built"
else [ "$st" -ne 0 ] || st=1; echo "build FAILED (status $st)" >&2; fi
exit "$st"' EXIT
bind=$(grep -E '^MEALQUEST_BIND_ADDRESS=' deploy/.env | cut -d = -f 2-)
[ -n "$bind" ] || fail "deploy/.env sets no MEALQUEST_BIND_ADDRESS"
running=$(curl -fsS "http://$bind:8090/health" | python3 -c 'import json, sys; print(json.load(sys.stdin)["version"])') \
|| fail "http://$bind:8090/health did not return a version: no running release to roll back to"
case "$running" in
dev|*-dirty) fail "the running release reports $running, which names no commit" ;;
esac
git fetch --tags origin
rollback=$(git rev-parse --verify --quiet "$running^{commit}") \
|| fail "running version $running is not a commit in this repository"
echo "rollback SHA: $rollback (running version $running)" # RECORD it for rollback
git checkout --detach "refs/remotes/origin/main^{commit}" # or the release tag, e.g. v1.0.0
dirty=$(git status --porcelain --untracked-files=no -- . ':(exclude)internal/frontend/dist')
[ -z "$dirty" ] || fail "the checkout has modified tracked files; clean it first:
$dirty"
v=$(git describe --tags --always)${dirty:+-dirty} # the Makefile's VERSION
make docker
docker image inspect "mealquest:$v" > /dev/null 2>&1 || fail "no image mealquest:$v: the build failed"
built="mealquest:$v"
)
```
### 2. Ship and restart
1. **Pre-restart gates.** Run CTXD `RUNBOOKS.MD` "Routine deploy and
rollback" > "Deploy" step 2 (a verified pre-deploy restore point; record
it with the rollback SHA) and step 3 (boot-required configuration). The
restart applies migrations the previous release refuses, so without that
restore point there is no rollback. If either fails, do not restart:
production stays on the previous release.
2. **Point the stack at the new image and restart.** `docker compose pull`
fetches the pinned `litestream` and `searxng` images before anything
stops; `mealquest` is never pulled. The restart applies the firewall,
then runs `down` and `up -d`, and returns once `mealquest` passes its
healthcheck (`/health` inside the container).
```sh
cd /home/overseer/mealquest && (
set -euo pipefail
fail() { echo "$*" >&2; exit 1; }
up=
trap 'st=$?
if [ "$st" -eq 0 ] && [ -n "$up" ]; then echo "restarted on: $up"
else [ "$st" -ne 0 ] || st=1; echo "restart FAILED (status $st)" >&2; fi
exit "$st"' EXIT
v=$(git describe --tags --always)
docker compose -f deploy/compose.yaml pull litestream searxng
docker tag "mealquest:$v" mealquest-mealquest
sudo systemctl restart mealquest
for s in mealquest litestream searxng; do
id=$(docker compose -f deploy/compose.yaml ps --status running --quiet "$s")
[ -n "$id" ] || fail "$s is not running"
done
id=$(docker compose -f deploy/compose.yaml ps --quiet mealquest)
health=$(docker inspect -f '{{.State.Health.Status}}' "$id")
[ "$health" = healthy ] || fail "mealquest is $health, not healthy"
up="mealquest:$v"
)
```
On failure read `journalctl -u mealquest` and
`docker compose -f deploy/compose.yaml logs mealquest`: a configuration
error names the variable. Fix it or roll back.
### 3. Verify the deployed version and a real screen
A `/health` 200 alone proves only that some build is up. This step checks,
on the LAN address and through Traefik, that:
- `/health` reports exactly the checkout's `git describe --tags --always`;
- `/` and the Plan screen's route `/plan` serve the built PWA shell (the
React root and a hashed `/assets/` script, not the placeholder page an
unbuilt embed serves);
- that script is served as JavaScript.
```sh
cd /home/overseer/mealquest && (
set -euo pipefail
fail() { echo "$*" >&2; exit 1; }
want=
trap 'st=$?
if [ "$st" -eq 0 ] && [ -n "$want" ]; then echo "deployed and verified: $want ($(git rev-parse HEAD))"
else [ "$st" -ne 0 ] || st=1; echo "verification FAILED (status $st): the deploy is not verified" >&2; fi
exit "$st"' EXIT
v=$(git describe --tags --always)
for base in http://10.60.1.14:8090 https://mealquest.cubecraftlabs.com; do
got=$(curl -fsS "$base/health" | python3 -c 'import json, sys; print(json.load(sys.stdin)["version"])') \
|| fail "$base/health did not return a version"
[ "$got" = "$v" ] || fail "$base/health reports version $got, the checkout is $v"
for path in / /plan; do
page=$(curl -fsS "$base$path") || fail "$base$path did not return 200"
grep -q '<div id="root"></div>' <<< "$page" || fail "$base$path is not the PWA shell"
js=$(grep -oE 'src="/assets/[^"]+\.js"' <<< "$page" | head -n 1 | cut -d '"' -f 2) || true
[ -n "$js" ] || fail "$base$path references no built /assets/ script"
type=$(curl -fsS -o /dev/null -w '%{content_type}' "$base$js") || fail "$base$js did not return 200"
case "$type" in
text/javascript*) ;;
*) fail "$base$js is $type, not JavaScript" ;;
esac
done
echo "$base: version $got, PWA shell and $js served"
done
want=$v
)
```
- `deployed and verified: <version> (<SHA>)` and `$?` = `0` → open
`https://mealquest.cubecraftlabs.com` in a browser: sign in and the Home
screen renders with the household's data. Then record the deployed
version and SHA, the rollback SHA, the pre-deploy restore point and the
date in the release's Maestro task.
- `verification FAILED`, any error or a non-zero `$?` → the deploy is not
done. A version mismatch means the stack runs a different image than the
checkout: repeat step 2, or roll back.
## Rollback to a previous release
Rolling back points `mealquest-mealquest` at the previous release's
`mealquest:<version>` image again; that release's checkout keeps the
compose file and the version check consistent with it.
1. **Check whether the rollback crosses a migration** with CTXD
`RUNBOOKS.MD` "Routine deploy and rollback" > "Rollback to a previous
SHA" (its first block), run against the rollback SHA that deploy step 1
recorded. If it prints versions unknown to the rollback SHA, restore the
database to the pre-deploy restore point first (CTXD `RUNBOOKS.MD`
"Restore the database", with `-timestamp <restore point>`); changes
since then are lost, so tell the household.
2. **Check out the rollback SHA and retag its image.** An image built before
this runbook was not tagged `mealquest:<version>`; `make docker` builds
it from the same commit.
```sh
cd /home/overseer/mealquest && (
set -euo pipefail
fail() { echo "$*" >&2; exit 1; }
P=<rollback SHA> # recorded by step 1, from the running release's /health
back=
trap 'st=$?
if [ "$st" -eq 0 ] && [ -n "$back" ]; then echo "rolled back to: $back"
else [ "$st" -ne 0 ] || st=1; echo "rollback FAILED (status $st)" >&2; fi
exit "$st"' EXIT
git checkout --detach "$P"
v=$(git describe --tags --always)
docker image inspect "mealquest:$v" > /dev/null 2>&1 || make docker
docker image inspect "mealquest:$v" > /dev/null 2>&1 || fail "no image mealquest:$v"
docker tag "mealquest:$v" mealquest-mealquest
sudo systemctl restart mealquest
back="mealquest:$v"
)
```
3. **Run step 3 above** unchanged. The rollback is done only when it prints
`deployed and verified:` with the previous release's version.
## Automated deploys
[`auto-deploy.sh`](auto-deploy.sh), installed as
`/usr/local/sbin/mealquest-auto-deploy`, runs this runbook's deploy unattended:
step 1 (build), the prerequisite `deploy/.env` and secrets check, CTXD
`RUNBOOKS.MD` "Deploy" steps 2 (pre-deploy restore point) and 3
(boot-required configuration), step 2 (ship and restart) and step 3
(verify). No check is skipped or weakened, and it stops at the first
failure. Every failure before the restart leaves production on the previous
release.
### What runs, and when
[`mealquest-auto-deploy.timer`](systemd/mealquest-auto-deploy.timer) starts
[`mealquest-auto-deploy.service`](systemd/mealquest-auto-deploy.service)
at every quarter hour (`OnCalendar=*-*-* *:0/15:00`), plus up to 5 minutes'
random delay; a check missed while the host was down runs as soon as it is
back (`Persistent=true`). The service runs the script as `overseer` from the
checkout, and only while `mealquest.service` is active: a stack stopped on
purpose (a database or media restore) is skipped, not redeployed. Output goes
to the journal (`journalctl -u mealquest-auto-deploy`).
Each run fetches `origin` and compares `origin/main` with the last
`deployed=` SHA in the deploy record `/home/overseer/mealquest-auto-deploy.log`:
| `origin/main` | Result |
| --- | --- |
| is the last deployed SHA | No-op: exit 0, no output. |
| CI (Gitea commit status) `pending` | Exit 0, no output; the next run checks again. |
| CI `failure`, `error`, `warning`, or no status yet | Error in the journal, exit 1, nothing deployed. |
| the last record line is `FAILED` for this SHA | Error in the journal, exit 1, nothing deployed: a failed deploy is never retried by the timer. |
| CI `success` | Deploy. |
The CI status is read from
`https://code.cubecraftlabs.com/api/v1/repos/Maniacs_Mansion/MealQuest/commits/<SHA>/status`
(public; no credential). A commit merged moments before a run can briefly
report no status, which fails that run; the next run reads it again.
A deploy appends one line to the record:
```text
2026-10-12T15:04:05Z previous=<rollback SHA> deployed=<SHA> restore_point=2026-10-12T15:01:59Z
```
`previous` is the rollback SHA (from the running release's `/health`,
never the checkout) and `restore_point` the verified pre-deploy restore
point: record both in the release's Maestro task. A failed deploy appends
`<date> FAILED target=<SHA> status=<exit status>` and logs
`auto-deploy FAILED (status <n>)` to the journal, adding whether it failed
before the restart (production untouched) or after it (read
`journalctl -u mealquest`, then fix or roll back). Either way the checkout is
left at the target commit; the running release on `/health` stays the
rollback target.
### Install on the host
The script and units are installed copies: repeat this whenever a release
changes `deploy/auto-deploy.sh` or the units. From the checkout, at a release
that contains them:
```sh
cd /home/overseer/mealquest
sudo install -m 0755 deploy/auto-deploy.sh /usr/local/sbin/mealquest-auto-deploy
sudo install -m 0644 deploy/systemd/mealquest-auto-deploy.service \
deploy/systemd/mealquest-auto-deploy.timer /etc/systemd/system/
sudo systemctl daemon-reload
```
The timer refuses to deploy until the record holds a deploy, so seed it with
one manual deploy of the release production should run (below), then start
the timer:
```sh
sudo /usr/local/sbin/mealquest-auto-deploy --ref refs/remotes/origin/main
sudo systemctl enable --now mealquest-auto-deploy.timer
systemctl list-timers mealquest-auto-deploy.timer
```
To stop automated deploys: `sudo systemctl disable --now mealquest-auto-deploy.timer`.
### Manual override
```sh
sudo /usr/local/sbin/mealquest-auto-deploy --ref <ref>
```
deploys an explicit ref (a SHA, tag or `origin/<branch>`) through the same
blocks, bypassing the CI and no-op gates: for a roll-forward fix, a re-run
after a `FAILED` record, or seeding the record. Run as root, the script
re-executes itself as `overseer`; it holds a lock, so it never runs alongside
the timer's run. It prints `deployed and verified:` and exits 0 only when
step 3 passed.
### Rollback
Rollback is not automated: follow "Rollback to a previous release" above
and CTXD `RUNBOOKS.MD` "Routine deploy and rollback" > "Rollback to a
previous SHA", using the `previous` SHA and `restore_point` from the record.
The timer does not undo a manual rollback: `origin/main` is still the last
recorded deploy, so its runs are no-ops until `main` moves, and a new `main`
then deploys as usual. Disable the timer if the rollback must hold.
-251
View File
@@ -1,251 +0,0 @@
#!/usr/bin/env bash
# MealQuest automated deploy: deploy/RUNBOOK.md, run unattended.
#
# mealquest-auto-deploy deploy origin/main if it is new and CI passed
# mealquest-auto-deploy --ref <ref> deploy <ref> now, bypassing the CI and no-op gates
#
# Installed as /usr/local/sbin/mealquest-auto-deploy and run as overseer by
# mealquest-auto-deploy.service (deploy/systemd); install, semantics and the
# manual override: deploy/RUNBOOK.md "Automated deploys". Run as root (sudo)
# it re-executes itself as overseer.
#
# Without --ref: exit 0 with no output when origin/main is the last deployed
# SHA in the deploy record or its CI is pending; exit 1, deploying nothing,
# when its CI failed or reports no status, or when the last attempt to deploy
# that SHA failed. Every deploy runs the runbook's blocks in order and stops
# at the first failure; nothing before the restart touches production. A
# verified deploy appends one line to the deploy record:
# <UTC date> previous=<SHA> deployed=<SHA> restore_point=<UTC date>
# a failed one appends:
# <UTC date> FAILED target=<SHA> status=<exit status>
set -euo pipefail
CHECKOUT=/home/overseer/mealquest
RECORD=/home/overseer/mealquest-auto-deploy.log
LOCK=/home/overseer/mealquest-auto-deploy.lock
STATUS_API=https://code.cubecraftlabs.com/api/v1/repos/Maniacs_Mansion/MealQuest/commits
PUBLIC=https://mealquest.cubecraftlabs.com
D=/var/lib/docker/volumes/mealquest-data/_data
COMPOSE=(docker compose -f deploy/compose.yaml)
fail() { echo "$*" >&2; exit 1; }
if [ "$(id -u)" -eq 0 ]; then
exec sudo -u overseer -H -- "$0" "$@"
fi
[ "$(id -un)" = overseer ] || fail "run as overseer (or root), not $(id -un)"
ref=
case "$#" in
0) ;;
2) [ "$1" = --ref ] && [ -n "$2" ] || fail "usage: $0 [--ref <ref>]"; ref=$2 ;;
*) fail "usage: $0 [--ref <ref>]" ;;
esac
exec 9>> "$LOCK"
flock -n 9 || fail "another mealquest-auto-deploy run holds $LOCK"
cd "$CHECKOUT"
git fetch --quiet --tags origin || fail "git fetch origin failed"
if [ -n "$ref" ]; then
target=$(git rev-parse --verify --quiet "$ref^{commit}") || fail "$ref is not a commit in $CHECKOUT"
else
# The full remote-tracking ref: a bare origin/main resolves a tag or local
# branch of that name first, which could deploy a commit not on main.
target=$(git rev-parse --verify --quiet "refs/remotes/origin/main^{commit}") \
|| fail "refs/remotes/origin/main is not a commit in $CHECKOUT"
[ -f "$RECORD" ] || fail "no deploy record at $RECORD: seed it with one manual deploy (deploy/RUNBOOK.md \"Automated deploys\")"
last=$(grep -oE 'deployed=[0-9a-f]{40}' "$RECORD" | tail -n 1 | cut -d = -f 2) || true
[ -n "$last" ] || fail "$RECORD records no deploy: seed it with one manual deploy (deploy/RUNBOOK.md \"Automated deploys\")"
[ "$target" != "$last" ] || exit 0
if tail -n 1 "$RECORD" | grep -q " FAILED target=$target status="; then
fail "auto-deploy of $target already failed (see $RECORD and the journal): fix it, then deploy with --ref"
fi
state=$(curl -fsS --max-time 30 "$STATUS_API/$target/status" \
| python3 -c 'import json, sys; d = json.load(sys.stdin); print(d["state"] if d["total_count"] else "absent")') \
|| fail "could not read the CI status of $target from $STATUS_API"
case "$state" in
success) ;;
pending) exit 0 ;;
*) fail "CI for origin/main $target is $state, not success: not deploying" ;;
esac
fi
echo "auto-deploy: deploying $target${ref:+ (--ref $ref)}"
restarted=
W=
on_exit() {
local st=$?
[ -z "$W" ] || rm -rf "$W" || st=1
if [ "$st" -ne 0 ]; then
echo "$(date -u +%Y-%m-%dT%H:%M:%SZ) FAILED target=$target status=$st" >> "$RECORD"
if [ -n "$restarted" ]; then
echo "auto-deploy FAILED (status $st) after the restart: production may be down or unverified; read journalctl -u mealquest, then fix or roll back (deploy/RUNBOOK.md)" >&2
else
echo "auto-deploy FAILED (status $st): production untouched" >&2
fi
fi
exit "$st"
}
trap on_exit EXIT
# deploy/RUNBOOK.md "Deploy" > "1. Build": record the rollback SHA from the
# RUNNING release, check out the target, reject a dirty checkout, build.
bind=$(grep -E '^MEALQUEST_BIND_ADDRESS=' deploy/.env | cut -d = -f 2-) || fail "deploy/.env sets no MEALQUEST_BIND_ADDRESS"
[ -n "$bind" ] || fail "deploy/.env sets no MEALQUEST_BIND_ADDRESS"
running=$(curl -fsS "http://$bind:8090/health" | python3 -c 'import json, sys; print(json.load(sys.stdin)["version"])') \
|| fail "http://$bind:8090/health did not return a version: no running release to roll back to"
case "$running" in
dev|*-dirty) fail "the running release reports $running, which names no commit" ;;
esac
rollback=$(git rev-parse --verify --quiet "$running^{commit}") \
|| fail "running version $running is not a commit in this repository"
echo "rollback SHA: $rollback (running version $running)"
git checkout --quiet --detach "$target"
dirty=$(git status --porcelain --untracked-files=no -- . ':(exclude)internal/frontend/dist')
[ -z "$dirty" ] || fail "the checkout has modified tracked files; clean it first:
$dirty"
v=$(git describe --tags --always)${dirty:+-dirty} # the Makefile's VERSION
make docker
docker image inspect "mealquest:$v" > /dev/null 2>&1 || fail "no image mealquest:$v: the build failed"
echo "built: mealquest:$v"
# deploy/RUNBOOK.md "Prerequisites": deploy/.env sets every variable of the
# target's .env.example, and every secret is mode 0600.
missing=$(comm -23 <(grep -oE '^MEALQUEST_[A-Z_]+' deploy/.env.example | sort -u) \
<(grep -oE '^MEALQUEST_[A-Z_]+' deploy/.env | sort -u))
[ -z "$missing" ] || fail "deploy/.env is missing: $missing"
loose=$(sudo find deploy/secrets -type f ! -perm 600)
[ -z "$loose" ] || fail "secrets not mode 0600: $loose"
# CTXD RUNBOOKS.MD "Routine deploy and rollback" > "Deploy" step 2: a verified
# pre-deploy restore point. The scripts are CTXD "Count check". They and the
# check files live in a private mktemp directory (mode 0700), not at shared
# /tmp paths, and python runs with -I: the snapshot runs as root, and a
# shared directory on its import path would let any local user run code as
# root.
W=$(mktemp -d)
cat > "$W/mqcount.py" <<'EOF'
import sqlite3, sys, hashlib
c = sqlite3.connect(sys.argv[1])
tabs = [r[0] for r in c.execute("select name from sqlite_master where type=\"table\" and name not like \"sqlite_%\" and name not like \"_litestream%\" order by name")]
for t in ("dinners", "week_plans", "schema_migrations"):
print(t, c.execute(f"select count(*) from {t}").fetchone()[0])
h = hashlib.sha256()
for t in tabs:
for row in c.execute(f"select * from \"{t}\" order by rowid"):
h.update(repr(row).encode())
print("tables", len(tabs), "content-sha256", h.hexdigest()[:16])
print("integrity", c.execute("pragma integrity_check").fetchone()[0])
EOF
cat > "$W/mqsnap.py" <<'EOF'
import os, sqlite3, sys
src, dst = sys.argv[1], sys.argv[2]
if os.path.exists(dst):
sys.exit(f"{dst} already exists")
s = sqlite3.connect(f"file:{src}?mode=ro", uri=True)
d = sqlite3.connect(dst)
s.backup(d)
s.close()
result = d.execute("pragma integrity_check").fetchall()
d.close()
if result != [("ok",)]:
sys.exit(f"snapshot integrity_check failed: {result}")
print("snapshot", dst, "integrity ok")
EOF
point=$(
set -eu
point=
trap 'st=$?
rm -rf "$W" || st=1
sudo sh -c "rm -f $D/predeploy-check.db*" || st=1
if [ "$st" -eq 0 ] && [ -n "$point" ]; then echo "pre-deploy restore point: $point" >&2; echo "$point"
else [ "$st" -ne 0 ] || st=1; echo "verification FAILED (status $st): do not restart" >&2; fi
exit "$st"' EXIT
exec 3>&1 1>&2 # block output to the journal; only the restore point to $point
mkdir "$W/mqcheck"
# a. Replication running, no log errors (grep: 0 match, 1 none, 2+ error), newest LTX listed
id=$("${COMPOSE[@]}" ps --status running --quiet litestream) || fail "docker compose ps failed"
[ -n "$id" ] || fail "litestream is not running"
"${COMPOSE[@]}" logs --since 24h litestream > "$W/mqcheck/litestream.log" || fail "docker compose logs failed"
gs=0
grep -i error "$W/mqcheck/litestream.log" > "$W/mqcheck/litestream.errors" || gs=$?
case "$gs" in
0) cat "$W/mqcheck/litestream.errors"; fail "litestream logged errors in the last 24h";;
1) ;;
*) fail "the litestream log scan failed (status $gs)";;
esac
"${COMPOSE[@]}" exec -T litestream \
litestream ltx -config /etc/litestream.yml -level all /data/mealquest.db
# b. Restore replica to a scratch file (live files untouched), move off the volume, snapshot live
"${COMPOSE[@]}" run --rm --no-deps litestream \
restore -config /etc/litestream.yml -o /data/predeploy-check.db /data/mealquest.db
sudo sh -c "mv $D/predeploy-check.db* $W/mqcheck/"
sudo python3 -I "$W/mqsnap.py" "$D/mealquest.db" "$W/mqcheck/live.db"
sudo chown -R "$(id -un)" "$W/mqcheck"
sudo ls -ln "$D"
# mealquest.db-wal and -shm owned by 10001, never 0 (CTXD "Count check")
root=$(sudo find "$D" -maxdepth 1 \( -name mealquest.db-wal -o -name mealquest.db-shm \) -uid 0)
[ -z "$root" ] || fail "owned by root, not 10001 (CTXD RUNBOOKS.MD \"Count check\"): $root"
# c. Restored replica: integrity ok and identical counts/content-sha256 to the live snapshot
python3 -I "$W/mqcount.py" "$W/mqcheck/predeploy-check.db" > "$W/mqcheck/restored.txt"
python3 -I "$W/mqcount.py" "$W/mqcheck/live.db" > "$W/mqcheck/live.txt"
cat "$W/mqcheck/restored.txt"
grep -qx 'integrity ok' "$W/mqcheck/restored.txt" || fail "the restored replica failed integrity_check"
diff "$W/mqcheck/restored.txt" "$W/mqcheck/live.txt" || fail "the restored replica differs from the live snapshot"
# d. Restore point: after every check, before the restart
point=$(date -u +%Y-%m-%dT%H:%M:%SZ)
exec 1>&3
)
# CTXD RUNBOOKS.MD "Routine deploy and rollback" > "Deploy" step 3:
# boot-required configuration, secrets read the way the service reads them.
[ -f deploy/.env ] || fail "missing deploy/.env"
cid=$(grep -m 1 -E '^MEALQUEST_GOOGLE_CLIENT_ID=' deploy/.env | cut -d = -f 2- | tr -d '"' | tr -d "'") \
|| fail "missing MEALQUEST_GOOGLE_CLIENT_ID in deploy/.env"
[ -n "$cid" ] || fail "missing MEALQUEST_GOOGLE_CLIENT_ID in deploy/.env"
[ -e deploy/secrets/google_client_secret ] || fail "missing deploy/secrets/google_client_secret"
[ -s deploy/secrets/google_client_secret ] || fail "deploy/secrets/google_client_secret is empty"
id=$("${COMPOSE[@]}" ps --status running --quiet mealquest) || fail "docker compose ps failed"
[ -n "$id" ] || fail "the mealquest container is not running: start it (the check reads secrets the way the service does)"
docker exec "$id" sh -c 'test -s /run/secrets/google_client_secret && test -r /run/secrets/google_client_secret' \
|| fail "google_client_secret is not readable by the service user inside the container"
echo "boot-required configuration present"
# deploy/RUNBOOK.md "2. Ship and restart".
"${COMPOSE[@]}" pull litestream searxng
docker tag "mealquest:$v" mealquest-mealquest
restarted=yes
sudo systemctl restart mealquest
for s in mealquest litestream searxng; do
id=$("${COMPOSE[@]}" ps --status running --quiet "$s")
[ -n "$id" ] || fail "$s is not running"
done
id=$("${COMPOSE[@]}" ps --quiet mealquest)
health=$(docker inspect -f '{{.State.Health.Status}}' "$id")
[ "$health" = healthy ] || fail "mealquest is $health, not healthy"
echo "restarted on: mealquest:$v"
# deploy/RUNBOOK.md "3. Verify the deployed version and a real screen".
for base in "http://$bind:8090" "$PUBLIC"; do
got=$(curl -fsS "$base/health" | python3 -c 'import json, sys; print(json.load(sys.stdin)["version"])') \
|| fail "$base/health did not return a version"
[ "$got" = "$v" ] || fail "$base/health reports version $got, the checkout is $v"
for path in / /plan; do
page=$(curl -fsS "$base$path") || fail "$base$path did not return 200"
grep -q '<div id="root"></div>' <<< "$page" || fail "$base$path is not the PWA shell"
js=$(grep -oE 'src="/assets/[^"]+\.js"' <<< "$page" | head -n 1 | cut -d '"' -f 2) || true
[ -n "$js" ] || fail "$base$path references no built /assets/ script"
type=$(curl -fsS -o /dev/null -w '%{content_type}' "$base$js") || fail "$base$js did not return 200"
case "$type" in
text/javascript*) ;;
*) fail "$base$js is $type, not JavaScript" ;;
esac
done
echo "$base: version $got, PWA shell and $js served"
done
deployed=$(git rev-parse HEAD)
echo "$(date -u +%Y-%m-%dT%H:%M:%SZ) previous=$rollback deployed=$deployed restore_point=$point" >> "$RECORD"
echo "deployed and verified: $v ($deployed), previous $rollback, restore point $point"
@@ -1,23 +0,0 @@
# Deploys origin/main to production when it is new and its CI passed
# (/usr/local/sbin/mealquest-auto-deploy, from deploy/auto-deploy.sh).
# Triggered by mealquest-auto-deploy.timer; output goes to the journal.
# Installing it and the manual override: deploy/RUNBOOK.md "Automated deploys".
#
# ExecCondition= skips the run while mealquest.service is not active, so a
# stack stopped on purpose (a database or media restore) is never redeployed
# under the operator. It is deliberately not Requires=/Requisite= on
# mealquest.service: those would start the stack, or stop this run when the
# deploy itself restarts mealquest.service. Runs as overseer, who owns the
# checkout and uses sudo for the runbook's privileged steps.
[Unit]
Description=MealQuest automated deploy of main
Wants=network-online.target
After=network-online.target docker.service mealquest.service
[Service]
Type=oneshot
User=overseer
Group=overseer
WorkingDirectory=/home/overseer/mealquest
ExecCondition=/usr/bin/systemctl is-active --quiet mealquest.service
ExecStart=/usr/local/sbin/mealquest-auto-deploy
@@ -1,13 +0,0 @@
# Checks for a new main at every quarter hour (plus up to 5 minutes' random
# delay). Persistent= runs a check at once when the host comes back if a
# scheduled one was missed while it was down.
[Unit]
Description=Periodic MealQuest automated deploy
[Timer]
OnCalendar=*-*-* *:0/15:00
RandomizedDelaySec=5min
Persistent=true
[Install]
WantedBy=timers.target
-6
View File
@@ -276,12 +276,6 @@ automatic** (D-021).
`grounded`, `failed` (with its reason) or `skipped`. A dinner whose write
fails is `failed` with `grounding_error` and left as it was; the others
still run, then the job fails with `internal_error`, keeping that result.
The responses returning a `dinners.Dinner` representation (the list; the
create, update, archive, favorite and grounding responses; the idea
responses) carry `photo_backfill_eligible`, and
`GET /api/v1/dinners/{id}` does not; it says whether the backfill grounds
that dinner, computed by the same condition it lists dinners by, so a
client offers a search only where one runs.
`GET /api/v1/dinners/{id}/recipe-candidates`
searches for the dinner's name and returns `{candidates: [{candidate_id,
title, site_name, photo_url, minutes}]}`, at most 3, ranked as `Ground`
-11
View File
@@ -25,17 +25,6 @@ const (
BackfillSkipped = "skipped"
)
// PhotoBackfillEligible reports whether a photo backfill grounds r: the
// condition ListPhotoBackfillDinners lists dinners by, and GroundFailedDinner
// and RefailDinnerGrounding write under, for one dinner. The dinner is
// unarchived, failed, with no photo and no pasted recipe_url.
func PhotoBackfillEligible(r db.Dinner) bool {
return r.Status != StatusArchived &&
r.GroundingStatus == GroundingFailed &&
!r.PhotoID.Valid &&
!r.RecipeUrl.Valid
}
// BackfillJob is the KindBackfillPhotos payload. DinnerID narrows the
// backfill to that dinner; empty backfills every dinner without a photo.
type BackfillJob struct {
-47
View File
@@ -192,53 +192,6 @@ func TestBackfillSkipsDinnersNotMissingAPhoto(t *testing.T) {
}
}
// TestPhotoBackfillEligibleIsWhatTheBackfillLists holds the API's
// photo_backfill_eligible to ListPhotoBackfillDinners for a dinner the
// backfill grounds and for each kind it never does.
func TestPhotoBackfillEligibleIsWhatTheBackfillLists(t *testing.T) {
t.Parallel()
e := newEnv(t)
e.model.QueueGenerate(recipestest.NormalizedReply("stovetop", 40))
grounded, err := e.svc.CreateRegular(t.Context(), dinners.NewRegular{Name: ptr("Chicken curry")}, member)
if err != nil {
t.Fatal(err)
}
e.waitJob(t, *grounded.GroundingJobID)
linked, err := e.svc.CreateRegular(t.Context(), dinners.NewRegular{Name: ptr("Mystery pie"), RecipeURL: ptr("https://d.example/pie")}, member)
if err != nil {
t.Fatal(err)
}
e.waitJob(t, *linked.GroundingJobID)
failed := e.failedDinner(t, "Abalone surprise")
archived, err := e.svc.Archive(t.Context(), e.failedDinner(t, "Old stew"), member)
if err != nil {
t.Fatal(err)
}
shown := map[string]dinners.Dinner{archived.ID: archived}
regulars, err := e.svc.List(t.Context(), dinners.StatusRegular)
if err != nil {
t.Fatal(err)
}
for _, d := range regulars {
shown[d.ID] = d
}
q := db.New(e.st.Reader)
for id, want := range map[string]bool{grounded.ID: false, linked.ID: false, failed: true, archived.ID: false} {
listed, err := q.ListPhotoBackfillDinners(t.Context(), sql.NullString{String: id, Valid: true})
if err != nil {
t.Fatal(err)
}
d, ok := shown[id]
if !ok {
t.Fatalf("dinner %s not shown", id)
}
if got := len(listed) == 1; got != want || d.PhotoBackfillEligible != want {
t.Errorf("%s: listed %t, photo_backfill_eligible %t, want both %t", d.Name, got, d.PhotoBackfillEligible, want)
}
}
}
func TestBackfillOfOneDinnerGroundsOnlyIt(t *testing.T) {
t.Parallel()
e := newUnstartedEnv(t)
-7
View File
@@ -79,11 +79,6 @@ type Dinner struct {
UpdatedBy *string `json:"updated_by"`
CreatedAt string `json:"created_at"`
UpdatedAt string `json:"updated_at"`
// PhotoBackfillEligible is whether a photo backfill grounds the dinner
// (PhotoBackfillEligible): false for one with a pasted recipe link,
// which a search never replaces.
PhotoBackfillEligible bool `json:"photo_backfill_eligible"`
}
// NewRegular is a Regular to add: its Name, which the household always
@@ -717,8 +712,6 @@ func fromRow(r db.Dinner) Dinner {
UpdatedBy: nullString(r.UpdatedBy),
CreatedAt: r.CreatedAt,
UpdatedAt: r.UpdatedAt,
PhotoBackfillEligible: PhotoBackfillEligible(r),
}
if r.MinutesTotal.Valid {
d.MinutesTotal = &r.MinutesTotal.Int64
@@ -52,8 +52,6 @@ function listed(is_favorite: boolean) {
is_favorite,
photo_url: null,
grounding_status: 'grounded',
grounding_error: null,
photo_backfill_eligible: false,
}
}
+2 -18
View File
@@ -20,12 +20,8 @@ export type Difficulty = (typeof DIFFICULTIES)[number]
export type GroundingStatus = (typeof GROUNDING_STATUSES)[number]
// Regular is the part of a dinner the Regulars screens read. photo_id is
// the stored photo's id, from photo_url. grounding_error is why its
// grounding failed, a reason code, set only while grounding_status is
// failed. photo_backfill_eligible is whether a photo backfill (Search
// again) searches for it: never for a dinner with a pasted recipe link.
// pending is true while an optimistic add waits for the server;
// its id is not the server's.
// the stored photo's id, from photo_url. pending is true while an
// optimistic add waits for the server; its id is not the server's.
export type Regular = {
id: string
name: string
@@ -35,8 +31,6 @@ export type Regular = {
is_favorite: boolean
photo_id: string | undefined
grounding_status: GroundingStatus
grounding_error: string | null
photo_backfill_eligible: boolean
pending?: true
}
@@ -119,9 +113,6 @@ export function parseDinner(where: string, body: unknown): Regular {
throw new Error(`${where}: response is not a dinner`)
}
if (typeof body.is_favorite !== 'boolean') throw new Error(`${where}: is_favorite is not a boolean`)
if (typeof body.photo_backfill_eligible !== 'boolean') {
throw new Error(`${where}: photo_backfill_eligible is not a boolean`)
}
return {
id: body.id,
name: body.name,
@@ -131,11 +122,6 @@ export function parseDinner(where: string, body: unknown): Regular {
is_favorite: body.is_favorite,
photo_id: photoId(where, body.photo_url),
grounding_status: oneOf(where, 'grounding_status', GROUNDING_STATUSES, body.grounding_status),
grounding_error: nullable(body.grounding_error, (v) => {
if (typeof v !== 'string') throw new Error(`${where}: grounding_error is not a string`)
return v
}),
photo_backfill_eligible: body.photo_backfill_eligible,
}
}
@@ -239,8 +225,6 @@ export function useAddRegular() {
is_favorite: false,
photo_id: undefined,
grounding_status: 'pending',
grounding_error: null,
photo_backfill_eligible: false,
pending: true,
}
await changeOptimistically(queryClient, (list) => [...list, optimistic])
-2
View File
@@ -114,8 +114,6 @@ function dinner(id: string, name: string, difficulty: string | null, minutes: nu
is_favorite: false,
photo_url: null,
grounding_status: 'grounded',
grounding_error: null,
photo_backfill_eligible: false,
}
}
@@ -300,27 +300,3 @@
.lock {
display: contents;
}
/* Find missing photos: its hint, button and progress, padded like a sheet. */
.card.backfill {
display: flex;
flex-direction: column;
gap: 12px;
padding: 16px;
}
/* Why a Regular's recipe was not found. */
.reason {
margin: 0;
padding: 12px 14px;
border-radius: 18px;
background: var(--color-neutral-100);
color: var(--color-neutral-900);
font-family: var(--font-body);
font-size: 14px;
line-height: 1.4;
}
.reasonLabel {
font-weight: 700;
}
+1 -203
View File
@@ -85,11 +85,7 @@ const TACOS = dinnerBody('d1', 'Beef tacos', {
photo_url: '/media/photos/p1-400.jpg',
})
const CHILI = dinnerBody('d2', 'Chili', { method: 'crockpot', minutes_total: 480, grounding_status: 'pending' })
const PIZZA = dinnerBody('d3', 'Pizza night', {
grounding_status: 'failed',
grounding_error: 'no_recipe_found',
photo_backfill_eligible: true,
})
const PIZZA = dinnerBody('d3', 'Pizza night', { grounding_status: 'failed', grounding_error: 'no_recipe_found' })
const ROAST = dinnerBody('d4', 'Pot roast', { method: 'oven', grounding_status: 'manual' })
function rows() {
@@ -500,204 +496,6 @@ describe('RegularsSettingsPage', () => {
})
})
describe('Photo backfill', () => {
const BACKFILL = 'POST /api/v1/settings/photo-backfill'
const JOB = 'GET /api/v1/jobs/j9'
// backfillJob is a GET /api/v1/jobs/j9 answer for the photo backfill.
function backfillJob(status: string, result: unknown = null, error: string | null = null) {
return jsonResponse(200, { id: 'j9', kind: 'backfill_dinner_photos', status, result, error })
}
// polls answers each poll of the job with the next answer, then the last
// one again; an answer may change the server first.
function polls(...answers: (() => Response)[]) {
let next = 0
return () => answers[Math.min(next++, answers.length - 1)]!()
}
function fixSheet() {
return within(screen.getByRole('dialog', { name: 'Fix recipe' }))
}
const NO_CANDIDATES = { 'GET /api/v1/dinners/d3/recipe-candidates': () => jsonResponse(200, { candidates: [] }) }
it('shows why the recipe was not found in the fix-recipe sheet', async () => {
stubServer([PIZZA], NO_CANDIDATES)
await renderPage()
openRow('Pizza night')
expect(fixSheet().getByText('No recipe page the search found had a recipe we could use.')).toBeInTheDocument()
expect(fixSheet().getByRole('button', { name: 'Search again' })).toBeEnabled()
})
it('offers no search again for a pasted link that had no usable recipe', async () => {
const linked = { ...PIZZA, grounding_error: 'recipe_unusable', photo_backfill_eligible: false }
stubServer([linked], NO_CANDIDATES)
await renderPage()
openRow('Pizza night')
expect(fixSheet().getByText('The pasted recipe link has no recipe we could use.')).toBeInTheDocument()
expect(fixSheet().queryByRole('button', { name: 'Search again' })).not.toBeInTheDocument()
})
it('offers no search again for a dinner with a pasted link whatever its reason', async () => {
const linked = { ...PIZZA, grounding_error: 'ai_failed', photo_backfill_eligible: false }
stubServer([linked], NO_CANDIDATES)
await renderPage()
openRow('Pizza night')
expect(fixSheet().getByText("The AI assistant couldn't read the recipe.")).toBeInTheDocument()
expect(fixSheet().queryByRole('button', { name: 'Search again' })).not.toBeInTheDocument()
})
it('searches again from the fix-recipe sheet and shows the new reason when it still fails', async () => {
const server = stubServer([PIZZA], {
...NO_CANDIDATES,
[BACKFILL]: () => jsonResponse(202, { id: 'j9' }),
[JOB]: polls(
() => backfillJob('running'),
() => {
server.regulars = [{ ...PIZZA, grounding_error: 'search_failed' }]
return backfillJob('succeeded', { dinners: [{ dinner_id: 'd3', outcome: 'failed', reason: 'search_failed' }] })
},
),
})
await renderPage()
openRow('Pizza night')
fireEvent.click(fixSheet().getByRole('button', { name: 'Search again' }))
expect(await fixSheet().findByRole('button', { name: 'Searching again…' })).toBeDisabled()
expect(fixSheet().getByText('Searching again… this can take a few minutes.')).toBeInTheDocument()
expect(sent(server.fetchMock)).toEqual([{ request: BACKFILL, body: undefined }])
expect(await fixSheet().findByText('Searched again: still no recipe found.', {}, { timeout: 3000 })).toBeInTheDocument()
expect(await fixSheet().findByText("The recipe search couldn't be reached.")).toBeInTheDocument()
expect(fixSheet().getByRole('button', { name: 'Search again' })).toBeEnabled()
})
it('closes the fix-recipe sheet once searching again finds the recipe and its photo', async () => {
const server = stubServer([PIZZA], {
...NO_CANDIDATES,
[BACKFILL]: () => jsonResponse(202, { id: 'j9' }),
[JOB]: () => {
server.regulars = [{ ...PIZZA, grounding_status: 'grounded', grounding_error: null, photo_url: '/media/photos/p8-400.jpg' }]
return backfillJob('succeeded', { dinners: [{ dinner_id: 'd3', outcome: 'grounded', reason: null }] })
},
})
await renderPage()
openRow('Pizza night')
fireEvent.click(fixSheet().getByRole('button', { name: 'Search again' }))
await waitFor(() => expect(screen.queryByRole('dialog')).not.toBeInTheDocument())
expect(within(row('Pizza night')).getByText('Recipe found')).toBeInTheDocument()
expect(within(row('Pizza night')).getByRole('img', { name: 'Pizza night' })).toHaveAttribute('src', '/media/photos/p8-400.jpg')
})
it('says when a dinner was not included in the search', async () => {
stubServer([PIZZA], {
...NO_CANDIDATES,
[BACKFILL]: () => jsonResponse(202, { id: 'j9' }),
[JOB]: () => backfillJob('succeeded', { dinners: [] }),
})
await renderPage()
openRow('Pizza night')
fireEvent.click(fixSheet().getByRole('button', { name: 'Search again' }))
expect(await fixSheet().findByText('This dinner was not included in that search.')).toBeInTheDocument()
})
it('finds missing photos: the placeholder shows until the backfill finds a photo', async () => {
const found = { ...PIZZA, grounding_status: 'grounded', grounding_error: null, photo_url: '/media/photos/p8-400.jpg' }
const server = stubServer([TACOS, PIZZA], {
[BACKFILL]: () => jsonResponse(202, { id: 'j9' }),
[JOB]: polls(
() => backfillJob('queued'),
() => {
server.regulars = [TACOS, found]
return backfillJob('succeeded', { dinners: [{ dinner_id: 'd3', outcome: 'grounded', reason: null }] })
},
),
})
await renderPage()
// The photoless dinner shows the striped placeholder, not an image.
const placeholder = within(row('Pizza night')).getByRole('img', { name: 'Pizza night' })
expect(placeholder).not.toHaveAttribute('src')
expect(placeholder).toHaveTextContent('dish photo')
const trigger = screen.getByRole('button', { name: 'Find missing photos' })
expectMinHitArea(trigger)
fireEvent.click(trigger)
expect(await screen.findByRole('button', { name: 'Finding photos…' })).toBeDisabled()
expect(await screen.findByText('Finding missing photos… this can take a few minutes.')).toBeInTheDocument()
expect(within(row('Pizza night')).getByRole('img', { name: 'Pizza night' })).not.toHaveAttribute('src')
expect(sent(server.fetchMock)).toEqual([{ request: BACKFILL, body: undefined }])
expect(await screen.findByText('Photo search done: 1 photo found.', {}, { timeout: 3000 })).toBeInTheDocument()
await waitFor(() =>
expect(within(row('Pizza night')).getByRole('img', { name: 'Pizza night' })).toHaveAttribute('src', '/media/photos/p8-400.jpg'),
)
expect(screen.getByRole('button', { name: 'Find missing photos' })).toBeEnabled()
expect(reads(server.fetchMock)).toBe(2)
})
it('sums up what the backfill did', async () => {
stubServer([PIZZA], {
[BACKFILL]: () => jsonResponse(202, { id: 'j9' }),
[JOB]: () =>
backfillJob('succeeded', {
dinners: [
{ dinner_id: 'd3', outcome: 'grounded', reason: null },
{ dinner_id: 'd5', outcome: 'grounded', reason: null },
{ dinner_id: 'd6', outcome: 'failed', reason: 'no_recipe_found' },
{ dinner_id: 'd7', outcome: 'skipped', reason: null },
],
}),
})
await renderPage()
fireEvent.click(screen.getByRole('button', { name: 'Find missing photos' }))
expect(
await screen.findByText(
'Photo search done: 2 photos found · 1 dinner still needs a recipe · 1 dinner changed meanwhile, left as it was.',
),
).toBeInTheDocument()
})
it('says when no dinner was missing a photo', async () => {
stubServer([TACOS], {
[BACKFILL]: () => jsonResponse(202, { id: 'j9' }),
[JOB]: () => backfillJob('succeeded', { dinners: [] }),
})
await renderPage()
fireEvent.click(screen.getByRole('button', { name: 'Find missing photos' }))
expect(await screen.findByText('No dinners were missing a photo.')).toBeInTheDocument()
})
it('says why the backfill stopped', async () => {
stubServer([PIZZA], {
[BACKFILL]: () => jsonResponse(202, { id: 'j9' }),
[JOB]: () => backfillJob('failed', { dinners: [] }, 'internal_error'),
})
await renderPage()
fireEvent.click(screen.getByRole('button', { name: 'Find missing photos' }))
expect(await screen.findByRole('alert')).toHaveTextContent('Photo search stopped: Something went wrong on the server.')
expect(screen.getByRole('button', { name: 'Find missing photos' })).toBeEnabled()
})
it('says why the backfill could not start', async () => {
stubServer([PIZZA], { [BACKFILL]: () => errorResponse(403, 'csrf_failed') })
await renderPage()
fireEvent.click(screen.getByRole('button', { name: 'Find missing photos' }))
expect(await screen.findByRole('alert')).toHaveTextContent('Photo search not started: csrf_failed message')
expect(screen.getByRole('button', { name: 'Find missing photos' })).toBeEnabled()
})
})
it('refreshes grounding status while any Regular is finding its recipe, then stops', async () => {
// Only intervals are faked: TanStack Query polls with setInterval, and
// Testing Library's waits keep their real timers.
+10 -147
View File
@@ -34,7 +34,6 @@ import { Sheet } from '../ui/Sheet.tsx'
import { Skeleton } from '../ui/Skeleton.tsx'
import { Tag } from '../ui/Tag.tsx'
import { TextInput } from '../ui/TextInput.tsx'
import { usePhotoBackfill, type BackfillResult, type PhotoBackfill } from './photoBackfill.ts'
import styles from './RegularsSettingsPage.module.css'
// The most minutes a dinner may take, as the dinners API allows: a day.
@@ -58,8 +57,7 @@ const LINK = /^https?:\/\//i
// method and minutes, favorite heart and whether its recipe was found. The
// field at the top adds one by name, or by pasted recipe link with a name
// the household gives it; tapping one
// edits or removes it, or fixes its recipe when none was found. Find
// missing photos searches again for every dinner left without one.
// edits or removes it, or fixes its recipe when none was found.
export function RegularsSettingsPage() {
const navigate = useNavigate()
return (
@@ -88,7 +86,6 @@ function RegularsManager() {
const remove = useRemoveRegular()
const favorite = useToggleFavorite()
const { refusals, track } = useRefusals()
const backfill = usePhotoBackfill()
const [draft, setDraft] = useState('')
const [linkName, setLinkName] = useState('')
const [open, setOpen] = useState<Open>(null)
@@ -97,10 +94,8 @@ function RegularsManager() {
const isLink = LINK.test(draft.trim())
const ready = draft.trim() !== '' && (!isLink || linkName.trim() !== '')
const list = regulars.data ?? []
// A Regular removed meanwhile closes its sheet, and one whose recipe was
// found meanwhile closes its fix-recipe sheet.
// A Regular removed meanwhile closes its sheet.
const opened = open && list.find((regular) => regular.id === open.id && !regular.pending)
const fixing = opened && open.sheet === 'fix' && opened.grounding_status === 'failed' ? opened : undefined
function onSubmit(event: FormEvent<HTMLFormElement>) {
event.preventDefault()
@@ -193,7 +188,6 @@ function RegularsManager() {
)}
</section>
)}
{regulars.isSuccess && <FindMissingPhotos backfill={backfill} />}
{opened && open.sheet === 'edit' && (
<EditRegularSheet
@@ -203,13 +197,12 @@ function RegularsManager() {
onRemove={() => onRemove(opened)}
/>
)}
{fixing && (
{opened && open.sheet === 'fix' && (
<FixRecipeSheet
key={fixing.id}
regular={fixing}
backfill={backfill}
key={opened.id}
regular={opened}
onClose={() => setOpen(null)}
onEdit={() => setOpen({ sheet: 'edit', id: fixing.id })}
onEdit={() => setOpen({ sheet: 'edit', id: opened.id })}
/>
)}
</>
@@ -238,88 +231,6 @@ function GroundingBadge({ status }: { status: GroundingStatus }) {
return <Tag className={`${styles.badge} ${badge.className}`}>{badge.text}</Tag>
}
// groundingReasons says why a dinner's recipe was not found, by its
// grounding_error reason code.
const groundingReasons: Record<string, string> = {
no_recipe_found: 'No recipe page the search found had a recipe we could use.',
recipe_unusable: 'The pasted recipe link has no recipe we could use.',
search_not_configured: 'Recipe search is not set up. An admin can set it up in Settings.',
search_failed: "The recipe search couldn't be reached.",
ai_failed: "The AI assistant couldn't read the recipe.",
interrupted: 'The server stopped while finding the recipe.',
grounding_error: 'Something went wrong on the server.',
}
// groundingReason says why a dinner's recipe was not found; an unknown
// reason code is shown as it is.
export function groundingReason(code: string): string {
return groundingReasons[code] ?? code
}
// backfillJobFailures says why a photo backfill job failed, by its failure
// code; an unknown code is shown as it is.
const backfillJobFailures: Record<string, string> = {
internal_error: 'Something went wrong on the server.',
}
// backfillProblem says what went wrong with a photo backfill, or null.
function backfillProblem(backfill: PhotoBackfill): string | null {
if (backfill.startError) return `Photo search not started: ${backfill.startError.message}`
if (backfill.status === 'failed' && backfill.error) {
return `Photo search stopped: ${backfillJobFailures[backfill.error.message] ?? backfill.error.message}`
}
if (backfill.error) return `Couldn't check on the photo search: ${backfill.error.message}`
return null
}
function count(n: number, one: string, many: string): string {
return `${n} ${n === 1 ? one : many}`
}
// backfillSummary says what a finished photo backfill did.
function backfillSummary(result: BackfillResult): string {
const tally = (outcome: string) => result.dinners.filter((dinner) => dinner.outcome === outcome).length
if (result.dinners.length === 0) return 'No dinners were missing a photo.'
const parts = [count(tally('grounded'), 'photo found', 'photos found')]
const failed = tally('failed')
if (failed > 0) parts.push(`${count(failed, 'dinner still needs', 'dinners still need')} a recipe`)
const skipped = tally('skipped')
if (skipped > 0) parts.push(`${count(skipped, 'dinner', 'dinners')} changed meanwhile, left as it was`)
return `Photo search done: ${parts.join(' · ')}.`
}
// FindMissingPhotos searches again, by name, for a recipe and photo for
// every dinner left without a photo because its recipe was not found.
function FindMissingPhotos({ backfill }: { backfill: PhotoBackfill }) {
const headingId = useId()
const problem = backfillProblem(backfill)
return (
<section aria-labelledby={headingId} className={styles.section}>
<h2 id={headingId} className={styles.groupLabel}>
Missing photos
</h2>
<Card className={`${styles.card} ${styles.backfill}`}>
<p className={styles.hint}>
Search again for a recipe and photo for every dinner whose recipe wasn&apos;t found. Dinners with a pasted
recipe link keep it.
</p>
<Button disabled={backfill.active} onClick={backfill.start}>
{backfill.active ? 'Finding photos…' : 'Find missing photos'}
</Button>
<p role="status" className={styles.status}>
{backfill.active && 'Finding missing photos… this can take a few minutes.'}
{backfill.status === 'succeeded' && backfill.result && backfillSummary(backfill.result)}
</p>
{problem && (
<p role="alert" className={styles.problem}>
{problem}
</p>
)}
</Card>
</section>
)
}
type RegularRowProps = {
regular: Regular
onOpen: () => void
@@ -536,7 +447,6 @@ const fixPaths = [
type FixRecipeSheetProps = {
regular: Regular
backfill: PhotoBackfill
onClose: () => void
onEdit: () => void
}
@@ -549,20 +459,10 @@ function lines(text: string): string[] {
.filter((line) => line !== '')
}
// retryOutcomes says what searching again did for this dinner.
const retryOutcomes = {
grounded: 'Searched again: a recipe and photo were found.',
failed: 'Searched again: still no recipe found.',
skipped: 'It changed while it was searched again, so it was left as it is.',
} as const
// FixRecipeSheet shows why a Regular's recipe was not found and grounds it:
// by searching its name again (a photo backfill), in one of up to three
// recipe pages found for its name, in a pasted recipe link, or in
// ingredients and steps typed in. Searching again is offered only for a
// dinner the backfill searches (photo_backfill_eligible): it never replaces
// a pasted link.
function FixRecipeSheet({ regular, backfill, onClose, onEdit }: FixRecipeSheetProps) {
// FixRecipeSheet grounds a Regular whose recipe was not found: in one of
// up to three recipe pages found for its name, in a pasted recipe link, or
// in ingredients and steps typed in.
function FixRecipeSheet({ regular, onClose, onEdit }: FixRecipeSheetProps) {
const [path, setPath] = useState<FixPath>('pick')
const ground = useGroundRegular()
@@ -575,15 +475,6 @@ function FixRecipeSheet({ regular, backfill, onClose, onEdit }: FixRecipeSheetPr
<p className={styles.hint}>
We couldn&apos;t find a recipe for {regular.name}. Pick a match below, paste a recipe link or type it in.
</p>
{regular.grounding_error !== null && (
<p className={styles.reason}>
<span className={styles.reasonLabel}>Why: </span>
{groundingReason(regular.grounding_error)}
</p>
)}
{regular.photo_backfill_eligible && (
<RetrySearch id={regular.id} backfill={backfill} disabled={ground.isPending} />
)}
<Segmented
label="How to fix the recipe"
options={fixPaths}
@@ -613,34 +504,6 @@ function FixRecipeSheet({ regular, backfill, onClose, onEdit }: FixRecipeSheetPr
)
}
// RetrySearch searches the dinner's name again with a photo backfill, and
// says what it did for the dinner once it ends. The backfill searches
// every dinner without a photo, this one with them.
function RetrySearch({ id, backfill, disabled }: { id: string; backfill: PhotoBackfill; disabled: boolean }) {
const problem = backfillProblem(backfill)
const tried = backfill.result?.dinners.find((dinner) => dinner.dinner_id === id)
let status: string | null = null
if (backfill.active) status = 'Searching again… this can take a few minutes.'
else if (backfill.status === 'succeeded') {
status = tried ? retryOutcomes[tried.outcome] : 'This dinner was not included in that search.'
}
return (
<>
<Button variant="outline" disabled={disabled || backfill.active} onClick={backfill.start}>
{backfill.active ? 'Searching again…' : 'Search again'}
</Button>
<p role="status" className={styles.status}>
{status}
</p>
{problem && (
<p role="alert" className={styles.problem}>
{problem}
</p>
)}
</>
)
}
function PickCandidate({ id, onPick }: { id: string; onPick: (candidateId: string) => void }) {
const candidates = useRecipeCandidates(id)
if (candidates.isPending) return <Skeleton className={styles.candidatesLoading} />
-63
View File
@@ -1,63 +0,0 @@
import { QueryClientProvider } from '@tanstack/react-query'
import { act, renderHook, waitFor } from '@testing-library/react'
import type { ReactNode } from 'react'
import { afterEach, describe, expect, it, vi } from 'vitest'
import { ApiProvider, createApiClient, createQueryClient, createVerifiedSession } from '../api/index.ts'
import { HOME_KEY } from '../home/api.ts'
import { planKey } from '../plan/api.ts'
import { createNetworkStatus } from '../pwa/networkStatus.ts'
import { fakeLocation, jsonResponse, ME, stubFetch } from '../test/fetchStub.ts'
import { parseBackfillResult, usePhotoBackfill } from './photoBackfill.ts'
afterEach(() => {
vi.unstubAllGlobals()
})
const RESULT = { dinners: [{ dinner_id: 'd3', outcome: 'grounded', reason: null }] }
function backfillJob(status: string, result: unknown) {
return jsonResponse(200, { id: 'j9', kind: 'backfill_dinner_photos', status, result, error: null })
}
describe('usePhotoBackfill', () => {
it('reads the plans, home and dinners again once the backfill ends, so a found photo shows everywhere', async () => {
const fetchMock = stubFetch(
jsonResponse(200, ME),
jsonResponse(202, { id: 'j9' }),
backfillJob('running', null),
backfillJob('succeeded', RESULT),
)
const api = createApiClient(fakeLocation(), createNetworkStatus(window), createVerifiedSession(localStorage))
await api.loadMe()
const queryClient = createQueryClient()
const plan = planKey('2026-10-05')
for (const key of [plan, HOME_KEY, ['dinners', 'quest_log']]) queryClient.setQueryData(key, {})
const wrapper = ({ children }: { children: ReactNode }) => (
<ApiProvider client={api}>
<QueryClientProvider client={queryClient}>{children}</QueryClientProvider>
</ApiProvider>
)
const { result } = renderHook(() => usePhotoBackfill(), { wrapper })
act(() => result.current.start())
await waitFor(() => expect(result.current.status).toBe('running'))
expect(result.current.active).toBe(true)
expect(queryClient.getQueryState(plan)?.isInvalidated).toBe(false)
await waitFor(() => expect(result.current.status).toBe('succeeded'), { timeout: 3000 })
expect(result.current).toMatchObject({ active: false, error: null, result: RESULT })
expect(fetchMock).toHaveBeenCalledTimes(4)
for (const key of [plan, HOME_KEY, ['dinners', 'quest_log']]) {
expect(queryClient.getQueryState(key)?.isInvalidated).toBe(true)
}
})
})
describe('parseBackfillResult', () => {
it('refuses a result that is not a photo backfill', () => {
expect(() => parseBackfillResult('job', { dinners: [{ dinner_id: 'd3', outcome: 'lost', reason: null }] })).toThrow(
'job: outcome "lost" is not one of grounded, failed, skipped',
)
expect(() => parseBackfillResult('job', { stage: 'x' })).toThrow('job: result is not a photo backfill')
})
})
-110
View File
@@ -1,110 +0,0 @@
import { useMutation, useQueryClient } from '@tanstack/react-query'
import { useEffect, useMemo, useState } from 'react'
import { useApi, useJob, type JobStatus } from '../api/index.ts'
import { isRecord } from '../api/errors.ts'
import { oneOf } from '../dinners/api.ts'
import { HOME_KEY } from '../home/api.ts'
export const PHOTO_BACKFILL_PATH = '/api/v1/settings/photo-backfill'
// BACKFILL_JOB_KIND is the kind of the job a photo backfill runs as.
export const BACKFILL_JOB_KIND = 'backfill_dinner_photos'
export const BACKFILL_OUTCOMES = ['grounded', 'failed', 'skipped'] as const
export type BackfillOutcome = (typeof BACKFILL_OUTCOMES)[number]
// BackfillDinner is what a photo backfill did for one dinner. reason is
// set only when it failed: the reason code now its grounding_error.
export type BackfillDinner = { dinner_id: string; outcome: BackfillOutcome; reason: string | null }
// BackfillResult is the photo backfill job's result: each dinner it tried,
// in the order it tried them.
export type BackfillResult = { dinners: BackfillDinner[] }
// Every query showing a dinner's photo, read again once a backfill ends:
// the dinners lists and details, the plans and the home dashboard.
const PHOTO_KEYS = [['dinners'], ['plans'], HOME_KEY] as const
// parseBackfillResult checks a photo backfill job's result, which useJob
// passes on unread.
export function parseBackfillResult(where: string, body: unknown): BackfillResult {
if (!isRecord(body) || !Array.isArray(body.dinners)) throw new Error(`${where}: result is not a photo backfill`)
return {
dinners: body.dinners.map((dinner): BackfillDinner => {
if (!isRecord(dinner) || typeof dinner.dinner_id !== 'string' || (dinner.reason !== null && typeof dinner.reason !== 'string')) {
throw new Error(`${where}: result has a dinner that is not a backfill outcome`)
}
return {
dinner_id: dinner.dinner_id,
outcome: oneOf(where, 'outcome', BACKFILL_OUTCOMES, dinner.outcome),
reason: dinner.reason,
}
}),
}
}
// PhotoBackfill is a photo backfill started from this screen, as it goes.
export type PhotoBackfill = {
// start asks the server for a backfill over every dinner without a
// photo. While one is already queued or running the server answers with
// that job, so starting never runs two at once.
start: () => void
// active is true from start until the job ends, except while a poll's
// failure leaves its progress unknown: starting again then asks the
// server for the job it is running and polls it anew.
active: boolean
// Why the backfill could not be started.
startError: Error | null
// The job's status, undefined until it is started and first polled.
status: JobStatus | undefined
// The job's result, once it has one.
result: BackfillResult | null
// The job's failure, a poll's failure, or a result that is not a backfill.
error: Error | null
}
// usePhotoBackfill starts a photo backfill (POST
// /api/v1/settings/photo-backfill, 202 {id}) and follows its job with
// useJob. When the job ends, every query showing a dinner's photo is read
// again, so dinners it grounded show their photo everywhere.
export function usePhotoBackfill(): PhotoBackfill {
const { apiFetch } = useApi()
const queryClient = useQueryClient()
const [jobId, setJobId] = useState<string | null>(null)
const job = useJob<unknown>(BACKFILL_JOB_KIND, jobId)
const begin = useMutation({
mutationFn: async (): Promise<string> => {
const body = await apiFetch<unknown>(PHOTO_BACKFILL_PATH, { method: 'POST' })
if (!isRecord(body) || typeof body.id !== 'string') throw new Error(`POST ${PHOTO_BACKFILL_PATH}: response has no job id`)
return body.id
},
onSuccess: (id) => {
setJobId(id)
job.poll()
},
})
const ended = job.status === 'succeeded' || job.status === 'failed'
useEffect(() => {
if (!ended) return
for (const queryKey of PHOTO_KEYS) void queryClient.invalidateQueries({ queryKey })
}, [ended, jobId, queryClient])
const parsed = useMemo(() => {
if (job.result === null) return { result: null, error: null }
try {
return { result: parseBackfillResult(`GET /api/v1/jobs/${jobId}`, job.result), error: null }
} catch (error) {
return { result: null, error: error as Error }
}
}, [job.result, jobId])
return {
start: () => begin.mutate(),
active: begin.isPending || (jobId !== null && !ended && job.error === null),
startError: begin.error,
status: job.status,
result: parsed.result,
error: job.error ?? parsed.error,
}
}
-2
View File
@@ -13,7 +13,6 @@ export type DinnerBody = {
photo_url: string | null
grounding_status: string
grounding_error: string | null
photo_backfill_eligible: boolean
updated_by: string | null
created_at: string
updated_at: string
@@ -36,7 +35,6 @@ export function dinnerBody(id: string, name: string, overrides: Partial<DinnerBo
photo_url: null,
grounding_status: 'grounded',
grounding_error: null,
photo_backfill_eligible: false,
updated_by: null,
created_at: '2026-10-01T15:00:00.000Z',
updated_at: '2026-10-01T15:00:00.000Z',