Make the pre-deploy count check runnable and split rollback by migration
CI / ci (push) Successful in 3m13s
CI / ci (pull_request) Successful in 3m14s

Write /tmp/mqcount.py from the Count check section with one heredoc
command, and give rollbacks that cross a migration an ordered
restore-then-start procedure separate from the plain restart.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-07 17:35:44 -04:00
co-authored by Claude Opus 5.5
parent 614c4febfa
commit a7f115f2f0
+33 -8
View File
@@ -508,7 +508,8 @@ docker compose -f deploy/compose.yaml build
point before the restart (see "Rollback to a previous SHA"). Check that point before the restart (see "Rollback to a previous SHA"). Check that
Litestream is replicating and that its replica restores to the current Litestream is replicating and that its replica restores to the current
data, with the commands from "Database — Litestream", "Restore the data, with the commands from "Database — Litestream", "Restore the
database" and "Count check" (`/tmp/mqcount.py`): database" and "Count check". First write `/tmp/mqcount.py` with the
`cat > /tmp/mqcount.py` command in "Count check", then:
```sh ```sh
D=/var/lib/docker/volumes/mealquest-data/_data D=/var/lib/docker/volumes/mealquest-data/_data
@@ -567,8 +568,17 @@ the restart; the service fails fast at boot naming any missing required variable
### Rollback to a previous SHA ### Rollback to a previous SHA
First check whether the rollback crosses a migration: compare
`migrations/` between the two SHAs.
```sh ```sh
cd /home/overseer/mealquest cd /home/overseer/mealquest
git diff --stat <previous SHA> HEAD -- migrations/ # empty: no migration crossed
```
**No migration crossed** — rebuild and restart the older SHA:
```sh
git checkout --detach <previous SHA> git checkout --detach <previous SHA>
docker compose -f deploy/compose.yaml build docker compose -f deploy/compose.yaml build
sudo systemctl restart mealquest sudo systemctl restart mealquest
@@ -576,12 +586,21 @@ docker compose -f deploy/compose.yaml ps
curl -fsS http://10.60.1.14:8090/health curl -fsS http://10.60.1.14:8090/health
``` ```
A rollback across a release that added a migration fails at start-up **A migration crossed** — the older binary refuses the newer schema, so the
(the older binary refuses the newer schema). In that case restore the database is restored first and the older SHA started after. Changes made
database to the pre-deploy restore point recorded in step 2 of "Deploy" — since the pre-deploy restore point are lost; say so to the household.
"Restore the database" with `-timestamp <pre-deploy restore point>` — then
start the older SHA. Changes made since that point are lost; say so to the 1. Check out and build the older SHA. Nothing restarts yet.
household.
```sh
git checkout --detach <previous SHA>
docker compose -f deploy/compose.yaml build
```
2. Run "Restore the database" steps 1–4, adding
`-timestamp <pre-deploy restore point>` (recorded in step 2 of
"Deploy") to the `restore` command. Its step 5 then starts the older
SHA and checks health.
## Backups ## Backups
@@ -712,7 +731,10 @@ digits; Litestream's own `_litestream*` tables are excluded) and
`PRAGMA integrity_check`. Run it on a **copy** of the database, never on the `PRAGMA integrity_check`. Run it on a **copy** of the database, never on the
live file; copy `-wal`/`-shm` along with it when the app is running. live file; copy `-wal`/`-shm` along with it when the app is running.
```python `/tmp` is temporary, so write the script before every check:
```sh
cat > /tmp/mqcount.py <<'EOF'
import sqlite3, sys, hashlib import sqlite3, sys, hashlib
c = sqlite3.connect(sys.argv[1]) c = sqlite3.connect(sys.argv[1])
tabs = [r[0] for r in c.execute("select name from sqlite_master where type=\"table\" and name not like \"sqlite_%\" and name not like \"_litestream%\" order by name")] tabs = [r[0] for r in c.execute("select name from sqlite_master where type=\"table\" and name not like \"sqlite_%\" and name not like \"_litestream%\" order by name")]
@@ -724,8 +746,11 @@ for t in tabs:
h.update(repr(row).encode()) h.update(repr(row).encode())
print("tables", len(tabs), "content-sha256", h.hexdigest()[:16]) print("tables", len(tabs), "content-sha256", h.hexdigest()[:16])
print("integrity", c.execute("pragma integrity_check").fetchone()[0]) print("integrity", c.execute("pragma integrity_check").fetchone()[0])
EOF
``` ```
Then copy the database and run it:
```sh ```sh
rm -rf /tmp/mqcheck && mkdir /tmp/mqcheck rm -rf /tmp/mqcheck && mkdir /tmp/mqcheck
sudo sh -c "cp -a $D/mealquest.db* /tmp/mqcheck/" && sudo chown -R "$USER" /tmp/mqcheck sudo sh -c "cp -a $D/mealquest.db* /tmp/mqcheck/" && sudo chown -R "$USER" /tmp/mqcheck