Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
765740b921 | ||
|
|
9ac3342664 | ||
|
|
9d0e943fc3 | ||
|
|
3b524ec118 |
@@ -3,6 +3,8 @@ design/.DS_Store
|
|||||||
|
|
||||||
# Secrets and local environment
|
# Secrets and local environment
|
||||||
.env*
|
.env*
|
||||||
|
!deploy/.env.example
|
||||||
|
deploy/.env
|
||||||
deploy/secrets/
|
deploy/secrets/
|
||||||
|
|
||||||
# Build artifacts and dependencies
|
# Build artifacts and dependencies
|
||||||
|
|||||||
@@ -0,0 +1,80 @@
|
|||||||
|
# MealQuest production environment. Copy to deploy/.env (never committed)
|
||||||
|
# and replace the placeholders. The service fails fast: a missing or invalid
|
||||||
|
# required variable stops it at boot with the variable named.
|
||||||
|
#
|
||||||
|
# Secrets are never set here. They live in deploy/secrets/ (never committed),
|
||||||
|
# one value per file, owned by the container user and readable only by it:
|
||||||
|
#
|
||||||
|
# mkdir -p deploy/secrets && chmod 700 deploy/secrets
|
||||||
|
# head -c 32 /dev/urandom | base64 > deploy/secrets/secret_key
|
||||||
|
# printf '%s\n' '<Authentik client secret>' > deploy/secrets/oidc_client_secret
|
||||||
|
# sudo chown 10001:10001 deploy/secrets/* && sudo chmod 600 deploy/secrets/*
|
||||||
|
#
|
||||||
|
# deploy/compose.yaml mounts them at /run/secrets/<name> and points the
|
||||||
|
# *_FILE variables at them.
|
||||||
|
|
||||||
|
# --- Compose -----------------------------------------------------------------
|
||||||
|
|
||||||
|
# Host address the service is published on, as <address>:8090. Production
|
||||||
|
# binds the LAN address 10.60.1.14 only; 127.0.0.1 suits a dev machine.
|
||||||
|
MEALQUEST_BIND_ADDRESS=127.0.0.1
|
||||||
|
|
||||||
|
# --- Service -----------------------------------------------------------------
|
||||||
|
|
||||||
|
# Public origin the household opens. Fixes the OIDC redirect URI
|
||||||
|
# (<url>/auth/callback) and the Secure cookies. Must be https:// (plain
|
||||||
|
# http:// is accepted only for localhost, 127.0.0.1 and [::1]).
|
||||||
|
MEALQUEST_PUBLIC_URL=https://mealquest.cubecraftlabs.com
|
||||||
|
|
||||||
|
# How long a graceful shutdown may take before in-flight requests are cut.
|
||||||
|
MEALQUEST_SHUTDOWN_TIMEOUT=10s
|
||||||
|
|
||||||
|
# Optional. Comma-separated IPs/CIDRs of reverse proxies whose
|
||||||
|
# X-Forwarded-For/-Proto headers are honoured. Production lists Traefik;
|
||||||
|
# unset trusts no peer.
|
||||||
|
MEALQUEST_TRUSTED_PROXY=10.60.1.33
|
||||||
|
|
||||||
|
# IANA zone the household settings are created with on first boot. Required
|
||||||
|
# until those settings exist; ignored afterwards (change the zone in
|
||||||
|
# household settings instead). Confirm the household's zone before the first
|
||||||
|
# deploy.
|
||||||
|
MEALQUEST_DEFAULT_TZ=America/New_York
|
||||||
|
|
||||||
|
# --- Authentik OIDC ----------------------------------------------------------
|
||||||
|
|
||||||
|
# Issuer of the Authentik application "mealquest". Discovered at boot, so it
|
||||||
|
# must be reachable from the container. Must be https://.
|
||||||
|
MEALQUEST_OIDC_ISSUER=https://auth.cubecraftlabs.com/application/o/mealquest/
|
||||||
|
MEALQUEST_OIDC_CLIENT_ID=mealquest
|
||||||
|
# Comma-separated Authentik groups whose members may sign in.
|
||||||
|
MEALQUEST_OIDC_ALLOWED_GROUPS=mealquest-admin
|
||||||
|
|
||||||
|
# Optional session lifetimes. A session ends after IDLE unused and always MAX
|
||||||
|
# after sign-in; MAX must be at least IDLE. Defaults: 720h and 4320h.
|
||||||
|
MEALQUEST_SESSION_IDLE=720h
|
||||||
|
MEALQUEST_SESSION_MAX=4320h
|
||||||
|
|
||||||
|
# --- Set by deploy/compose.yaml; do not set here -----------------------------
|
||||||
|
#
|
||||||
|
# MEALQUEST_HTTP_ADDR=:8080
|
||||||
|
# MEALQUEST_DATA_DIR=/data
|
||||||
|
# MEALQUEST_SECRET_KEY_FILE=/run/secrets/secret_key
|
||||||
|
# Base64 of 32 random bytes; encrypts secrets at rest (AES-256-GCM).
|
||||||
|
# MEALQUEST_OIDC_CLIENT_SECRET_FILE=/run/secrets/oidc_client_secret
|
||||||
|
# The Authentik provider's client secret.
|
||||||
|
|
||||||
|
# --- Planned; not read by the service yet -------------------------------------
|
||||||
|
#
|
||||||
|
# Added to deploy/compose.yaml and this file by the features that use them.
|
||||||
|
#
|
||||||
|
# Web Push (Feature 19). Generate the key pair with cmd/vapidgen.
|
||||||
|
# MEALQUEST_VAPID_PUBLIC_KEY=<base64url VAPID public key>
|
||||||
|
# MEALQUEST_VAPID_PRIVATE_KEY_FILE=/run/secrets/vapid_private_key
|
||||||
|
# MEALQUEST_VAPID_SUBJECT=mailto:<owner address>
|
||||||
|
#
|
||||||
|
# Google Calendar OAuth web client (Feature 16).
|
||||||
|
# MEALQUEST_GOOGLE_CLIENT_ID=<client id>.apps.googleusercontent.com
|
||||||
|
# MEALQUEST_GOOGLE_CLIENT_SECRET_FILE=/run/secrets/google_client_secret
|
||||||
|
#
|
||||||
|
# SearXNG recipe search sidecar on the compose network (Feature 07).
|
||||||
|
# MEALQUEST_SEARXNG_URL=http://searxng:8080
|
||||||
@@ -0,0 +1,318 @@
|
|||||||
|
# MealQuest deployment runbook
|
||||||
|
|
||||||
|
How MealQuest is deployed on office-srvr (10.60.1.14) and how its data is
|
||||||
|
backed up and restored. Sections marked *task 876134bc* are completed by
|
||||||
|
that task (Feature 07 deployment runbook); this file currently records the
|
||||||
|
reverse-proxy route, the backups and the restore procedure.
|
||||||
|
|
||||||
|
Paths used below:
|
||||||
|
|
||||||
|
| Name | Path |
|
||||||
|
| --- | --- |
|
||||||
|
| Checkout on the host | `/home/overseer/mealquest` |
|
||||||
|
| Data volume (`mealquest-data`) on the host | `/var/lib/docker/volumes/mealquest-data/_data` |
|
||||||
|
| Same volume inside the containers | `/data` |
|
||||||
|
| Database | `mealquest.db` (+ `mealquest.db-wal`, `mealquest.db-shm` while open) |
|
||||||
|
| Media | `media/` in the data volume |
|
||||||
|
| Backup target (NAS, over SSH) | `joshua@10.60.1.50:/volume1/homes/Joshua/mealquest-backups` |
|
||||||
|
| Same target over SFTP (Synology chroot) | `/homes/Joshua/mealquest-backups` |
|
||||||
|
|
||||||
|
## Stack and first deploy
|
||||||
|
|
||||||
|
*Completed by task 876134bc.* The compose stack is `deploy/compose.yaml`;
|
||||||
|
start it with `docker compose -f deploy/compose.yaml up -d --build`.
|
||||||
|
|
||||||
|
## Reverse proxy (Traefik)
|
||||||
|
|
||||||
|
MealQuest is published at <https://mealquest.cubecraftlabs.com> by the
|
||||||
|
shared Traefik instance on 10.60.1.33 (Feature 07, task 0c1dc172). The
|
||||||
|
route is a dynamic-configuration file that lives on the Traefik host only,
|
||||||
|
at `/home/overseer/Code/traefik/config/dynamic/mealquest-ccl.yml`. Its exact
|
||||||
|
contents:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
# MealQuest - https://mealquest.cubecraftlabs.com
|
||||||
|
# Feature 07 task 0c1dc172. Single Go binary (API + embedded PWA) on
|
||||||
|
# office-srvr 10.60.1.14:8090. HTTP -> HTTPS is handled globally by
|
||||||
|
# http-conf.yml, so only a websecure router is defined here.
|
||||||
|
# This file lives on the Traefik host only; the repo copy is recorded in
|
||||||
|
# deploy/RUNBOOK.md by task 876134bc.
|
||||||
|
http:
|
||||||
|
routers:
|
||||||
|
mealquest:
|
||||||
|
entryPoints:
|
||||||
|
- websecure
|
||||||
|
rule: "Host(`mealquest.cubecraftlabs.com`)"
|
||||||
|
priority: 100
|
||||||
|
middlewares:
|
||||||
|
- mealquest-headers
|
||||||
|
service: mealquest
|
||||||
|
tls:
|
||||||
|
certResolver: myresolver
|
||||||
|
domains:
|
||||||
|
- main: mealquest.cubecraftlabs.com
|
||||||
|
|
||||||
|
services:
|
||||||
|
mealquest:
|
||||||
|
loadBalancer:
|
||||||
|
servers:
|
||||||
|
- url: "http://10.60.1.14:8090"
|
||||||
|
|
||||||
|
middlewares:
|
||||||
|
mealquest-headers:
|
||||||
|
headers:
|
||||||
|
stsSeconds: 31536000
|
||||||
|
stsIncludeSubdomains: true
|
||||||
|
contentTypeNosniff: true
|
||||||
|
frameDeny: true
|
||||||
|
```
|
||||||
|
|
||||||
|
## Backups
|
||||||
|
|
||||||
|
### Database — Litestream
|
||||||
|
|
||||||
|
The `litestream` compose service streams every committed change to
|
||||||
|
`mealquest.db` to the NAS over SFTP (`deploy/litestream.yml`). It takes a
|
||||||
|
full snapshot daily (compaction level 9) and keeps a week of snapshots
|
||||||
|
plus the incremental LTX files between them.
|
||||||
|
|
||||||
|
One-time setup (already done on 10.60.1.14):
|
||||||
|
|
||||||
|
1. Generate the dedicated keypair, git-ignored, owned by the container
|
||||||
|
user:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
cd /home/overseer/mealquest
|
||||||
|
ssh-keygen -t ed25519 -N '' -C mealquest-litestream -f deploy/secrets/litestream_key
|
||||||
|
# append deploy/secrets/litestream_key.pub to joshua@10.60.1.50:~/.ssh/authorized_keys
|
||||||
|
rm deploy/secrets/litestream_key.pub
|
||||||
|
sudo chown 10001:10001 deploy/secrets/litestream_key
|
||||||
|
sudo chmod 0600 deploy/secrets/litestream_key
|
||||||
|
```
|
||||||
|
|
||||||
|
2. The NAS host key is pinned in `deploy/litestream.yml` (`host-key`).
|
||||||
|
|
||||||
|
Check replication — list every LTX file at every level. Level 9 holds the
|
||||||
|
full snapshots; the default listing (`-level 0`) shows only L0 files:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
docker compose -f deploy/compose.yaml exec -T litestream \
|
||||||
|
litestream ltx -config /etc/litestream.yml -level all /data/mealquest.db
|
||||||
|
```
|
||||||
|
|
||||||
|
### Media — nightly rsync
|
||||||
|
|
||||||
|
`mealquest-media-backup.timer` runs `mealquest-media-backup.service` at
|
||||||
|
03:30 every night. The service runs `/usr/local/sbin/mealquest-media-backup`
|
||||||
|
(`deploy/systemd/mealquest-media-backup.sh`) as the unprivileged host user
|
||||||
|
`mealquest` (uid/gid 10001, the app's own user) with no capabilities.
|
||||||
|
systemd bind-mounts the data volume read-only at `/run/mealquest-data` and
|
||||||
|
passes the Litestream key in as the credential `nas_key`; uid 10001 cannot
|
||||||
|
otherwise reach `/var/lib/docker` or `deploy/secrets/`. The script refuses a
|
||||||
|
`media` path that is a symlink and fails (exit 1) if `media` is missing.
|
||||||
|
|
||||||
|
One-time setup (already done on 10.60.1.14):
|
||||||
|
|
||||||
|
```sh
|
||||||
|
cd /home/overseer/mealquest
|
||||||
|
# Host user for the backup; ssh needs a passwd entry for its uid.
|
||||||
|
sudo groupadd --system --gid 10001 mealquest
|
||||||
|
sudo useradd --system --uid 10001 --gid 10001 --no-create-home \
|
||||||
|
--home-dir /nonexistent --shell /usr/sbin/nologin mealquest
|
||||||
|
# Pinned NAS host keys, root-owned and world-readable.
|
||||||
|
sudo install -d -o root -g root -m 0755 /etc/mealquest
|
||||||
|
sudo install -o root -g root -m 0644 deploy/known_hosts /etc/mealquest/known_hosts
|
||||||
|
# The media directory; the backup fails if it is ever missing.
|
||||||
|
sudo install -d -o 10001 -g 10001 -m 0755 /var/lib/docker/volumes/mealquest-data/_data/media
|
||||||
|
# Script and units.
|
||||||
|
sudo install -o root -g root -m 0755 deploy/systemd/mealquest-media-backup.sh /usr/local/sbin/mealquest-media-backup
|
||||||
|
sudo install -o root -g root -m 0644 deploy/systemd/mealquest-media-backup.service \
|
||||||
|
deploy/systemd/mealquest-media-backup.timer /etc/systemd/system/
|
||||||
|
sudo systemctl daemon-reload
|
||||||
|
sudo systemctl enable --now mealquest-media-backup.timer
|
||||||
|
```
|
||||||
|
|
||||||
|
Run it by hand and read the result:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
sudo systemctl start mealquest-media-backup.service
|
||||||
|
journalctl -u mealquest-media-backup --no-pager -n 30
|
||||||
|
```
|
||||||
|
|
||||||
|
Restore media by reversing the copy into the (empty or stale) media
|
||||||
|
directory as uid 10001:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
sudo rsync -a --chown=10001:10001 \
|
||||||
|
-e "ssh -i deploy/secrets/litestream_key -o IdentitiesOnly=yes -o UserKnownHostsFile=/etc/mealquest/known_hosts -o StrictHostKeyChecking=yes" \
|
||||||
|
joshua@10.60.1.50:/volume1/homes/Joshua/mealquest-backups/media/ \
|
||||||
|
/var/lib/docker/volumes/mealquest-data/_data/media/
|
||||||
|
```
|
||||||
|
|
||||||
|
## Restore the database
|
||||||
|
|
||||||
|
SQLite in WAL mode keeps recent commits in `mealquest.db-wal`, indexed by
|
||||||
|
`mealquest.db-shm`. **A restored database must never sit beside an old WAL
|
||||||
|
or shared-memory file**: on open, SQLite would replay the old WAL's pages
|
||||||
|
onto the restored database and corrupt it or mix in state from the
|
||||||
|
database it replaced. So all three files are moved aside together, with
|
||||||
|
Litestream's local state directory `.mealquest.db-litestream`, before the
|
||||||
|
restore — and the restore runs into an empty volume.
|
||||||
|
|
||||||
|
Run from `/home/overseer/mealquest`:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
D=/var/lib/docker/volumes/mealquest-data/_data
|
||||||
|
A=/var/tmp/mealquest-restore-aside-$(date +%Y%m%d%H%M)
|
||||||
|
|
||||||
|
# 1. Stop the whole stack (app and Litestream). Nothing may hold the
|
||||||
|
# database open; a clean close checkpoints the WAL into mealquest.db.
|
||||||
|
docker compose -f deploy/compose.yaml down
|
||||||
|
|
||||||
|
# 2. Move the database, its -wal and -shm files and Litestream's state
|
||||||
|
# aside — all of them, every time. Keep the directory until the
|
||||||
|
# restored stack is verified.
|
||||||
|
sudo mkdir "$A"
|
||||||
|
for f in mealquest.db mealquest.db-wal mealquest.db-shm .mealquest.db-litestream; do
|
||||||
|
if sudo test -e "$D/$f"; then sudo mv "$D/$f" "$A/"; fi
|
||||||
|
done
|
||||||
|
sudo ls -la "$D" # must list only media/
|
||||||
|
|
||||||
|
# 3. Restore from the NAS. --no-deps keeps the app stopped so it cannot
|
||||||
|
# create a fresh, empty database first.
|
||||||
|
docker compose -f deploy/compose.yaml run --rm --no-deps litestream \
|
||||||
|
restore -config /etc/litestream.yml -o /data/mealquest.db /data/mealquest.db
|
||||||
|
sudo ls -la "$D" # mealquest.db only: no -wal, no -shm
|
||||||
|
|
||||||
|
# 4. Check the restored file before starting the app (see "Count check").
|
||||||
|
|
||||||
|
# 5. Start the stack and check health.
|
||||||
|
docker compose -f deploy/compose.yaml up -d
|
||||||
|
docker compose -f deploy/compose.yaml ps
|
||||||
|
curl -s -o /dev/null -w "health %{http_code}\n" http://10.60.1.14:8090/health
|
||||||
|
```
|
||||||
|
|
||||||
|
To restore to a point in time, add `-timestamp 2026-10-05T23:00:00Z` (or
|
||||||
|
`-txid`) to the `restore` command.
|
||||||
|
|
||||||
|
### Count check
|
||||||
|
|
||||||
|
`/tmp/mqcount.py` prints row counts for key tables, the number of
|
||||||
|
application tables, a SHA-256 over every application row (first 16 hex
|
||||||
|
digits; Litestream's own `_litestream*` tables are excluded) and
|
||||||
|
`PRAGMA integrity_check`. Run it on a **copy** of the database, never on the
|
||||||
|
live file; copy `-wal`/`-shm` along with it when the app is running.
|
||||||
|
|
||||||
|
```python
|
||||||
|
import sqlite3, sys, hashlib
|
||||||
|
c = sqlite3.connect(sys.argv[1])
|
||||||
|
tabs = [r[0] for r in c.execute("select name from sqlite_master where type=\"table\" and name not like \"sqlite_%\" and name not like \"_litestream%\" order by name")]
|
||||||
|
for t in ("dinners", "week_plans", "schema_migrations"):
|
||||||
|
print(t, c.execute(f"select count(*) from {t}").fetchone()[0])
|
||||||
|
h = hashlib.sha256()
|
||||||
|
for t in tabs:
|
||||||
|
for row in c.execute(f"select * from \"{t}\" order by rowid"):
|
||||||
|
h.update(repr(row).encode())
|
||||||
|
print("tables", len(tabs), "content-sha256", h.hexdigest()[:16])
|
||||||
|
print("integrity", c.execute("pragma integrity_check").fetchone()[0])
|
||||||
|
```
|
||||||
|
|
||||||
|
```sh
|
||||||
|
rm -rf /tmp/mqcheck && mkdir /tmp/mqcheck
|
||||||
|
sudo sh -c "cp -a $D/mealquest.db* /tmp/mqcheck/" && sudo chown -R "$USER" /tmp/mqcheck
|
||||||
|
python3 /tmp/mqcount.py /tmp/mqcheck/mealquest.db
|
||||||
|
```
|
||||||
|
|
||||||
|
Before and after must match exactly.
|
||||||
|
|
||||||
|
## Restore drill record — 2026-10-05
|
||||||
|
|
||||||
|
Drill on 10.60.1.14 against the live (pre-launch, empty-household)
|
||||||
|
database, Litestream 0.5.17. Commands as in "Restore the database";
|
||||||
|
output trimmed only of compose progress lines.
|
||||||
|
|
||||||
|
Replica before the drill (`ltx -level all`; level 9 is the full snapshot):
|
||||||
|
|
||||||
|
```
|
||||||
|
level min_txid max_txid size created
|
||||||
|
0 0000000000000003 0000000000000003 14110 2026-10-05T23:13:30Z
|
||||||
|
1 0000000000000001 0000000000000002 14110 2026-10-05T23:12:38Z
|
||||||
|
1 0000000000000003 0000000000000003 14110 2026-10-05T23:13:30Z
|
||||||
|
2 0000000000000001 0000000000000002 14110 2026-10-05T23:12:38Z
|
||||||
|
2 0000000000000003 0000000000000003 14110 2026-10-05T23:13:30Z
|
||||||
|
3 0000000000000001 0000000000000002 14110 2026-10-05T23:12:38Z
|
||||||
|
9 0000000000000001 0000000000000002 14110 2026-10-05T23:12:38Z
|
||||||
|
```
|
||||||
|
|
||||||
|
1. `docker compose -f deploy/compose.yaml down` — both containers stopped
|
||||||
|
and removed. The volume then held `mealquest.db`,
|
||||||
|
`.mealquest.db-litestream/` and `media/`; the clean close had
|
||||||
|
checkpointed and removed `-wal` and `-shm`.
|
||||||
|
2. Before (copy of the stopped database):
|
||||||
|
|
||||||
|
```
|
||||||
|
dinners 0
|
||||||
|
week_plans 0
|
||||||
|
schema_migrations 7
|
||||||
|
tables 26 content-sha256 836e230e16c58dc7
|
||||||
|
integrity ok
|
||||||
|
```
|
||||||
|
|
||||||
|
3. Moved aside to `/var/tmp/mealquest-restore-aside-20261005`:
|
||||||
|
`mealquest.db` and `.mealquest.db-litestream` (no `-wal`/`-shm` existed
|
||||||
|
after the clean stop; the loop moves them whenever they do). Volume
|
||||||
|
afterwards: `media/` only.
|
||||||
|
4. `docker compose -f deploy/compose.yaml run --rm --no-deps litestream restore -config /etc/litestream.yml -o /data/mealquest.db /data/mealquest.db`
|
||||||
|
— volume afterwards: `mealquest.db` (258048 bytes) and `media/`; no
|
||||||
|
`-wal`, no `-shm`.
|
||||||
|
5. After (copy of the restored file, before the app started):
|
||||||
|
|
||||||
|
```
|
||||||
|
dinners 0
|
||||||
|
week_plans 0
|
||||||
|
schema_migrations 7
|
||||||
|
tables 26 content-sha256 836e230e16c58dc7
|
||||||
|
integrity ok
|
||||||
|
```
|
||||||
|
|
||||||
|
6. `docker compose -f deploy/compose.yaml up -d` — `mealquest-mealquest-1`
|
||||||
|
healthy, `mealquest-litestream-1` up; `GET /health` → `200`. Counts on a
|
||||||
|
copy of the running database (`mealquest.db`, `-wal`, `-shm`) were
|
||||||
|
identical to the above.
|
||||||
|
7. Litestream resumed against the restored database:
|
||||||
|
|
||||||
|
```
|
||||||
|
level=INFO msg="detected database behind replica" db=mealquest.db db_txid=0000000000000000 replica_txid=0000000000000003
|
||||||
|
level=INFO msg="fetched latest L0 file from replica" db=mealquest.db min_txid=0000000000000003 max_txid=0000000000000003
|
||||||
|
level=INFO msg="compaction complete" db=mealquest.db level=1 txid.min=0000000000000004 txid.max=0000000000000004 size=14110
|
||||||
|
```
|
||||||
|
|
||||||
|
Replica after the drill (`ltx -level all`):
|
||||||
|
|
||||||
|
```
|
||||||
|
level min_txid max_txid size created
|
||||||
|
0 0000000000000004 0000000000000004 14110 2026-10-05T23:27:21Z
|
||||||
|
1 0000000000000001 0000000000000002 14110 2026-10-05T23:12:38Z
|
||||||
|
1 0000000000000003 0000000000000003 14110 2026-10-05T23:13:30Z
|
||||||
|
1 0000000000000004 0000000000000004 14110 2026-10-05T23:27:21Z
|
||||||
|
2 0000000000000001 0000000000000002 14110 2026-10-05T23:12:38Z
|
||||||
|
2 0000000000000003 0000000000000003 14110 2026-10-05T23:13:30Z
|
||||||
|
3 0000000000000001 0000000000000002 14110 2026-10-05T23:12:38Z
|
||||||
|
9 0000000000000001 0000000000000002 14110 2026-10-05T23:12:38Z
|
||||||
|
```
|
||||||
|
|
||||||
|
The first drill earlier the same day (before `-level all` was recorded)
|
||||||
|
gave the same before/after result: checksum `836e230e16c58dc7`, 26 tables,
|
||||||
|
integrity ok.
|
||||||
|
|
||||||
|
### Media backup checks — 2026-10-05
|
||||||
|
|
||||||
|
`sudo systemctl start mealquest-media-backup.service` on 10.60.1.14:
|
||||||
|
|
||||||
|
| Case | Result |
|
||||||
|
| --- | --- |
|
||||||
|
| Normal run | `Result=success`; journal: `synced /run/mealquest-data/media to joshua@10.60.1.50:/volume1/homes/Joshua/mealquest-backups/media/ as uid 10001` |
|
||||||
|
| A test file added | `Number of regular files transferred: 1`; removing it then gave `Number of deleted files: 1 (reg: 1)` |
|
||||||
|
| `media` replaced by a symlink to `/etc` | `Result=exit-code`, status 1; `/run/mealquest-data/media is a symlink to /etc; refusing to back it up` |
|
||||||
|
| `media` removed | `Result=exit-code`, status 1; `media directory /run/mealquest-data/media is missing` |
|
||||||
|
| `sudo -u mealquest` writing into the volume via `/var/lib/docker/...` | `Permission denied` — uid 10001 cannot traverse `/var/lib/docker` (mode 0710 root) |
|
||||||
@@ -0,0 +1,95 @@
|
|||||||
|
# Production stack for MealQuest on 10.60.1.14 (D-018). Build and start on
|
||||||
|
# the host with:
|
||||||
|
#
|
||||||
|
# docker compose -f deploy/compose.yaml up -d --build
|
||||||
|
#
|
||||||
|
# Configuration comes from deploy/.env (copy deploy/.env.example) and secrets
|
||||||
|
# from deploy/secrets/; neither is committed. Compose bind-mounts file secrets
|
||||||
|
# with their host owner and mode, so each file in deploy/secrets/ must be
|
||||||
|
# owned by the container user 10001:10001 with mode 0600.
|
||||||
|
name: mealquest
|
||||||
|
|
||||||
|
services:
|
||||||
|
mealquest:
|
||||||
|
build:
|
||||||
|
context: ..
|
||||||
|
dockerfile: Dockerfile
|
||||||
|
user: "10001:10001"
|
||||||
|
restart: unless-stopped
|
||||||
|
env_file:
|
||||||
|
- path: .env
|
||||||
|
required: true
|
||||||
|
# Fixed by this file's container shape (listen port, volume, secret
|
||||||
|
# mounts); deploy/.env must not set them.
|
||||||
|
environment:
|
||||||
|
MEALQUEST_HTTP_ADDR: ":8080"
|
||||||
|
MEALQUEST_DATA_DIR: /data
|
||||||
|
MEALQUEST_SECRET_KEY_FILE: /run/secrets/secret_key
|
||||||
|
MEALQUEST_OIDC_CLIENT_SECRET_FILE: /run/secrets/oidc_client_secret
|
||||||
|
secrets:
|
||||||
|
- secret_key
|
||||||
|
- oidc_client_secret
|
||||||
|
volumes:
|
||||||
|
- mealquest-data:/data
|
||||||
|
ports:
|
||||||
|
- "${MEALQUEST_BIND_ADDRESS:?set MEALQUEST_BIND_ADDRESS in deploy/.env}:8090:8080"
|
||||||
|
networks:
|
||||||
|
- mealquest
|
||||||
|
healthcheck:
|
||||||
|
test: ["CMD", "wget", "-q", "-O", "/dev/null", "http://127.0.0.1:8080/health"]
|
||||||
|
interval: 30s
|
||||||
|
timeout: 3s
|
||||||
|
start_period: 10s
|
||||||
|
retries: 3
|
||||||
|
security_opt:
|
||||||
|
- no-new-privileges:true
|
||||||
|
cap_drop:
|
||||||
|
- ALL
|
||||||
|
|
||||||
|
# Litestream streams the database to the Synology NAS over SFTP
|
||||||
|
# (config: deploy/litestream.yml). The SFTP private key
|
||||||
|
# deploy/secrets/litestream_key is a dedicated keypair, owned by
|
||||||
|
# 10001:10001 with mode 0600 like the other secrets.
|
||||||
|
#
|
||||||
|
# The data volume is mounted read-write on purpose: Litestream writes its
|
||||||
|
# tracking tables into the database and checkpoints the WAL. It runs as
|
||||||
|
# the app's own user (owner of the volume) with a read-only root
|
||||||
|
# filesystem, no privilege escalation and no Linux capabilities.
|
||||||
|
#
|
||||||
|
# Restore: follow "Restore the database" in deploy/RUNBOOK.md (stop the
|
||||||
|
# stack, move the database with its -wal and -shm aside, restore, check).
|
||||||
|
litestream:
|
||||||
|
image: litestream/litestream:0.5.17
|
||||||
|
user: "10001:10001"
|
||||||
|
restart: unless-stopped
|
||||||
|
command: ["replicate", "-config", "/etc/litestream.yml"]
|
||||||
|
depends_on:
|
||||||
|
mealquest:
|
||||||
|
condition: service_healthy
|
||||||
|
volumes:
|
||||||
|
- mealquest-data:/data
|
||||||
|
- ./litestream.yml:/etc/litestream.yml:ro
|
||||||
|
- ./secrets/litestream_key:/etc/litestream/sftp_key:ro
|
||||||
|
networks:
|
||||||
|
- mealquest
|
||||||
|
read_only: true
|
||||||
|
tmpfs:
|
||||||
|
- /tmp
|
||||||
|
security_opt:
|
||||||
|
- no-new-privileges:true
|
||||||
|
cap_drop:
|
||||||
|
- ALL
|
||||||
|
|
||||||
|
secrets:
|
||||||
|
secret_key:
|
||||||
|
file: ./secrets/secret_key
|
||||||
|
oidc_client_secret:
|
||||||
|
file: ./secrets/oidc_client_secret
|
||||||
|
|
||||||
|
volumes:
|
||||||
|
mealquest-data:
|
||||||
|
name: mealquest-data
|
||||||
|
|
||||||
|
networks:
|
||||||
|
mealquest:
|
||||||
|
name: mealquest
|
||||||
@@ -0,0 +1,6 @@
|
|||||||
|
# Host keys of the Synology NAS (10.60.1.50), the backup target. Installed
|
||||||
|
# root-owned with mode 0644 as /etc/mealquest/known_hosts and pinned by the
|
||||||
|
# media backup (UserKnownHostsFile, StrictHostKeyChecking=yes). The ECDSA
|
||||||
|
# key is also pinned in deploy/litestream.yml (host-key).
|
||||||
|
10.60.1.50 ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIKHxIfgjavBABMRE5GW/EFY4C4gO6mo0sKbhkUW2NcAo
|
||||||
|
10.60.1.50 ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNujay9nYEiXuRQf4wDRDU1gp+CDvpeMdhuhC2XypIihzKGEtB654kwrUGfA2kDWAoSPGlmojeS3OIRtHPkj6GY=
|
||||||
@@ -0,0 +1,31 @@
|
|||||||
|
# Litestream continuous backup of the MealQuest SQLite database (Feature
|
||||||
|
# 07). The ``litestream`` compose service mounts the same named volume as
|
||||||
|
# the app and streams every committed change to the Synology NAS over
|
||||||
|
# SFTP, so the backup lives outside the Docker host.
|
||||||
|
#
|
||||||
|
# Nothing here is secret. The SFTP private key is a dedicated, git-ignored
|
||||||
|
# keypair (deploy/secrets/litestream_key) mounted read-only by compose at
|
||||||
|
# /etc/litestream/sftp_key; its public half is in
|
||||||
|
# joshua@10.60.1.50:~/.ssh/authorized_keys. ``host-key`` pins the NAS's
|
||||||
|
# ECDSA host key, the algorithm Litestream's SSH client negotiates with
|
||||||
|
# this NAS.
|
||||||
|
#
|
||||||
|
# The Synology SFTP server is chrooted to the shared folders, so the
|
||||||
|
# directory seen over SSH as /volume1/homes/Joshua/mealquest-backups is
|
||||||
|
# /homes/Joshua/mealquest-backups over SFTP.
|
||||||
|
|
||||||
|
# Take a full snapshot daily and keep a week of snapshots plus the
|
||||||
|
# incremental changes between them (point-in-time restore window).
|
||||||
|
snapshot:
|
||||||
|
interval: 24h
|
||||||
|
retention: 168h
|
||||||
|
|
||||||
|
dbs:
|
||||||
|
- path: /data/mealquest.db
|
||||||
|
replicas:
|
||||||
|
- type: sftp
|
||||||
|
host: 10.60.1.50
|
||||||
|
user: joshua
|
||||||
|
path: /homes/Joshua/mealquest-backups
|
||||||
|
key-path: /etc/litestream/sftp_key
|
||||||
|
host-key: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNujay9nYEiXuRQf4wDRDU1gp+CDvpeMdhuhC2XypIihzKGEtB654kwrUGfA2kDWAoSPGlmojeS3OIRtHPkj6GY=
|
||||||
@@ -0,0 +1,22 @@
|
|||||||
|
# Applies the MealQuest DOCKER-USER rules (/usr/local/sbin/mealquest-firewall).
|
||||||
|
#
|
||||||
|
# PartOf= re-runs this unit whenever docker.service is stopped or restarted,
|
||||||
|
# and WantedBy=docker.service starts it whenever docker.service starts. Both
|
||||||
|
# are one-way: a failure here fails this unit visibly and never fails
|
||||||
|
# docker.service.
|
||||||
|
#
|
||||||
|
# Start MealQuest only after this unit has succeeded:
|
||||||
|
# systemctl start mealquest-firewall && docker compose up -d
|
||||||
|
[Unit]
|
||||||
|
Description=Restrict MealQuest port 8090 to Traefik (DOCKER-USER)
|
||||||
|
After=docker.service
|
||||||
|
Requires=docker.service
|
||||||
|
PartOf=docker.service
|
||||||
|
|
||||||
|
[Service]
|
||||||
|
Type=oneshot
|
||||||
|
RemainAfterExit=yes
|
||||||
|
ExecStart=/usr/local/sbin/mealquest-firewall
|
||||||
|
|
||||||
|
[Install]
|
||||||
|
WantedBy=multi-user.target docker.service
|
||||||
Executable
+29
@@ -0,0 +1,29 @@
|
|||||||
|
#!/bin/sh
|
||||||
|
# Restrict the MealQuest published host port (10.60.1.14:8090) to Traefik
|
||||||
|
# (10.60.1.33) and the Docker host itself (10.60.1.14); drop every other
|
||||||
|
# source.
|
||||||
|
#
|
||||||
|
# DOCKER-USER runs after Docker's DNAT, so the destination there is the
|
||||||
|
# container address and port, not the published ones. Match the original
|
||||||
|
# destination address and port with conntrack instead of -d/--dport.
|
||||||
|
#
|
||||||
|
# The rules must sit at the top of DOCKER-USER, ahead of any other project's
|
||||||
|
# rules (an earlier RETURN or DROP would decide MealQuest traffic first).
|
||||||
|
# Every run deletes all rules tagged with the mealquest-firewall comment and
|
||||||
|
# re-inserts them at positions 1-3, so re-runs leave the same ordered rules
|
||||||
|
# and no duplicates. Installed as /usr/local/sbin/mealquest-firewall.
|
||||||
|
set -eu
|
||||||
|
|
||||||
|
tag="-m comment --comment mealquest-firewall"
|
||||||
|
match="-m conntrack --ctorigdst 10.60.1.14 --ctorigdstport 8090 --ctdir ORIGINAL"
|
||||||
|
|
||||||
|
rules=$(iptables -w -S DOCKER-USER)
|
||||||
|
printf '%s\n' "$rules" | while read -r _ chain spec; do
|
||||||
|
case " $spec " in
|
||||||
|
*" --comment mealquest-firewall "*) iptables -w -D "$chain" $spec ;;
|
||||||
|
esac
|
||||||
|
done
|
||||||
|
|
||||||
|
iptables -w -I DOCKER-USER 1 -s 10.60.1.33/32 $match $tag -j RETURN
|
||||||
|
iptables -w -I DOCKER-USER 2 -s 10.60.1.14/32 $match $tag -j RETURN
|
||||||
|
iptables -w -I DOCKER-USER 3 $match $tag -j DROP
|
||||||
@@ -0,0 +1,29 @@
|
|||||||
|
# Runs the nightly MealQuest media backup (/usr/local/sbin/mealquest-media-backup).
|
||||||
|
# Triggered by mealquest-media-backup.timer; run by hand with
|
||||||
|
# systemctl start mealquest-media-backup
|
||||||
|
# and read the result with
|
||||||
|
# journalctl -u mealquest-media-backup
|
||||||
|
#
|
||||||
|
# The script runs as the app's user (10001, host user "mealquest") with no
|
||||||
|
# capabilities. systemd, as root, only bind-mounts the data volume's root
|
||||||
|
# (a root-owned path the app cannot replace) read-only at
|
||||||
|
# /run/mealquest-data and copies the NAS key into the credentials directory.
|
||||||
|
[Unit]
|
||||||
|
Description=Back up MealQuest media to the Synology NAS
|
||||||
|
After=network-online.target
|
||||||
|
Wants=network-online.target
|
||||||
|
|
||||||
|
[Service]
|
||||||
|
Type=oneshot
|
||||||
|
ExecStart=/usr/local/sbin/mealquest-media-backup
|
||||||
|
User=mealquest
|
||||||
|
Group=mealquest
|
||||||
|
LoadCredential=nas_key:/home/overseer/mealquest/deploy/secrets/litestream_key
|
||||||
|
BindReadOnlyPaths=/var/lib/docker/volumes/mealquest-data/_data:/run/mealquest-data
|
||||||
|
ProtectSystem=strict
|
||||||
|
ProtectHome=yes
|
||||||
|
PrivateTmp=yes
|
||||||
|
PrivateDevices=yes
|
||||||
|
NoNewPrivileges=yes
|
||||||
|
CapabilityBoundingSet=
|
||||||
|
RestrictAddressFamilies=AF_INET AF_INET6 AF_UNIX
|
||||||
Executable
+44
@@ -0,0 +1,44 @@
|
|||||||
|
#!/bin/sh
|
||||||
|
# Nightly copy of the MealQuest media directory (recipe and dinner photos)
|
||||||
|
# to the Synology NAS. The database itself is replicated continuously by
|
||||||
|
# Litestream; this covers the files beside it.
|
||||||
|
#
|
||||||
|
# Runs as the unprivileged app user (10001, host user "mealquest"), never
|
||||||
|
# root: mealquest-media-backup.service bind-mounts the data volume
|
||||||
|
# read-only at /run/mealquest-data and hands the NAS key in as a systemd
|
||||||
|
# credential. Anything the app could plant in the volume is therefore read
|
||||||
|
# with the app's own permissions only.
|
||||||
|
#
|
||||||
|
# The script changes into the media directory and checks that the physical
|
||||||
|
# path is the expected one, so a media path that is a symlink is refused.
|
||||||
|
# rsync then copies "./" — the directory the shell is already in — so
|
||||||
|
# swapping the path afterwards cannot redirect it. Symlinks inside the
|
||||||
|
# directory are copied as links (-a), never followed.
|
||||||
|
#
|
||||||
|
# The media directory is created at deploy time (deploy/RUNBOOK.md). If it
|
||||||
|
# is missing the job fails; it is never treated as "nothing to back up".
|
||||||
|
#
|
||||||
|
# rsync mirrors into .../mealquest-backups/media/ only: the target ends in
|
||||||
|
# a slash, so --delete removes files from that directory and nothing
|
||||||
|
# outside it. Re-runs transfer only what changed. Output goes to the
|
||||||
|
# journal via mealquest-media-backup.service.
|
||||||
|
set -eu
|
||||||
|
|
||||||
|
src=/run/mealquest-data/media
|
||||||
|
dest=joshua@10.60.1.50:/volume1/homes/Joshua/mealquest-backups/media/
|
||||||
|
key=$CREDENTIALS_DIRECTORY/nas_key
|
||||||
|
known_hosts=/etc/mealquest/known_hosts
|
||||||
|
|
||||||
|
if ! cd "$src"; then
|
||||||
|
echo "mealquest-media-backup: media directory $src is missing" >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
if [ "$(pwd -P)" != "$src" ]; then
|
||||||
|
echo "mealquest-media-backup: $src is a symlink to $(pwd -P); refusing to back it up" >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
rsync -a --delete --stats \
|
||||||
|
-e "ssh -F none -i $key -o IdentitiesOnly=yes -o BatchMode=yes -o UserKnownHostsFile=$known_hosts -o GlobalKnownHostsFile=/dev/null -o StrictHostKeyChecking=yes" \
|
||||||
|
./ "$dest"
|
||||||
|
echo "mealquest-media-backup: synced $src to $dest as uid $(id -u)"
|
||||||
@@ -0,0 +1,11 @@
|
|||||||
|
# Nightly trigger for mealquest-media-backup.service. Persistent= runs a
|
||||||
|
# missed backup at the next boot.
|
||||||
|
[Unit]
|
||||||
|
Description=Nightly MealQuest media backup
|
||||||
|
|
||||||
|
[Timer]
|
||||||
|
OnCalendar=*-*-* 03:30:00
|
||||||
|
Persistent=true
|
||||||
|
|
||||||
|
[Install]
|
||||||
|
WantedBy=timers.target
|
||||||
Reference in New Issue
Block a user