Compare commits

..
Author SHA1 Message Date
overseerandClaude Opus 5.5 442339f887 Contain the Litestream sidecar's write access to the database volume
Review finding F-01: the litestream service mounts nestquest-db
read-write. A read-only mount is not possible: Litestream 0.5.17 fails
with "unable to open database file (14)" on a :ro mount, even with
meta-path moved to a separate writable volume, because it writes its
_litestream_seq/_litestream_lock tables into the database, opens the
-wal/-shm files for writing and checkpoints the WAL.

Keep the write mount and contain it instead: run as the API's non-root
volume owner (999:999) with a read-only root filesystem (tmpfs /tmp),
no-new-privileges and all capabilities dropped. Document why in
compose and api/README.md, including the key chown the uid change needs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-24 08:28:57 -04:00
overseerandClaude Opus 5.5 89a10edc38 Add Litestream continuous backup of the SQLite database
Task 47b27ccb: a litestream sidecar (litestream/litestream:0.5.17) in
deploy/compose.yaml mounts the nestquest-db volume and replicates
/var/lib/nestquest/nestquest.db over SFTP to the Synology NAS
(10.60.1.50, /homes/Joshua/nestquest-backups), outside the Docker host.
deploy/litestream.yml holds no secrets: the NAS host key is pinned and
the dedicated SFTP private key is mounted read-only from the
git-ignored deploy/litestream_key. api/README.md documents what runs,
where it writes, key setup, and a high-level restore.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-24 08:15:53 -04:00
dev_agent f6d222764c Task 9d121e76: Reverse proxy and TLS for the API (#214) 2026-09-24 08:00:04 -04:00
dev_agent 9b068926a9 Containerize the API with a persistent volume (#183) 2026-09-23 12:41:44 -04:00
14 changed files with 14 additions and 860 deletions
+2 -6
View File
@@ -14,16 +14,12 @@
# the SQLite database lives on the mounted volume at
# /var/lib/nestquest (compose supplies NESTQUEST_DB_PATH pointing
# there — configuration is environment-supplied, never baked in).
#
# Both stages use the same base image pinned by digest, and
# api/requirements.txt pins exact versions, so a rebuild reproduces
# the deployed image instead of floating to whatever is newest.
FROM python:3.12-slim@sha256:2f17fc044b579bab302c2e8054d3a686e2cb9a83de48e70534b94cd8ebbe06a9 AS builder
FROM python:3.12-slim AS builder
COPY api/requirements.txt /tmp/requirements.txt
RUN pip install --no-cache-dir --prefix=/install -r /tmp/requirements.txt
FROM python:3.12-slim@sha256:2f17fc044b579bab302c2e8054d3a686e2cb9a83de48e70534b94cd8ebbe06a9
FROM python:3.12-slim
# The staged dependency install from the builder (no pip, no wheels).
COPY --from=builder /install /usr/local
-131
View File
@@ -1,136 +1,5 @@
# NestQuest Release Notes
## Version 0.7.0 — 2026-09-24
### Scope
Seventh release of NestQuest, promoting the re-imagined architecture's admin
surface from `dev` onto 0.6.0: the API deployment and identity layer, the
admin-plane API work, and the standalone admin PWA that uses them.
- **Feature 17 — API Deployment and Identity:** the NestQuest API is packaged as
a container (`Dockerfile`, `deploy/compose.yaml`, pinned base image and exact
dependencies) and deployed behind a TLS reverse proxy (the shared Traefik, ACME
via the Cloudflare DNS-01 challenge); the Authentik application, OIDC provider
and `nestquest-admins` group are wired and the API validates the provider's
issuer/audience/JWKS (RS256); the panel plane is restricted to the Home
Assistant box while the admin plane is internet-facing behind Authentik;
continuous backup runs as a contained Litestream sidecar over SFTP to a NAS and
a restore is proven; a deployment runbook (`deploy/RUNBOOK.md`) and the
host-firewall unit (`deploy/systemd/nestquest-api-firewall.service`) ship.
(PRs #183, #214, #215, #216, #218.)
- **Feature 19 — Admin PWA:** the parent-facing admin surface is now a
standalone mobile-first React progressive web app (`admin/`), implementing
`design/ADMIN-SPEC.md`: Authentik OIDC authorization-code + PKCE login with
an explicit refusal for a user outside `nestquest-admins`; an installable
PWA (manifest + service worker); and the Today, Definitions, Schedule and
History screens. Every mutating action goes through the API admin routes;
the API remains the security boundary (D-003/D-012). Today shows the
household snapshot; Definitions supports create/edit with multi-select
assignees, a recurrence rule with interval and per-window due times, a
skip-on-away toggle, a shipped Lucide icon subset and a backend-computed
occurrence preview; Schedule has a presence month grid (cycle position from
the anchor date, never ISO parity, D-004) and an override editor with a live
"removes N upcoming tasks" consequence count; History has All/Reversals/
Missed filters, day-grouped rows, per-period stats, CSV export and the
append-only footer. Undo reverses a completion, and live updates arrive over
the admin SSE stream without a manual reload. (PRs #187–#210, plus the
feature-review fix #212.)
- **Feature 21 — Admin Today Snapshot API:** `GET /api/v1/admin/snapshot`
serves the Today view's household snapshot on the admin plane (reusing the
core snapshot builder with missed instances included, D-009). (PRs #190/#191.)
- **Feature 22 — Admin Definitions Read and skip_on_away:**
`GET /api/v1/admin/quest-definitions` lists all definitions (active and
inactive), and a persisted per-definition `skip_on_away` boolean is accepted
by create/update and honoured by materialization. Adds an optional
`assignee_child_ids` to the edit route so the PWA can reassign children.
(PRs #193, #194, #196, #197.)
- **Feature 23 — Admin API Completion (preview, presence, live):** a bounded
rule occurrence-preview route; presence-schedule and presence-override
reads; a live override consequence count computed from the same core
generator the materializer uses; and an admin-plane transition SSE stream
(`GET /api/v1/admin/events`) off the one in-process publisher. (PRs
#199–#203.)
### Requirements
- Minimum Home Assistant version: **2024.6.0**.
- A running NestQuest API service, reachable at the base URL and panel service
token configured in the integration options. The deployable ships in this
release: `Dockerfile`, `deploy/compose.yaml` (API + a contained Litestream
backup sidecar), `api/requirements.txt`, the host-firewall unit and
`deploy/RUNBOOK.md`. Deploying it needs a host, the DNS/TLS path and the
Authentik/Synology services described in the runbook.
- The admin PWA is built to run at `https://nestquest.cubecraftlabs.com` behind
Authentik. Serving the built `admin/` static bundle at that hostname is part of
the deployment described in the runbook.
### Breaking changes
- **Schema version advances 7 → 8.** Migration 8 adds the additive
`skip_on_away` column to `quest_definitions` (default 1, so every existing
definition keeps today's skip-when-absent behaviour). The migration runs on
the first start of this version.
- **Once migrated, a plain downgrade to 0.6.0 is impossible** — older builds
refuse a newer schema version. **Back up `nestquest.db` (plus `-wal`/`-shm`)
before upgrading**; see Rollback.
- No Home Assistant entity, service, event or automation surface changes in
this release. The API's new admin routes are additive.
### Known Limitations
- **The API and its identity/backup layer are deployed by the owner's
infrastructure, not by this release alone.** The deployment artifacts and
runbook ship here, but a running service still requires the host, the shared
reverse proxy, the Authentik application/provider/`nestquest-admins` group, the
backup target and the panel/admin network separation to be in place as the
runbook describes.
- **Serving the admin PWA is still outstanding.** `admin/` builds and is
unit/integration tested and the API it talks to is live, but the built static
bundle must still be hosted at `https://nestquest.cubecraftlabs.com` behind
Authentik; until then the PWA is not reachable end-to-end.
- **Feature 12's TouchHub/DAKOS runbook task remains physically unverified**
(needs the wall panel).
- Moving the household database from Home Assistant to the API box is a manual
file copy (the schema is unchanged by that move beyond the v8 migration).
- Single household; single API service.
### Rollback
**Schema version advances 7 → 8.** Migration 8 is additive, but a migrated
database **cannot** be read by 0.6.0 or earlier (they refuse a newer schema
version). The database is owned and migrated by the **NestQuest API service**,
not by Home Assistant: it is the file at the API's configured
`NESTQUEST_DB_PATH` on the API host, which need not be anywhere near the Home
Assistant config directory. To return to 0.6.0 you must restore a backup of
that file taken before the upgrade; a plain downgrade is impossible once
migrated.
**API host (database):**
1. Before upgrading, stop the NestQuest API service and back up the database
file at its configured `NESTQUEST_DB_PATH`, plus its `-wal` and `-shm`
sidecars (`<NESTQUEST_DB_PATH>-wal`, `<NESTQUEST_DB_PATH>-shm`) if present.
Only then deploy 0.7.0 and start the API service, which applies migration 8.
2. To roll back: stop the API service, restore the backed-up database and
sidecars to `NESTQUEST_DB_PATH`, redeploy the API service at 0.6.0, and roll
back the Home Assistant integration to 0.6.0 (below). Then restart the API
service and Home Assistant.
**Home Assistant integration — HACS installs** (integration component only;
the database lives with the API, see above):
1. To roll back: HACS > Integrations > NestQuest > Redownload > 0.6.0, then
restart Home Assistant.
**Home Assistant integration — Manual installs** (integration component only;
the database lives with the API, see above):
1. To roll back: replace `custom_components/nestquest/` with the 0.6.0 tree,
then restart Home Assistant.
**Repository operators:** git-revert the 0.7.0 release merge on `main`.
## Version 0.6.0 — 2026-09-23
### Scope
+2 -2
View File
@@ -36,8 +36,8 @@ which serves ``https://nestquest.cubecraftlabs.com`` (Let's Encrypt,
auto-renewed, HTTP redirected to HTTPS) and forwards to the API box
on port 8080 — so on that box set ``NESTQUEST_BIND_ADDRESS`` to its
LAN address (``10.60.1.14``). Traefik only allows
``/api/v1/panel`` from the Home Assistant host. The full deployment,
restore and verification procedure is in ``deploy/RUNBOOK.md``.
``/api/v1/panel`` from the Home Assistant host. Network separation
and the full restore runbook are later Feature 17 tasks.
## Continuous backup (Litestream)
+6 -29
View File
@@ -7,32 +7,9 @@
# runtime dependencies HERE — this file is what the container image
# installs (see the Dockerfile at the repository root).
#
# EXACT pins: the versions installed in the deployed image
# (nestquest-api:latest on the API box, read with importlib.metadata),
# so a rebuild resolves the same packages. The four direct
# dependencies come first; the rest is their resolved transitive
# closure, pinned too so nothing floats. The dev group's floors in
# pyproject.toml are unchanged. Bump deliberately, together with the
# Dockerfile's digest-pinned base image.
fastapi==0.141.1
uvicorn==0.53.0
pyjwt[crypto]==2.15.0
httpx==0.28.1
# Transitive dependencies of the above.
annotated-doc==0.0.5
annotated-types==0.8.0
anyio==4.15.1
certifi==2026.7.22
cffi==2.1.1
click==8.5.0
cryptography==50.0.1
h11==0.16.0
httpcore==1.0.9
idna==3.20
pycparser==3.0
pydantic==2.13.5
pydantic_core==2.46.5
starlette==1.7.0
typing-inspection==0.4.4
typing_extensions==4.16.0
# Compatible ranges mirroring the dev group's floors; the API is a
# server, so these are the only packages its runtime needs.
fastapi>=0.115
uvicorn>=0.30
pyjwt[crypto]>=2.9
httpx>=0.28
+1 -1
View File
@@ -15,5 +15,5 @@
"iot_class": "local_polling",
"issue_tracker": "https://github.com/Maniacs-Mansion/NestQuest/issues",
"requirements": [],
"version": "0.7.0"
"version": "0.6.0"
}
-11
View File
@@ -21,14 +21,3 @@ NESTQUEST_OIDC_JWKS_URL=
# unset; on the API box set the LAN address the reverse proxy
# forwards to, e.g. 10.60.1.14.
NESTQUEST_BIND_ADDRESS=
# Admin PWA build args (compose interpolation into the pwa service's
# VITE_* build args; baked into the bundle, so rebuild after changing).
# PUBLIC values, not secrets: the Authentik application is a PKCE client
# with no client secret. All three are required — the pwa image build
# fails when any is empty.
# NESTQUEST_AUTHENTIK_ISSUER=https://auth.cubecraftlabs.com/application/o/nestquest/
# NESTQUEST_API_BASE_URL=https://nestquest.cubecraftlabs.com
NESTQUEST_AUTHENTIK_CLIENT_ID=
NESTQUEST_AUTHENTIK_ISSUER=
NESTQUEST_API_BASE_URL=
-553
View File
@@ -1,553 +0,0 @@
# NestQuest API — deployment and restore runbook
Operator procedure to rebuild the production NestQuest API deployment end
to end from this repository, restore its database from backup, and verify
it. Every value below is the real, non-secret production value. **No
secret appears in this file** — secrets are named and their location is
given, never their content.
| Piece | Where | Real value |
| --- | --- | --- |
| API box (Docker host) | this repository's `deploy/` | `10.60.1.14`, port `8080` (API), `8081` (admin PWA, section 10) |
| Reverse proxy / TLS | shared Traefik on `nukapi-03` | `10.60.1.33` |
| Public hostname | Cloudflare DNS | `nestquest.cubecraftlabs.com` |
| Identity | Authentik | `https://auth.cubecraftlabs.com` |
| Home Assistant box (panel client) | LAN | `10.60.1.80` |
| Local DNS | Pi-hole | `10.60.1.101` |
| Backup target | Synology NAS `CubeCraft-SRVR1`, SFTP | `10.60.1.50:/homes/Joshua/nestquest-backups` |
All `docker compose` commands below are run **on the API box, from the
repository root**. The shorthand `dc` means:
alias dc='docker compose -f deploy/compose.yaml'
Compose v2.24 or newer is required (`env_file` uses the `path`/`required`
form).
---
## 1. Image build
The image is defined by the root `Dockerfile` and is built from the
**repository root** as the build context: the API (`api/`) loads the
bundled domain core from `custom_components/nestquest/core/` by file
location, so both trees must be in the context. `.dockerignore` keeps
everything else (including `deploy/`, `.env` files and `*.db`) out of the
build.
- Multi-stage on `python:3.12-slim`: the builder installs
`api/requirements.txt`, the runtime copies only the installed packages
plus `api/` and `custom_components/nestquest/core/` into `/app`.
- Runs as the non-root system user `nestquest` (uid/gid `999`), which owns
`/var/lib/nestquest`.
- Command: `uvicorn --factory api.app:create_app --host 0.0.0.0 --port 8080`.
Get the code and build:
git clone ssh://code.cubecraftlabs.com:2288/Maniacs_Mansion/NestQuest.git
cd NestQuest
git checkout <release branch or tag being deployed>
dc build api # equivalent: docker build -t nestquest-api:latest .
Confirm the runtime user (the Litestream sidecar and the key ownership in
section 5 depend on uid 999):
docker run --rm --entrypoint id nestquest-api:latest
# uid=999(nestquest) gid=999(nestquest) groups=999(nestquest)
## 2. Compose
`deploy/compose.yaml` defines three services and one named volume:
| Service | Image | Role |
| --- | --- | --- |
| `api` | `nestquest-api:latest` (built, section 1) | FastAPI service on `8080`, healthcheck on `/health` |
| `litestream` | `litestream/litestream:0.5.17` | Continuous SFTP backup of the database (section 7); starts only once `api` is healthy |
| `pwa` | `nestquest-pwa:latest` (built from `deploy/pwa`, section 10) | Admin PWA served by nginx on `8081`; independent of the other two |
- **Volume `nestquest-db`** is mounted at `/var/lib/nestquest` in both
services. The database is `NESTQUEST_DB_PATH=/var/lib/nestquest/nestquest.db`
(fixed by compose, not a secret). `docker compose down` keeps named
volumes; **never run `down -v`** on the API box. Docker prefixes the
volume with the compose project name — with the default project
(`deploy`, the compose file's directory) it is `deploy_nestquest-db`;
confirm with `docker volume ls | grep nestquest-db`.
- **Published port**: `${NESTQUEST_BIND_ADDRESS:-127.0.0.1}:8080:8080`.
The default binds localhost only, so an unconfigured host is never
exposed. On the API box `deploy/.env` sets
`NESTQUEST_BIND_ADDRESS=10.60.1.14` so Traefik on nukapi-03 can reach
`http://10.60.1.14:8080`.
- **Secrets** come from `deploy/.env` (git-ignored, section 5) via
`env_file`.
Bring the stack up (after section 5's secrets exist):
dc up -d --build
dc ps # api "healthy", litestream "Up"
curl -s http://10.60.1.14:8080/health # {"status":"ok"}
## 3. TLS and reverse proxy (Traefik on nukapi-03)
TLS is terminated by the **shared** Traefik instance on `nukapi-03`
(`10.60.1.33`), not by this repository. Its NestQuest routing is the
owner-authored dynamic file-provider config
**`config/dynamic/nestquest-ccl.yml`** in the Traefik deployment on
nukapi-03 — that live file is authoritative; it is not stored in this
repository. What it must contain:
| Router | Rule | Middleware | Service |
| --- | --- | --- | --- |
| `nestquest-panel` (priority 200) | ``Host(`nestquest.cubecraftlabs.com`) && PathPrefix(`/api/v1/panel`)`` | `nestquest-panel-lan`: `ipAllowList`, `sourceRange: ["10.60.1.80/32"]` (HA box only) | `nestquest-api` → `http://10.60.1.14:8080` |
| `nestquest-admin` (priority 100: `/api/v1/admin`, `/health`, `/openapi.json`, `/docs`) | ``Host(`nestquest.cubecraftlabs.com`) && (PathPrefix(`/api/v1/admin`) \|\| Path(`/health`) \|\| Path(`/openapi.json`) \|\| PathPrefix(`/docs`))`` | `nestquest-headers` — no source restriction; the API itself enforces Authentik JWTs (section 4) | `nestquest-api` → `http://10.60.1.14:8080` |
| `nestquest-app` (priority 50: everything else — the admin PWA) | see section 10 | none at Traefik; nginx allow-lists the proxy | `nestquest-pwa` → `http://10.60.1.14:8081` |
All routers use the HTTPS entrypoint `websecure` with `tls.certResolver: myresolver`
(HTTP is redirected to HTTPS by the shared Traefik). Priorities are set
explicitly so the order never depends on rule length: panel (200) above
admin (100), and the PWA's `nestquest-app` router (50) below both. The
admin router is scoped to `/api/v1/admin`, `/health`, `/openapi.json`
and `/docs` — a bare ``Host(...)`` rule at a higher priority would
swallow `/` and every PWA route. Any other `/api/*` path matches no
router (the app router excludes `/api`), so Traefik answers `404`.
Reference shape, to rebuild the file if it is ever lost (entrypoint names
must match the shared Traefik's static config on nukapi-03):
```yaml
http:
routers:
nestquest-panel:
rule: "Host(`nestquest.cubecraftlabs.com`) && PathPrefix(`/api/v1/panel`)"
priority: 200
entryPoints: [websecure]
middlewares: [nestquest-panel-lan]
service: nestquest-api
tls:
certResolver: myresolver
nestquest-admin:
rule: "Host(`nestquest.cubecraftlabs.com`) && (PathPrefix(`/api/v1/admin`) || Path(`/health`) || Path(`/openapi.json`) || PathPrefix(`/docs`))"
priority: 100
entryPoints: [websecure]
middlewares: [nestquest-headers]
service: nestquest-api
tls:
certResolver: myresolver
nestquest-app:
rule: "Host(`nestquest.cubecraftlabs.com`) && !PathPrefix(`/api`) && !Path(`/health`) && !Path(`/openapi.json`) && !PathPrefix(`/docs`)"
priority: 50
entryPoints: [websecure]
service: nestquest-pwa
tls:
certResolver: myresolver
middlewares:
nestquest-panel-lan:
ipAllowList:
sourceRange: ["10.60.1.80/32"]
nestquest-headers:
# headers middleware; copy its definition from the live file
services:
nestquest-api:
loadBalancer:
servers:
- url: "http://10.60.1.14:8080"
nestquest-pwa:
loadBalancer:
servers:
- url: "http://10.60.1.14:8081"
```
**Certificates.** `myresolver` is the shared Traefik's ACME (Let's
Encrypt) resolver using the **Cloudflare DNS-01 challenge**; the
Cloudflare API token is held by the Traefik deployment on
nukapi-03, never in this repository. Because DNS-01 proves control
through a DNS TXT record, issuance and renewal need no inbound HTTP and
work regardless of the Cloudflare proxy setting. Certificates renew
automatically.
**Cloudflare proxy note.** Public traffic for
`nestquest.cubecraftlabs.com` resolves through Cloudflare, so a request
from the internet reaches Traefik from a Cloudflare edge address, never
from `10.60.1.80` — which is exactly why the panel allowlist refuses it
(403). The HA box does not go through Cloudflare at all: the local DNS
override in section 6 sends it straight to Traefik on the LAN, so Traefik
sees its real address. Do **not** add Cloudflare ranges to the panel
allowlist or make the allowlist trust `X-Forwarded-For`; that would let
any internet client through to the panel plane.
## 4. Authentik (admin identity)
Admin-plane requests (`/api/v1/admin/*`) carry an Authentik OIDC access
token (JWT). The API verifies issuer, audience, RS256 signature against
the JWKS, and expiry, then requires the `groups` claim to contain
`nestquest-admins` (`api/auth.py`). Configure in Authentik
(`https://auth.cubecraftlabs.com`, admin interface):
1. **Provider** — *OAuth2/OpenID Provider*:
- Client type: **Public** (no client secret; the admin PWA is a browser
SPA).
- PKCE: the PWA always uses the authorization-code flow with PKCE,
`code_challenge_method=S256`, and never sends a client secret.
- Redirect URI (strict): `https://nestquest.cubecraftlabs.com/auth/callback`.
- Signing key: an **RSA** certificate (the API only accepts `RS256`).
- Scopes: the default `openid`, `email`, `profile` mappings (the PWA
requests `openid email profile`; Authentik's default `profile`
mapping emits the `groups` claim the API checks).
- Note the generated **Client ID** — it is the audience below and the
PWA's `VITE_AUTHENTIK_CLIENT_ID`.
2. **Application** — slug **`nestquest`**, bound to the provider above.
The slug fixes the issuer URL.
3. **Group** — **`nestquest-admins`**. Add each parent's Authentik user.
Membership is the single source of truth for "who is an admin"; the
API refuses a signed-in non-member with 403.
API settings (in `deploy/.env`, section 5):
| Variable | Value |
| --- | --- |
| `NESTQUEST_OIDC_ISSUER` | `https://auth.cubecraftlabs.com/application/o/nestquest/` |
| `NESTQUEST_OIDC_AUDIENCE` | the provider's Client ID |
| `NESTQUEST_OIDC_JWKS_URL` | `https://auth.cubecraftlabs.com/application/o/nestquest/jwks/` |
Check them against Authentik's discovery document (the issuer must match
byte for byte, including the trailing slash):
curl -s https://auth.cubecraftlabs.com/application/o/nestquest/.well-known/openid-configuration \
| python3 -c 'import json,sys; d=json.load(sys.stdin); print(d["issuer"]); print(d["jwks_uri"])'
## 5. Secrets
Nothing below is ever committed; `.gitignore` excludes `.env`,
`**/.env`, `deploy/*key*` and `deploy/litestream_key*`.
**`deploy/.env`** — created on the API box from the committed example:
cp deploy/.env.example deploy/.env
chmod 600 deploy/.env
# edit deploy/.env and fill in:
# NESTQUEST_PANEL_TOKEN = a new random token, e.g.
# python3 -c "import secrets; print(secrets.token_urlsafe(32))"
# (the same value is configured in the HA integration)
# NESTQUEST_OIDC_ISSUER = section 4
# NESTQUEST_OIDC_AUDIENCE = section 4 (Authentik Client ID)
# NESTQUEST_OIDC_JWKS_URL = section 4
# NESTQUEST_BIND_ADDRESS = 10.60.1.14
The API fails fast at startup if any of the four API
variables is unset or empty (`api/config.py`); `dc logs api` names the
missing one. `NESTQUEST_BIND_ADDRESS` is read by compose only.
**Litestream SFTP key `deploy/litestream_key`** — a dedicated keypair that
lives only on the API box. Create it once, before the first `up`:
ssh-keygen -t ed25519 -N "" -C nestquest-litestream -f deploy/litestream_key
sudo chown 999:999 deploy/litestream_key
chmod 600 deploy/litestream_key
Then append `restrict ` followed by the contents of
`deploy/litestream_key.pub` as one line to
`joshua@10.60.1.50:~/.ssh/authorized_keys` on the NAS. Compose mounts the
private key read-only at `/etc/litestream/sftp_key` in the sidecar, which
runs as uid 999 — hence the ownership. The NAS host key is pinned in the
committed `deploy/litestream.yml` (`host-key`, the NAS's ECDSA key), so no
`known_hosts` is involved.
| Secret | Supplied by | Consumed by |
| --- | --- | --- |
| `NESTQUEST_PANEL_TOKEN` | `deploy/.env` → `env_file` | `api` (panel plane); also set in the HA integration |
| `NESTQUEST_OIDC_*` (not secret, but environment-supplied) | `deploy/.env` → `env_file` | `api` (admin plane) |
| SFTP private key | `deploy/litestream_key` → read-only bind mount | `litestream` |
| Cloudflare DNS API token | Traefik deployment on nukapi-03 | Traefik ACME `myresolver` |
## 6. Network separation and firewall
- **Panel plane** (`/api/v1/panel/*`, static service token): reachable
only from the HA box. Traefik's `ipAllowList` on the panel router allows
`10.60.1.80/32` and returns **403** to every other source, before the
request reaches the API. The API additionally requires the panel token
(401 without it).
- **Admin plane** (`/api/v1/admin/*`) and `/health`, `/docs`: public
through Cloudflare and Traefik; admin routes are protected by the API's
Authentik JWT check (401 without a valid token, 403 without
`nestquest-admins` or when a panel token is presented).
- **Local DNS override** so the HA box reaches Traefik directly, keeping
its real source address: in Pi-hole (`10.60.1.101`) local DNS records,
add `nestquest.cubecraftlabs.com` → `10.60.1.33`. The HA box
must use Pi-hole as its resolver. Check from the HA box:
nslookup nestquest.cubecraftlabs.com 10.60.1.101 # Address: 10.60.1.33
- **Direct `:8080` access is firewalled.** The API box publishes `8080`
on `10.60.1.14`, which alone would let any LAN host bypass Traefik and
the panel `ipAllowList`. This is enforced on the API box in Docker's
`DOCKER-USER` chain (which Docker evaluates before its own forwarding
rules for published ports), in this order:
-A DOCKER-USER -s 10.60.1.33/32 -p tcp --dport 8080 -j RETURN # Traefik on nukapi-03
-A DOCKER-USER -s 10.60.1.14/32 -p tcp --dport 8080 -j RETURN # the API box itself
-A DOCKER-USER -p tcp --dport 8080 -j DROP # every other source
The rules are persisted by the systemd oneshot unit
**`nestquest-api-firewall.service`**, committed as
`deploy/systemd/nestquest-api-firewall.service` (ordered after
`docker.service`, it idempotently applies the three rules at every
boot). Install and enable it on the API box, from the repository
checkout, **before** exposing the API:
sudo cp deploy/systemd/nestquest-api-firewall.service /etc/systemd/system/ \
&& sudo systemctl daemon-reload \
&& sudo systemctl enable --now nestquest-api-firewall.service
The HA box and every other client must use the Traefik hostname,
never `:8080` directly. Check it:
sudo systemctl is-enabled nestquest-api-firewall.service # enabled
sudo systemctl is-active nestquest-api-firewall.service # active
sudo iptables -S DOCKER-USER # the three rules above, in that order
# on nukapi-03 (10.60.1.33) — allowed:
curl -s -o /dev/null -w '%{http_code}\n' http://10.60.1.14:8080/health # 200
# on any other LAN host, e.g. 10.60.1.101 — dropped:
curl -s -m 5 -o /dev/null -w '%{http_code}\n' http://10.60.1.14:8080/health # 000, timeout
# public path still works:
curl -s -o /dev/null -w '%{http_code}\n' https://nestquest.cubecraftlabs.com/health # 200
On a rebuilt API box, install the unit with the commands above before
exposing the API. The committed file is the authoritative copy; the
deployed unit must match it:
diff deploy/systemd/nestquest-api-firewall.service \
/etc/systemd/system/nestquest-api-firewall.service # no output
## 7. Backup (Litestream to the Synology NAS)
The `litestream` sidecar (config `deploy/litestream.yml`, mounted at
`/etc/litestream.yml`) replicates `/var/lib/nestquest/nestquest.db` to:
- SFTP `joshua@10.60.1.50`, path `/homes/Joshua/nestquest-backups` (the
NAS's SFTP view is chrooted to the shared folders; over SSH the same
directory is `/volume1/homes/Joshua/nestquest-backups`).
- Every committed change is shipped as an LTX file within about a second.
Litestream 0.5.17 compacts them into coarser levels on its built-in
schedule (level 1 every 30s, level 2 every 5m, level 3 every 1h).
- **Snapshots**: `snapshot.interval: 24h` (a full snapshot daily);
**retention**: `snapshot.retention: 168h` — a week of snapshots plus the
changes between them, i.e. a 7-day point-in-time restore window.
The sidecar runs as uid 999 with a read-only root filesystem,
`no-new-privileges` and all capabilities dropped. It needs the database
volume read-write (it keeps its `_litestream_*` tables and writes the
WAL); its local state lives beside the database in
`/var/lib/nestquest/.nestquest.db-litestream/`.
Check it:
dc ps litestream # Up
dc logs --tail 50 litestream # no "error" lines
dc exec litestream litestream status # status "ok", local txid advancing
dc exec litestream litestream ltx /var/lib/nestquest/nestquest.db | tail # newest LTX files and timestamps
A change made in the app should appear as a new level-0 LTX file within
seconds. On the NAS, the files are under
`/volume1/homes/Joshua/nestquest-backups/ltx/`.
## 8. Restore
Restores the database from the SFTP replica into the `nestquest-db`
volume at `NESTQUEST_DB_PATH` (`/var/lib/nestquest/nestquest.db`). The
same procedure serves both a damaged database on the existing box and a
rebuilt box. The restore runs in one-off `litestream` containers so it
has the same user (999), volume, config and SFTP key as the sidecar.
**0. Fresh box only.** Complete sections 1 and 5, then start **only the
API** once so the volume exists and is owned by uid 999:
dc up -d api
Do not start `litestream` yet — it would begin replicating the new,
empty database the API just created. The swap in step 4 moves that empty
file aside, and step 5 creates and starts the sidecar with
`dc up -d litestream` once the API is healthy.
**1. Stop writers** — both the API and the sidecar (Litestream must not
replicate while the file is being replaced):
dc stop api litestream
**2. Restore beside the live database** (never over it), with a full
integrity check:
dc run --rm --no-deps litestream restore \
-config /etc/litestream.yml \
-integrity-check full \
-o /var/lib/nestquest/nestquest.db.restored \
/var/lib/nestquest/nestquest.db
Expect `post-restore integrity check passed` and exit code 0. For a
point in time within the retention window, add
`-timestamp 2026-09-24T12:00:00Z` (UTC). To preview without writing, add
`-dry-run`.
**3. Inspect the restored file** before switching to it — confirm known
children and definitions are present:
dc run --rm --no-deps --entrypoint sqlite3 litestream \
/var/lib/nestquest/nestquest.db.restored \
'PRAGMA integrity_check;' \
'SELECT id, display_name FROM children;' \
'SELECT id, title FROM quest_definitions;'
Expect `ok`, then the household's children and quest definitions.
**4. Swap it into place**, keeping the old files (database, its
`-wal`/`-shm`, and Litestream's local state) in a timestamped directory
on the volume:
dc run --rm --no-deps --entrypoint sh litestream -c '
set -e
cd /var/lib/nestquest
ts=$(date -u +%Y%m%dT%H%M%SZ)
mkdir -p "pre-restore-$ts"
for f in nestquest.db nestquest.db-wal nestquest.db-shm .nestquest.db-litestream; do
if [ -e "$f" ]; then mv "$f" "pre-restore-$ts/"; fi
done
mv nestquest.db.restored nestquest.db
ls -la'
**5. Start the API, then the sidecar:**
dc start api
dc ps api # wait for "healthy"
dc up -d litestream
dc ps litestream # Up
dc logs --tail 20 litestream
Use `dc up -d litestream`, not `dc start`: on a fresh box step 0 never
created the sidecar container, and `start` cannot start a container that
does not exist. `up -d` creates it if needed (or starts the existing one),
so continuous replication is back online on both paths.
The sidecar resumes replication from the restored position (it logs
`detected database behind replica` / `fetched latest L0 file from
replica`, then continues normally).
**6. Verify through the API** — the panel snapshot lists the restored
children. On the API box against `10.60.1.14:8080` directly (other LAN
hosts, including the HA box, are dropped on `:8080` by the section 6
firewall; from the HA box use the Traefik hostname instead):
PANEL_TOKEN=$(grep '^NESTQUEST_PANEL_TOKEN=' deploy/.env | cut -d= -f2-)
curl -s -H "Authorization: Bearer $PANEL_TOKEN" \
http://10.60.1.14:8080/api/v1/panel/snapshot
Expect HTTP 200 and `children` naming the known child(ren). Then run
section 9.
Once satisfied, remove the old copy:
`dc run --rm --no-deps --entrypoint rm litestream -rf /var/lib/nestquest/pre-restore-<ts>`.
This exact sequence (stop, restore with integrity check, inspect, swap,
start, snapshot shows the restored child, replication resumes) was
rehearsed against a local file replica with the committed compose file;
the live proof against the SFTP replica is recorded separately by the
controller.
## 9. Verification checklist
Run after every deploy and every restore. `$PANEL_TOKEN` is the value from
`deploy/.env`; never paste it into shared logs.
| # | Check | Command | Expect |
| --- | --- | --- | --- |
| 1 | Health over TLS | `curl -s -o /dev/null -w '%{http_code}\n' https://nestquest.cubecraftlabs.com/health` | `200` |
| 2 | Valid certificate | `curl -sv https://nestquest.cubecraftlabs.com/health 2>&1 \| grep -E 'issuer\|expire date'` | Let's Encrypt issuer, future expiry (no `-k` needed) |
| 3 | Docs public | `curl -s -o /dev/null -w '%{http_code}\n' https://nestquest.cubecraftlabs.com/docs` | `200` |
| 4 | Panel refused externally | from any host other than `10.60.1.80` (e.g. a phone off Wi-Fi, or a LAN machine): `curl -s -o /dev/null -w '%{http_code}\n' -H "Authorization: Bearer $PANEL_TOKEN" https://nestquest.cubecraftlabs.com/api/v1/panel/snapshot` | `403` (Traefik allowlist) |
| 5 | Panel allowed from HA box | on `10.60.1.80`: same command | `200`, snapshot JSON |
| 6 | Panel needs the token | on `10.60.1.80`, without the header | `401` |
| 7 | Admin JWT accepted | `curl -s -w '\n%{http_code}\n' -H "Authorization: Bearer $ADMIN_JWT" https://nestquest.cubecraftlabs.com/api/v1/admin/ping` with a token of a `nestquest-admins` member | `200`, `{"status":"ok","sub":...}` |
| 8 | Non-admin refused | same, with a token of a user **not** in `nestquest-admins` | `403`, "Admin access requires membership in the nestquest-admins group" |
| 9 | No/invalid token refused | same, without a header or with garbage | `401` |
| 10 | Panel token refused on admin | same, with `$PANEL_TOKEN` | `403` |
| 11 | Backup running | `dc exec litestream litestream status` | status `ok` |
To obtain `$ADMIN_JWT`: sign in to the admin PWA as the user, open the
browser developer tools, and copy the session-storage value
`nestquest.access_token`. Tokens are short-lived and are credentials —
do not store them.
## 10. Admin PWA hosting
The admin PWA (`admin/`, a Vite + React single-page app) is served from
the API box by the compose service **`pwa`**: nginx on container port
`80`, published on `${NESTQUEST_BIND_ADDRESS:-127.0.0.1}:8081` (on the API
box `10.60.1.14:8081`). Traefik sends every path of
`nestquest.cubecraftlabs.com` that is not the API to it.
**Image** — `deploy/pwa/Dockerfile`, built from the **repository root**
(it needs `admin/`); `deploy/pwa/Dockerfile.dockerignore` limits the
context to `admin/` (without `node_modules`, `dist` or `.env` files) and
`deploy/pwa/nginx.conf`.
- Stage 1, `node:22-alpine` (pinned by digest): `npm ci`, then
`npm run build` with the three `VITE_*` build args below. Vite bakes
them into the bundle, so **changing any of them needs a rebuild**. The
build fails if any is empty.
- Stage 2, `nginx:alpine` pinned to
`sha256:1ed1b0e1d7652937d6cbdaf4018c7b6fc009a7dd6c3047351e2eddda745de43f`
(nginx 1.31.6): the built `dist/` in `/usr/share/nginx/html` and
`deploy/pwa/nginx.conf` as `conf.d/default.conf`.
**Build args** — compose reads them from `deploy/.env` (documented in
`deploy/.env.example`). They are **public values, not secrets**: the
Authentik application is a PKCE client with no client secret.
| `deploy/.env` variable | Build arg | Production value |
| --- | --- | --- |
| `NESTQUEST_AUTHENTIK_CLIENT_ID` | `VITE_AUTHENTIK_CLIENT_ID` | client id of the Authentik NestQuest application (Authentik admin → Applications → Providers) |
| `NESTQUEST_AUTHENTIK_ISSUER` | `VITE_AUTHENTIK_ISSUER` | `https://auth.cubecraftlabs.com/application/o/nestquest/` |
| `NESTQUEST_API_BASE_URL` | `VITE_API_BASE_URL` | `https://nestquest.cubecraftlabs.com` |
The Authentik application's redirect URI must include
`https://nestquest.cubecraftlabs.com/auth/callback`.
**nginx** (`deploy/pwa/nginx.conf`):
- `allow 10.60.1.33; allow 10.60.1.14; deny all;` — only Traefik on
nukapi-03 and the API box itself get content; any other source gets
`403`, even with the port reachable on the LAN. Checks run on the API
box must use its LAN address (`10.60.1.14:8081`), not `127.0.0.1`.
- SPA fallback: unknown paths (e.g. `/auth/callback`) serve `index.html`;
`/assets/*` (content-hashed, cached for a year) returns a real `404`
when missing.
- `/sw.js` and `/manifest.webmanifest` (`application/manifest+json`) are
served from the site root with `Cache-Control: no-cache`, as is
`index.html`, so a new release's service worker is picked up.
Build and start (the `api` and `litestream` services are unaffected):
dc build pwa
dc up -d pwa
dc ps pwa # "Up"
**Traefik router** — in `config/dynamic/nestquest-ccl.yml` on nukapi-03
(reference shape in section 3):
| Router | Rule | Priority | Service |
| --- | --- | --- | --- |
| `nestquest-app` | ``Host(`nestquest.cubecraftlabs.com`) && !PathPrefix(`/api`) && !Path(`/health`) && !Path(`/openapi.json`) && !PathPrefix(`/docs`)`` | `50` — below `nestquest-admin` (100) and `nestquest-panel` (200) | `nestquest-pwa` → `http://10.60.1.14:8081` |
Same `websecure` entrypoint and `myresolver` certificate as the API
routers; no Traefik middleware (nginx enforces the source allow-list).
The file provider reloads the dynamic file on save.
**Verification** (in addition to section 9):
| # | Check | Command | Expect |
| --- | --- | --- | --- |
| 1 | PWA root over TLS | `curl -s -w '\n%{http_code}\n' https://nestquest.cubecraftlabs.com/ \| tail -3` | `200`, the `NestQuest Admin` shell |
| 2 | Client route fallback | `curl -s -o /dev/null -w '%{http_code} %{content_type}\n' https://nestquest.cubecraftlabs.com/auth/callback` | `200 text/html` |
| 3 | Manifest and worker | `curl -s -o /dev/null -w '%{http_code} %{content_type}\n' https://nestquest.cubecraftlabs.com/manifest.webmanifest` and the same for `/sw.js` | `200 application/manifest+json`, `200 application/javascript` |
| 4 | API still routed to the API | `curl -s -o /dev/null -w '%{http_code}\n' https://nestquest.cubecraftlabs.com/health` | `200` (`{"status":"ok"}`, not HTML) |
| 5 | Panel refused externally | section 9 check 4 | `403` |
| 6 | Non-Traefik source refused | from a LAN machine other than `10.60.1.33`/`10.60.1.14`: `curl -s -o /dev/null -w '%{http_code}\n' http://10.60.1.14:8081/` | `403` (nginx `deny all`) |
| 7 | Proxy source allowed | on `10.60.1.14`: `curl -s -o /dev/null -w '%{http_code}\n' http://10.60.1.14:8081/` | `200` |
+1 -20
View File
@@ -42,25 +42,6 @@ services:
retries: 5
start_period: 15s
# Admin PWA (Feature 24): the Vite bundle from admin/ served by nginx
# (deploy/pwa/). Traefik routes every non-API path of the public host
# here on port 8081; nginx itself only accepts the proxy (10.60.1.33)
# and this host (10.60.1.14). The three build args are PUBLIC values
# (the OIDC client is a PKCE client, no secret) read from deploy/.env
# by compose interpolation; the image build fails if any is empty.
pwa:
build:
context: ..
dockerfile: deploy/pwa/Dockerfile
args:
VITE_AUTHENTIK_CLIENT_ID: ${NESTQUEST_AUTHENTIK_CLIENT_ID:-}
VITE_AUTHENTIK_ISSUER: ${NESTQUEST_AUTHENTIK_ISSUER:-}
VITE_API_BASE_URL: ${NESTQUEST_API_BASE_URL:-}
image: nestquest-pwa:latest
restart: unless-stopped
ports:
- "${NESTQUEST_BIND_ADDRESS:-127.0.0.1}:8081:80"
# Continuous backup: streams the SQLite database on the shared named
# volume to the Synology NAS over SFTP (config: deploy/litestream.yml).
# The SFTP private key deploy/litestream_key is git-ignored and must be
@@ -75,7 +56,7 @@ services:
# (uid/gid 999, owner of the volume) with a read-only root filesystem,
# no privilege escalation and no Linux capabilities.
litestream:
image: litestream/litestream:0.5.17@sha256:4b02b9859a6b6b4087d8b8944e15f7e984bd7957cba322bbeee38b0e27b9656a
image: litestream/litestream:0.5.17
restart: unless-stopped
command: ["replicate", "-config", "/etc/litestream.yml"]
user: "999:999"
-29
View File
@@ -1,29 +0,0 @@
# Admin PWA image (Feature 24): builds the Vite bundle from admin/ and
# serves it with nginx. Build context is the REPOSITORY ROOT (it needs
# admin/); deploy/pwa/Dockerfile.dockerignore limits that context to
# admin/ and this directory's nginx.conf.
#
# docker compose -f deploy/compose.yaml build pwa
#
# The VITE_* values are baked into the bundle at build time. None is a
# secret (the OIDC client is a public PKCE client), and all three are
# required: the build fails fast rather than relying on app defaults.
FROM node:22-alpine@sha256:0a7108bf6c7bf5de370ffb1a3ed6be93d405b43ff159f681a8d18c0e2bc2e402 AS build
ARG VITE_AUTHENTIK_CLIENT_ID
ARG VITE_AUTHENTIK_ISSUER
ARG VITE_API_BASE_URL
WORKDIR /src/admin
COPY admin/package.json admin/package-lock.json ./
RUN npm ci
COPY admin/ ./
RUN test -n "$VITE_AUTHENTIK_CLIENT_ID" && test -n "$VITE_AUTHENTIK_ISSUER" && test -n "$VITE_API_BASE_URL" \
|| { echo "VITE_AUTHENTIK_CLIENT_ID, VITE_AUTHENTIK_ISSUER and VITE_API_BASE_URL build args are required" >&2; exit 1; }
RUN npm run build
FROM nginx:alpine@sha256:1ed1b0e1d7652937d6cbdaf4018c7b6fc009a7dd6c3047351e2eddda745de43f
COPY deploy/pwa/nginx.conf /etc/nginx/conf.d/default.conf
COPY --from=build /src/admin/dist/ /usr/share/nginx/html/
-11
View File
@@ -1,11 +0,0 @@
# Build-context filter for deploy/pwa/Dockerfile (BuildKit reads
# <Dockerfile>.dockerignore in place of the root .dockerignore, which
# excludes deploy/). Only admin/ sources and nginx.conf enter the
# build; local dependencies, build output and .env files never do.
*
!admin/
!deploy/pwa/nginx.conf
admin/node_modules
admin/dist
**/.env
**/.env.*
-53
View File
@@ -1,53 +0,0 @@
# nginx server for the NestQuest admin PWA (Feature 24). Replaces the
# image's /etc/nginx/conf.d/default.conf. The shared Traefik on
# nukapi-03 (10.60.1.33) terminates TLS for nestquest.cubecraftlabs.com
# and forwards every path that is not the API (/api, /health,
# /openapi.json, /docs) here; see deploy/RUNBOOK.md section 10.
server {
listen 80;
server_name _;
# Only the reverse proxy and the API box itself may connect; every
# other source is refused (403) even if the port is reachable.
allow 10.60.1.33;
allow 10.60.1.14;
deny all;
root /usr/share/nginx/html;
index index.html;
server_tokens off;
# Content-hashed bundle files: cache forever, and a missing asset is
# a real 404 rather than the SPA shell.
location /assets/ {
try_files $uri =404;
add_header Cache-Control "public, max-age=31536000, immutable";
}
# The service worker must be revalidated on every load so a new
# release's worker (fresh revision, see admin/vite-sw-plugin.ts) is
# picked up.
location = /sw.js {
try_files $uri =404;
add_header Cache-Control "no-cache";
}
# nginx's stock mime.types has no .webmanifest entry.
location = /manifest.webmanifest {
types { }
default_type application/manifest+json;
try_files $uri =404;
add_header Cache-Control "no-cache";
}
# SPA fallback: client routes such as /auth/callback resolve to the
# app shell, which is never cached so it always names the current
# hashed assets.
location / {
try_files $uri $uri/ /index.html;
}
location = /index.html {
add_header Cache-Control "no-cache";
}
}
@@ -1,12 +0,0 @@
[Unit]
Description=Restrict NestQuest API port 8080 to Traefik (DOCKER-USER)
After=docker.service
Requires=docker.service
[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=/bin/sh -c "iptables -C DOCKER-USER -s 10.60.1.33/32 -p tcp --dport 8080 -j RETURN 2>/dev/null || iptables -I DOCKER-USER 1 -s 10.60.1.33 -p tcp --dport 8080 -j RETURN; iptables -C DOCKER-USER -s 10.60.1.14/32 -p tcp --dport 8080 -j RETURN 2>/dev/null || iptables -I DOCKER-USER 2 -s 10.60.1.14 -p tcp --dport 8080 -j RETURN; iptables -C DOCKER-USER -p tcp --dport 8080 -j DROP 2>/dev/null || iptables -A DOCKER-USER -p tcp --dport 8080 -j DROP"
[Install]
WantedBy=multi-user.target
+1 -1
View File
@@ -1,6 +1,6 @@
[project]
name = "nestquest"
version = "0.7.0"
version = "0.6.0"
description = "NestQuest Home Assistant Integration"
readme = "README.md"
requires-python = ">=3.11"
Generated
+1 -1
View File
@@ -373,7 +373,7 @@ wheels = [
[[package]]
name = "nestquest"
version = "0.7.0"
version = "0.6.0"
source = { virtual = "." }
[package.dev-dependencies]