Add SearXNG recipe-search sidecar to the compose stack
deploy/searxng/settings.yml enables the json format, turns off the public limiter and bot detection (internal use only) and keeps only general web search engines; the secret key comes from SEARXNG_SECRET in the git-ignored deploy/secrets/searxng.env. deploy/compose.yaml adds the searxng service (digest-pinned image, mealquest network only, no published port, healthcheck, read-only root) and sets MEALQUEST_SEARXNG_URL=http://searxng:8080 for mealquest. deploy/compose.dev.yaml publishes it on 127.0.0.1:8888 for development. RUNBOOK and .env.example document the new secret and checks. Task 641f468b-68e9-42e2-b640-3199984cf75c. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
+7
-3
@@ -10,6 +10,11 @@
|
|||||||
# printf '%s\n' '<Authentik client secret>' > deploy/secrets/oidc_client_secret
|
# printf '%s\n' '<Authentik client secret>' > deploy/secrets/oidc_client_secret
|
||||||
# sudo chown 10001:10001 deploy/secrets/* && sudo chmod 600 deploy/secrets/*
|
# sudo chown 10001:10001 deploy/secrets/* && sudo chmod 600 deploy/secrets/*
|
||||||
#
|
#
|
||||||
|
# SearXNG's secret key is the exception: compose reads it itself, so it
|
||||||
|
# stays owned by you (mode 0600). Create it after the chown above:
|
||||||
|
#
|
||||||
|
# (umask 077; printf 'SEARXNG_SECRET=%s\n' "$(openssl rand -hex 32)" > deploy/secrets/searxng.env)
|
||||||
|
#
|
||||||
# deploy/compose.yaml mounts them at /run/secrets/<name> and points the
|
# deploy/compose.yaml mounts them at /run/secrets/<name> and points the
|
||||||
# *_FILE variables at them.
|
# *_FILE variables at them.
|
||||||
|
|
||||||
@@ -62,6 +67,8 @@ MEALQUEST_SESSION_MAX=4320h
|
|||||||
# Base64 of 32 random bytes; encrypts secrets at rest (AES-256-GCM).
|
# Base64 of 32 random bytes; encrypts secrets at rest (AES-256-GCM).
|
||||||
# MEALQUEST_OIDC_CLIENT_SECRET_FILE=/run/secrets/oidc_client_secret
|
# MEALQUEST_OIDC_CLIENT_SECRET_FILE=/run/secrets/oidc_client_secret
|
||||||
# The Authentik provider's client secret.
|
# The Authentik provider's client secret.
|
||||||
|
# MEALQUEST_SEARXNG_URL=http://searxng:8080
|
||||||
|
# The SearXNG recipe-search sidecar on the compose network.
|
||||||
|
|
||||||
# --- Planned; not read by the service yet -------------------------------------
|
# --- Planned; not read by the service yet -------------------------------------
|
||||||
#
|
#
|
||||||
@@ -75,6 +82,3 @@ MEALQUEST_SESSION_MAX=4320h
|
|||||||
# Google Calendar OAuth web client (Feature 16).
|
# Google Calendar OAuth web client (Feature 16).
|
||||||
# MEALQUEST_GOOGLE_CLIENT_ID=<client id>.apps.googleusercontent.com
|
# MEALQUEST_GOOGLE_CLIENT_ID=<client id>.apps.googleusercontent.com
|
||||||
# MEALQUEST_GOOGLE_CLIENT_SECRET_FILE=/run/secrets/google_client_secret
|
# MEALQUEST_GOOGLE_CLIENT_SECRET_FILE=/run/secrets/google_client_secret
|
||||||
#
|
|
||||||
# SearXNG recipe search sidecar on the compose network (Feature 07).
|
|
||||||
# MEALQUEST_SEARXNG_URL=http://searxng:8080
|
|
||||||
|
|||||||
+70
-5
@@ -44,7 +44,8 @@ Elsewhere:
|
|||||||
|
|
||||||
The compose stack is `deploy/compose.yaml`: the `mealquest` service (built
|
The compose stack is `deploy/compose.yaml`: the `mealquest` service (built
|
||||||
from the repository `Dockerfile`, runs as uid/gid 10001, listens on 8080 in
|
from the repository `Dockerfile`, runs as uid/gid 10001, listens on 8080 in
|
||||||
the container) and the `litestream` service. Data lives in the named volume
|
the container), the `litestream` service and the `searxng` recipe-search
|
||||||
|
sidecar (see "Recipe search (SearXNG)"). Data lives in the named volume
|
||||||
`mealquest-data`.
|
`mealquest-data`.
|
||||||
|
|
||||||
### 1. Choose and check the port
|
### 1. Choose and check the port
|
||||||
@@ -146,15 +147,20 @@ service refuses to start.
|
|||||||
| `secret_key` | base64 of 32 random bytes; encrypts stored secrets (AES-256-GCM) | `/run/secrets/secret_key` |
|
| `secret_key` | base64 of 32 random bytes; encrypts stored secrets (AES-256-GCM) | `/run/secrets/secret_key` |
|
||||||
| `oidc_client_secret` | Client Secret of the Authentik provider `mealquest` | `/run/secrets/oidc_client_secret` |
|
| `oidc_client_secret` | Client Secret of the Authentik provider `mealquest` | `/run/secrets/oidc_client_secret` |
|
||||||
| `litestream_key` | dedicated SSH private key for the NAS (see "Backups") | `/etc/litestream/sftp_key` |
|
| `litestream_key` | dedicated SSH private key for the NAS (see "Backups") | `/etc/litestream/sftp_key` |
|
||||||
|
| `searxng.env` | `SEARXNG_SECRET=<64 hex characters>`; SearXNG's secret key | not mounted; compose passes it as an environment variable |
|
||||||
|
|
||||||
|
`searxng.env` is the one exception to the owner rule: compose itself reads
|
||||||
|
it (as `env_file`), so it is owned by `overseer`, mode 0600.
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
mkdir -p deploy/secrets && chmod 700 deploy/secrets
|
mkdir -p deploy/secrets && chmod 700 deploy/secrets
|
||||||
head -c 32 /dev/urandom | base64 > deploy/secrets/secret_key
|
head -c 32 /dev/urandom | base64 > deploy/secrets/secret_key
|
||||||
printf '%s\n' '<Authentik client secret>' > deploy/secrets/oidc_client_secret
|
printf '%s\n' '<Authentik client secret>' > deploy/secrets/oidc_client_secret
|
||||||
# litestream_key: see "Backups" > "Database — Litestream"
|
# litestream_key: see "Backups" > "Database — Litestream"
|
||||||
sudo chown 10001:10001 deploy/secrets/*
|
sudo chown 10001:10001 deploy/secrets/secret_key deploy/secrets/oidc_client_secret deploy/secrets/litestream_key
|
||||||
sudo chmod 600 deploy/secrets/*
|
sudo chmod 600 deploy/secrets/*
|
||||||
sudo ls -la deploy/secrets # three files, mealquest mealquest, -rw-------
|
(umask 077; printf 'SEARXNG_SECRET=%s\n' "$(openssl rand -hex 32)" > deploy/secrets/searxng.env)
|
||||||
|
sudo ls -la deploy/secrets # three files mealquest mealquest, searxng.env overseer overseer; all -rw-------
|
||||||
```
|
```
|
||||||
|
|
||||||
The `mealquest` host user (uid/gid 10001) is created in "Media — nightly
|
The `mealquest` host user (uid/gid 10001) is created in "Media — nightly
|
||||||
@@ -169,7 +175,7 @@ directory (see "Media — nightly rsync"), then build and start:
|
|||||||
```sh
|
```sh
|
||||||
docker compose -f deploy/compose.yaml build
|
docker compose -f deploy/compose.yaml build
|
||||||
sudo systemctl enable --now mealquest # starts mealquest-firewall first
|
sudo systemctl enable --now mealquest # starts mealquest-firewall first
|
||||||
docker compose -f deploy/compose.yaml ps # mealquest healthy, litestream up
|
docker compose -f deploy/compose.yaml ps # mealquest healthy, litestream up, searxng healthy
|
||||||
curl -fsS http://10.60.1.14:8090/health
|
curl -fsS http://10.60.1.14:8090/health
|
||||||
curl -s -o /dev/null -w "%{http_code}\n" https://mealquest.cubecraftlabs.com/health # 200
|
curl -s -o /dev/null -w "%{http_code}\n" https://mealquest.cubecraftlabs.com/health # 200
|
||||||
```
|
```
|
||||||
@@ -192,6 +198,12 @@ git clean -n deploy # dry run: must not list .env or secrets/ (both git-ign
|
|||||||
git clean -f deploy
|
git clean -f deploy
|
||||||
```
|
```
|
||||||
|
|
||||||
|
On 2026-10-06 `deploy/compose.yaml` was replaced and
|
||||||
|
`deploy/compose.dev.yaml` and `deploy/searxng/settings.yml` were copied in
|
||||||
|
from the Feature 10 branch (task 641f468b) to start the SearXNG sidecar;
|
||||||
|
they too must go before that checkout (`git clean -n -d deploy` lists
|
||||||
|
them).
|
||||||
|
|
||||||
Then follow "Routine deploy"; the checkout brings the tracked copies back.
|
Then follow "Routine deploy"; the checkout brings the tracked copies back.
|
||||||
|
|
||||||
## Authentik
|
## Authentik
|
||||||
@@ -399,6 +411,48 @@ mealquest`, never with `docker compose up` or `down`. Per-service compose
|
|||||||
commands (`stop mealquest`, `up -d --force-recreate litestream`) are fine
|
commands (`stop mealquest`, `up -d --force-recreate litestream`) are fine
|
||||||
while the unit is active, because the firewall is already in place.
|
while the unit is active, because the firewall is already in place.
|
||||||
|
|
||||||
|
## Recipe search (SearXNG)
|
||||||
|
|
||||||
|
The `searxng` service in `deploy/compose.yaml` is a private SearXNG
|
||||||
|
instance that the `mealquest` service queries for recipes
|
||||||
|
(`MEALQUEST_SEARXNG_URL=http://searxng:8080`, set by `deploy/compose.yaml`).
|
||||||
|
|
||||||
|
| Item | Value |
|
||||||
|
| --- | --- |
|
||||||
|
| Image | `searxng/searxng:2026.10.4-d48c4b555`, pinned by digest `sha256:76b0bf285aca014c7191fc4d9234c4bfb358624ac33d8883833d496c059ec072` |
|
||||||
|
| Settings | `deploy/searxng/settings.yml`, mounted read-only at `/etc/searxng/settings.yml` |
|
||||||
|
| Secret | `deploy/secrets/searxng.env` (`SEARXNG_SECRET`), see "Secrets" |
|
||||||
|
| Network | `mealquest` only; **no published port** |
|
||||||
|
| Health | `wget http://127.0.0.1:8080/healthz` inside the container |
|
||||||
|
|
||||||
|
The settings enable the `json` output format, turn off the public limiter
|
||||||
|
and bot detection (the only client is the `mealquest` service), and keep
|
||||||
|
only the general web search engines DuckDuckGo, Brave, Google, Bing,
|
||||||
|
Mojeek and Startpage. Engines that block the host (CAPTCHA, "too many
|
||||||
|
requests") are suspended by SearXNG for a while and listed under
|
||||||
|
`unresponsive_engines` in the JSON; the others still answer.
|
||||||
|
|
||||||
|
Check from inside the app container (the only place it is reachable):
|
||||||
|
|
||||||
|
```sh
|
||||||
|
docker compose -f deploy/compose.yaml exec mealquest \
|
||||||
|
wget -qO- 'http://searxng:8080/search?q=chicken+tikka+masala+recipe&format=json'
|
||||||
|
docker compose -f deploy/compose.yaml port searxng 8080 # prints ":0": nothing published
|
||||||
|
```
|
||||||
|
|
||||||
|
On 2026-10-06 that query returned 17 results (Bing and Mojeek answered;
|
||||||
|
Brave, DuckDuckGo, Google and Startpage were rate-limited or served a
|
||||||
|
CAPTCHA). From 10.60.1.85 nothing on 10.60.1.14 answered as SearXNG:
|
||||||
|
8888 refused the connection and 8080 is NestQuest's API.
|
||||||
|
|
||||||
|
For development, `deploy/compose.dev.yaml` publishes SearXNG on the
|
||||||
|
dev machine's loopback only (`127.0.0.1:8888`). It is never used on
|
||||||
|
office-srvr.
|
||||||
|
|
||||||
|
To move to a newer image, pull the new tag, take its digest from
|
||||||
|
`docker image inspect --format '{{json .RepoDigests}}' searxng/searxng:<tag>`,
|
||||||
|
update both in `deploy/compose.yaml`, then deploy as usual.
|
||||||
|
|
||||||
## Routine deploy and rollback
|
## Routine deploy and rollback
|
||||||
|
|
||||||
Production runs `main` (a release); only `overseer` merges `dev` → `main`.
|
Production runs `main` (a release); only `overseer` merges `dev` → `main`.
|
||||||
@@ -413,7 +467,7 @@ git rev-parse --short HEAD # the SHA being replaced: note it for rollba
|
|||||||
git checkout --detach origin/main # or a release tag
|
git checkout --detach origin/main # or a release tag
|
||||||
docker compose -f deploy/compose.yaml build
|
docker compose -f deploy/compose.yaml build
|
||||||
sudo systemctl restart mealquest # applies the firewall, then down + up -d
|
sudo systemctl restart mealquest # applies the firewall, then down + up -d
|
||||||
docker compose -f deploy/compose.yaml ps # mealquest healthy, litestream up
|
docker compose -f deploy/compose.yaml ps # mealquest healthy, litestream up, searxng healthy
|
||||||
curl -fsS http://10.60.1.14:8090/health
|
curl -fsS http://10.60.1.14:8090/health
|
||||||
curl -s -o /dev/null -w "%{http_code}\n" https://mealquest.cubecraftlabs.com/health # 200
|
curl -s -o /dev/null -w "%{http_code}\n" https://mealquest.cubecraftlabs.com/health # 200
|
||||||
git rev-parse HEAD # the deployed SHA
|
git rev-parse HEAD # the deployed SHA
|
||||||
@@ -868,6 +922,17 @@ pushes. Rotate only when the private key may have leaked, write both
|
|||||||
halves, recreate the service with `--force-recreate mealquest`, and tell
|
halves, recreate the service with `--force-recreate mealquest`, and tell
|
||||||
the household to re-enable notifications on each device.
|
the household to re-enable notifications on each device.
|
||||||
|
|
||||||
|
### SearXNG secret (`deploy/secrets/searxng.env`)
|
||||||
|
|
||||||
|
Only signs SearXNG's own cookies and image-proxy links; nothing stored
|
||||||
|
depends on it. Replace it and recreate the service:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
(umask 077; printf 'SEARXNG_SECRET=%s\n' "$(openssl rand -hex 32)" > deploy/secrets/searxng.env)
|
||||||
|
docker compose -f deploy/compose.yaml up -d --force-recreate searxng
|
||||||
|
docker compose -f deploy/compose.yaml ps searxng # healthy
|
||||||
|
```
|
||||||
|
|
||||||
### Litestream / NAS key (`deploy/secrets/litestream_key`)
|
### Litestream / NAS key (`deploy/secrets/litestream_key`)
|
||||||
|
|
||||||
Generate a new keypair as in "Database — Litestream", add the new public
|
Generate a new keypair as in "Database — Litestream", add the new public
|
||||||
|
|||||||
@@ -0,0 +1,12 @@
|
|||||||
|
# Development override: publishes SearXNG on the dev machine's loopback so
|
||||||
|
# it can be queried directly, e.g.
|
||||||
|
#
|
||||||
|
# docker compose -f deploy/compose.yaml -f deploy/compose.dev.yaml up -d searxng
|
||||||
|
# curl 'http://127.0.0.1:8888/search?q=chicken+tikka+masala+recipe&format=json'
|
||||||
|
#
|
||||||
|
# Loopback only, never a LAN address. Never used in production: the
|
||||||
|
# systemd unit starts deploy/compose.yaml alone.
|
||||||
|
services:
|
||||||
|
searxng:
|
||||||
|
ports:
|
||||||
|
- "127.0.0.1:8888:8080"
|
||||||
+39
-1
@@ -11,7 +11,8 @@
|
|||||||
# Configuration comes from deploy/.env (copy deploy/.env.example) and secrets
|
# Configuration comes from deploy/.env (copy deploy/.env.example) and secrets
|
||||||
# from deploy/secrets/; neither is committed. Compose bind-mounts file secrets
|
# from deploy/secrets/; neither is committed. Compose bind-mounts file secrets
|
||||||
# with their host owner and mode, so each file in deploy/secrets/ must be
|
# with their host owner and mode, so each file in deploy/secrets/ must be
|
||||||
# owned by the container user 10001:10001 with mode 0600.
|
# owned by the container user 10001:10001 with mode 0600 (except
|
||||||
|
# searxng.env, read by compose itself; see the searxng service).
|
||||||
name: mealquest
|
name: mealquest
|
||||||
|
|
||||||
services:
|
services:
|
||||||
@@ -31,6 +32,7 @@ services:
|
|||||||
MEALQUEST_DATA_DIR: /data
|
MEALQUEST_DATA_DIR: /data
|
||||||
MEALQUEST_SECRET_KEY_FILE: /run/secrets/secret_key
|
MEALQUEST_SECRET_KEY_FILE: /run/secrets/secret_key
|
||||||
MEALQUEST_OIDC_CLIENT_SECRET_FILE: /run/secrets/oidc_client_secret
|
MEALQUEST_OIDC_CLIENT_SECRET_FILE: /run/secrets/oidc_client_secret
|
||||||
|
MEALQUEST_SEARXNG_URL: http://searxng:8080
|
||||||
secrets:
|
secrets:
|
||||||
- secret_key
|
- secret_key
|
||||||
- oidc_client_secret
|
- oidc_client_secret
|
||||||
@@ -85,6 +87,42 @@ services:
|
|||||||
cap_drop:
|
cap_drop:
|
||||||
- ALL
|
- ALL
|
||||||
|
|
||||||
|
# SearXNG, the recipe-search sidecar (Feature 10; settings:
|
||||||
|
# deploy/searxng/settings.yml). It sits on the mealquest network only and
|
||||||
|
# publishes no port: the mealquest service reaches it at
|
||||||
|
# http://searxng:8080 and nothing outside the compose network can.
|
||||||
|
# deploy/compose.dev.yaml publishes it on 127.0.0.1 for development.
|
||||||
|
#
|
||||||
|
# The secret key comes from deploy/secrets/searxng.env as SEARXNG_SECRET.
|
||||||
|
# Compose itself reads that file (it is not mounted), so unlike the other
|
||||||
|
# secrets it is owned by overseer, mode 0600. It runs as the image's
|
||||||
|
# searxng user (977) with a read-only root filesystem.
|
||||||
|
searxng:
|
||||||
|
image: searxng/searxng:2026.10.4-d48c4b555@sha256:76b0bf285aca014c7191fc4d9234c4bfb358624ac33d8883833d496c059ec072
|
||||||
|
user: "977:977"
|
||||||
|
restart: "no"
|
||||||
|
env_file:
|
||||||
|
- path: ./secrets/searxng.env
|
||||||
|
required: true
|
||||||
|
volumes:
|
||||||
|
- ./searxng/settings.yml:/etc/searxng/settings.yml:ro
|
||||||
|
networks:
|
||||||
|
- mealquest
|
||||||
|
healthcheck:
|
||||||
|
test: ["CMD", "wget", "-q", "-O", "/dev/null", "http://127.0.0.1:8080/healthz"]
|
||||||
|
interval: 30s
|
||||||
|
timeout: 3s
|
||||||
|
start_period: 15s
|
||||||
|
retries: 3
|
||||||
|
read_only: true
|
||||||
|
tmpfs:
|
||||||
|
- /tmp
|
||||||
|
- /var/cache/searxng:uid=977,gid=977
|
||||||
|
security_opt:
|
||||||
|
- no-new-privileges:true
|
||||||
|
cap_drop:
|
||||||
|
- ALL
|
||||||
|
|
||||||
secrets:
|
secrets:
|
||||||
secret_key:
|
secret_key:
|
||||||
file: ./secrets/secret_key
|
file: ./secrets/secret_key
|
||||||
|
|||||||
@@ -0,0 +1,61 @@
|
|||||||
|
# SearXNG settings for the MealQuest recipe-search sidecar (Feature 10).
|
||||||
|
# Mounted read-only at /etc/searxng/settings.yml by deploy/compose.yaml.
|
||||||
|
#
|
||||||
|
# Internal use only: SearXNG is reachable on the compose network
|
||||||
|
# `mealquest` and nowhere else, and its only client is the mealquest
|
||||||
|
# service. That is why the public limiter and bot detection are off.
|
||||||
|
#
|
||||||
|
# The secret key is not in this file. SearXNG reads it from SEARXNG_SECRET,
|
||||||
|
# set from deploy/secrets/searxng.env (never committed). Without it the
|
||||||
|
# default "ultrasecretkey" stays in place and SearXNG refuses to start.
|
||||||
|
#
|
||||||
|
# Settings reference: https://docs.searxng.org/admin/settings/
|
||||||
|
|
||||||
|
use_default_settings:
|
||||||
|
engines:
|
||||||
|
# General web search engines only; every other default engine is
|
||||||
|
# dropped.
|
||||||
|
keep_only:
|
||||||
|
- duckduckgo
|
||||||
|
- brave
|
||||||
|
- google
|
||||||
|
- bing
|
||||||
|
- mojeek
|
||||||
|
- startpage
|
||||||
|
|
||||||
|
general:
|
||||||
|
instance_name: MealQuest search
|
||||||
|
|
||||||
|
server:
|
||||||
|
# Internal use: no public limiter and no bot detection.
|
||||||
|
limiter: false
|
||||||
|
public_instance: false
|
||||||
|
image_proxy: false
|
||||||
|
|
||||||
|
search:
|
||||||
|
safe_search: 0
|
||||||
|
# No autocomplete: the mealquest service never asks for suggestions.
|
||||||
|
autocomplete: ""
|
||||||
|
# json is the format the mealquest service reads; html stays for
|
||||||
|
# debugging through deploy/compose.dev.yaml.
|
||||||
|
formats:
|
||||||
|
- html
|
||||||
|
- json
|
||||||
|
|
||||||
|
# Some general web engines are disabled or inactive by default; enable all
|
||||||
|
# of the kept ones.
|
||||||
|
engines:
|
||||||
|
- name: duckduckgo
|
||||||
|
disabled: false
|
||||||
|
- name: brave
|
||||||
|
disabled: false
|
||||||
|
- name: google
|
||||||
|
disabled: false
|
||||||
|
- name: bing
|
||||||
|
disabled: false
|
||||||
|
- name: mojeek
|
||||||
|
disabled: false
|
||||||
|
inactive: false
|
||||||
|
- name: startpage
|
||||||
|
disabled: false
|
||||||
|
inactive: false
|
||||||
Reference in New Issue
Block a user