Harden recipe fetcher: public-only check, cache copies, rate-limit slots
CI / ci (push) Successful in 1m35s
CI / ci (pull_request) Successful in 1m33s

- IsPublic now refuses every non-globally-reachable IPv4 special-purpose
  range (adds 192.0.0.0/24, TEST-NET-1/2/3, 192.88.99.0/24, benchmarking
  198.18.0.0/15) and admits IPv6 only from global unicast 2000::/3, minus
  its documentation, 6to4, Teredo and IETF protocol-assignment ranges.
  Test fixtures move off TEST-NET-3 to public addresses; Fetch-level tests
  cover the new blocked ranges and the public IPv6 path.
- The cache stores and returns copies of Page.Body, so callers cannot
  alter cached bytes or race with other readers.
- The per-host rate limiter advances only when a request is let through;
  a wait that ends with its context holds no slot.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-05 12:01:12 -04:00
co-authored by Claude Opus 5.5
parent e7dbcf3d2c
commit 177b2dd2e7
3 changed files with 165 additions and 49 deletions
+9 -5
View File
@@ -210,15 +210,19 @@ automatic** (D-021).
(`searxng` or `brave`, per `recipe_search_settings`).
- Fetching is **SSRF-safe** (`recipes.Fetcher`): https:// only, at every
redirect hop. Its dialer resolves the host once and refuses unless every
answer is public — loopback, RFC 1918, link-local, carrier-grade NAT
(100.64/10), multicast, unspecified, reserved, IPv6 unique-local,
link-local and multicast, and IPv4-mapped or IPv4-embedding (NAT64, 6to4,
Teredo) forms are refused — then dials the address it checked, so a
answer is public — every non-globally-reachable IPv4 special-purpose
range (loopback, RFC 1918, link-local, carrier-grade NAT, documentation
TEST-NETs, benchmarking 198.18/15, multicast, reserved and more) is
refused, IPv4-mapped forms included, and IPv6 must be global unicast
(2000::/3) outside documentation, 6to4, Teredo and other special ranges —
then dials the address it checked, so a
rebinding DNS answer cannot swap in a private one. Keep-alives and
environment proxies are off, so every request dials and checks afresh. At
most 3 redirects; a body over 2 MB is an error, never truncated; 10 s for
the whole fetch; a descriptive `User-Agent`; requests to one host at least
1 s apart; 2xx pages cached by URL for an hour (64 pages, oldest evicted).
1 s apart, counted from requests actually sent (an abandoned wait holds no
slot); 2xx pages cached by URL for an hour (64 pages, oldest evicted), and
callers get their own copy of a cached body.
- Recipe data is parsed from schema.org `Recipe` JSON-LD.
- Photos are cached locally (`photos`) and processed with
`golang.org/x/image`.