Reject malformed search bodies and report unranked candidates

SearXNG now requires a results array and Brave requires "type": "search"
with a results array in any web object, so `{}` is ErrSearchResponse
instead of an empty search. Rank no longer drops failed candidates: it
returns the ranked ones with ErrCandidatesUnranked joined with each
unranked candidate's URL and reason, or ErrNoUsableRecipe when none rank.
Adds an HTTPS ranking test through the SSRF-safe Fetcher.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

docs/ARCHITECTURE.md

@@ -208,8 +208,61 @@ automatic** (D-021).
 
 - Dinners are grounded in real recipe pages found through recipe search
   (`searxng` or `brave`, per `recipe_search_settings`).
-- Fetching is **SSRF-safe**.
-- Recipe data is parsed from schema.org `Recipe` JSON-LD.
+- Fetching is **SSRF-safe** (`recipes.Fetcher`): https:// only, at every
+  redirect hop. Its dialer resolves the host once and refuses unless every
+  answer is public — every non-globally-reachable IPv4 special-purpose
+  range (loopback, RFC 1918, link-local, carrier-grade NAT, documentation
+  TEST-NETs, benchmarking 198.18/15, multicast, reserved and more) is
+  refused, IPv4-mapped forms included, and IPv6 must be global unicast
+  (2000::/3) outside documentation, 6to4, Teredo and other special ranges —
+  then dials the address it checked, so a
+  rebinding DNS answer cannot swap in a private one. Keep-alives and
+  environment proxies are off, so every request dials and checks afresh. At
+  most 3 redirects; a body over 2 MB is an error, never truncated; 10 s for
+  the whole fetch; a descriptive `User-Agent`; requests to one host at least
+  1 s apart, counted from requests actually sent (an abandoned wait holds no
+  slot); 2xx pages cached by URL for an hour (64 pages, oldest evicted), and
+  callers get their own copy of a cached body.
+- Recipe data is parsed from schema.org `Recipe` JSON-LD
+  (`recipes.Extract`): a small targeted scan of the page's
+  `application/ld+json` scripts and `og:site_name` (no HTML parser
+  dependency), finding the Recipe node at a script's top level, in a
+  top-level array or in an `@graph`, with `@type` a string or an array of
+  terms or absolute schema.org IRIs (`https://schema.org/Recipe`). An image
+  or publisher given as an `{"@id": ...}` reference resolves to the node
+  with that `@id` anywhere in the page's JSON-LD. The largest image (by ImageObject width × height; the first of equals) is
+  resolved to an absolute URL; instructions — one string of lines, or a
+  list of strings, HowToSteps and HowToSections — are flattened in order;
+  ISO-8601 prep/cook/total times become minutes, total derived as prep +
+  cook when only those are given; the site name is the publisher's, else
+  `og:site_name`, else the host. A page with no Recipe is `ErrNoRecipe`,
+  with two is `ErrManyRecipes`; a shape schema.org does not allow — such as
+  a non-ISO-8601 duration like "20 minutes" or an empty one, or an image
+  `@id` no node has — is `ErrMalformedRecipe`, never guessed at.
+- Recipe search (`recipes.SearchProvider`: `Search(ctx, query, n)` →
+  `[]Candidate{URL, Title, Snippet}`, n 1–20) appends " recipe" to a query
+  without that word. SearXNG is called at
+  `{MEALQUEST_SEARXNG_URL}/search?q=…&format=json&categories=general` —
+  directly, not through the Fetcher, since this configured endpoint may be
+  on the internal network; Brave calls the Brave Web Search API with the
+  key decrypted from `recipe_search_settings.api_key_ct`
+  (`recipes.NewSearchProvider`). Redirects are not followed; 401/403 is
+  `ErrSearchRefused`, 429 `ErrSearchRateLimited`, other non-2xx a
+  `SearchStatusError`, and a body that is not the provider's JSON (SearXNG
+  without a `results` array; Brave without `"type": "search"` or with a
+  `web` object lacking `results` — Brave omits `web` when nothing matched),
+  over 1 MB or with a result lacking an absolute http(s) URL
+  `ErrSearchResponse`.
+- Ranking (`recipes.Rank`) fetches and extracts the top 5 candidates
+  concurrently, rejects any whose recipe lacks ingredients or an image, and
+  orders the rest by the token similarity of the recipe's title to the dish
+  name (`harness.Similarity`, 0–1) plus a quarter for each of ingredients,
+  image, steps and total minutes; ties keep search order. No unranked
+  candidate is dropped silently: when some are ranked, they are returned
+  with `ErrCandidatesUnranked`; when none are, the error is
+  `ErrNoUsableRecipe`. Either is joined with each unranked candidate's URL
+  and reason (fetch failure, no valid recipe, missing ingredients or
+  image), so the caller decides per item what a failure means.
 - Photos are cached locally (`photos`) and processed with
   `golang.org/x/image`.
 

internal/ai/harness/dedupe.go

@@ -53,7 +53,14 @@ func Normalize(name string) string {
 // similarity of their normalized forms is at least SimilarThreshold. A name
 // with no words left after normalizing is similar to nothing.
 func Similar(a, b string) bool {
-	return similarity(tokenSet(a), tokenSet(b)) >= SimilarThreshold
+	return Similarity(a, b) >= SimilarThreshold
+}
+
+// Similarity is the token-set (Jaccard) similarity of a's and b's normalized
+// forms, from 0 (no shared word) to 1 (the same words). A name with no words
+// left after normalizing scores 0 against everything.
+func Similarity(a, b string) float64 {
+	return similarity(tokenSet(a), tokenSet(b))
 }
 
 // tokenSet returns the distinct words of name's normalized form.

internal/ai/harness/dedupe_test.go

@@ -114,6 +114,22 @@ func TestSimilar(t *testing.T) {
 	}
 }
 
+func TestSimilarity(t *testing.T) {
+	for _, tc := range []struct {
+		a, b string
+		want float64
+	}{
+		{"Easy Chicken Curry", "chicken curry", 1},
+		{"Thai Green Curry", "Chicken curry", 0.25},
+		{"Beef Stew", "Chicken curry", 0},
+		{"The Recipe", "chicken curry", 0},
+	} {
+		if got := Similarity(tc.a, tc.b); got != tc.want {
+			t.Errorf("Similarity(%q, %q) = %v, want %v", tc.a, tc.b, got, tc.want)
+		}
+	}
+}
+
 func TestDeduper(t *testing.T) {
 	d := NewDeduper(DedupeSources{
 		Regulars: []string{"Beef Tacos", "Spaghetti Bolognese"},

internal/recipes/extract.go

@@ -0,0 +1,631 @@
+package recipes
+
+import (
+	"encoding/json"
+	"errors"
+	"fmt"
+	"html"
+	"io"
+	"math"
+	"net/url"
+	"strconv"
+	"strings"
+)
+
+var (
+	// ErrNoRecipe is returned for a page with no schema.org Recipe node in
+	// its JSON-LD.
+	ErrNoRecipe = errors.New("recipes: page has no schema.org Recipe")
+	// ErrManyRecipes is returned for a page with more than one Recipe node:
+	// which one the page is about is not knowable.
+	ErrManyRecipes = errors.New("recipes: page has more than one schema.org Recipe")
+	// ErrMalformedRecipe is returned when the page's JSON-LD, or a field of
+	// its Recipe node, is not in a shape schema.org allows.
+	ErrMalformedRecipe = errors.New("recipes: malformed schema.org Recipe")
+)
+
+// Recipe is the schema.org Recipe extracted from a recipe page.
+type Recipe struct {
+	// Title is the recipe's name.
+	Title string
+	// SiteName is the publisher's name, else the page's og:site_name, else
+	// its host without a leading "www.".
+	SiteName string
+	// ImageURL is the largest image, absolute; empty when the recipe has
+	// none.
+	ImageURL    string
+	Ingredients []string
+	// Steps are the instructions in order, HowToSection groups flattened.
+	Steps []string
+	// PrepMinutes, CookMinutes and TotalMinutes are nil when not given.
+	// TotalMinutes is prep plus cook when the page gives both but no total.
+	PrepMinutes  *int
+	CookMinutes  *int
+	TotalMinutes *int
+	// Yield is recipeYield as text; empty when not given.
+	Yield string
+	// JSONLD is the Recipe node as found.
+	JSONLD json.RawMessage
+}
+
+// Extract finds the schema.org Recipe in a page's application/ld+json
+// scripts. Recipe nodes are found at the top level of a script, in a
+// top-level array, or in an @graph. An image or publisher given as an
+// {"@id": ...} reference resolves to the node with that @id found in the
+// same places, in any of the page's scripts. pageURL is the page's own
+// absolute URL; relative image URLs resolve against it.
+func Extract(page []byte, pageURL string) (Recipe, error) {
+	base, err := url.Parse(pageURL)
+	if err != nil || !base.IsAbs() || base.Host == "" {
+		return Recipe{}, fmt.Errorf("recipes: page URL %q is not absolute", pageURL)
+	}
+	doc := scanHTML(string(page))
+	g := graph{ids: map[string][]map[string]any{}}
+	for i, script := range doc.jsonLD {
+		v, err := decodeJSON(script)
+		if err != nil {
+			return Recipe{}, fmt.Errorf("%w: JSON-LD script %d: %v", ErrMalformedRecipe, i+1, err)
+		}
+		g.collect(v)
+	}
+	nodes := g.recipes
+	switch len(nodes) {
+	case 0:
+		return Recipe{}, ErrNoRecipe
+	case 1:
+	default:
+		return Recipe{}, fmt.Errorf("%w: found %d", ErrManyRecipes, len(nodes))
+	}
+	r, err := recipeFrom(nodes[0], g, base, doc.ogSiteName)
+	if err != nil {
+		return Recipe{}, fmt.Errorf("%w: %v", ErrMalformedRecipe, err)
+	}
+	return r, nil
+}
+
+func decodeJSON(s string) (any, error) {
+	dec := json.NewDecoder(strings.NewReader(s))
+	dec.UseNumber()
+	var v any
+	if err := dec.Decode(&v); err != nil {
+		return nil, err
+	}
+	if _, err := dec.Token(); err != io.EOF {
+		return nil, errors.New("trailing data after JSON value")
+	}
+	return v, nil
+}
+
+// graph holds the page's JSON-LD nodes: the Recipe nodes, and every node
+// with an @id by that @id.
+type graph struct {
+	recipes []map[string]any
+	ids     map[string][]map[string]any
+}
+
+// collect gathers the nodes in v: v itself, the elements of an array, and
+// the members of an @graph. A bare {"@id": ...} reference is not a node.
+func (g *graph) collect(v any) {
+	switch n := v.(type) {
+	case []any:
+		for _, e := range n {
+			g.collect(e)
+		}
+	case map[string]any:
+		if hasType(n, "Recipe") {
+			g.recipes = append(g.recipes, n)
+		}
+		if id, ok := n["@id"].(string); ok && len(n) > 1 {
+			g.ids[id] = append(g.ids[id], n)
+		}
+		if members, ok := n["@graph"]; ok {
+			g.collect(members)
+		}
+	}
+}
+
+// resolve returns the node a bare {"@id": ...} reference names, and v
+// itself when v is not a reference. found is false for a reference to no
+// node; a reference to more than one node is an error.
+func (g graph) resolve(v any) (node any, found bool, err error) {
+	m, ok := v.(map[string]any)
+	if !ok || len(m) != 1 {
+		return v, true, nil
+	}
+	id, ok := m["@id"].(string)
+	if !ok {
+		return v, true, nil
+	}
+	switch nodes := g.ids[id]; len(nodes) {
+	case 0:
+		return v, false, nil
+	case 1:
+		return nodes[0], true, nil
+	default:
+		return nil, false, fmt.Errorf("@id %q names %d nodes", id, len(nodes))
+	}
+}
+
+// hasType reports whether node's @type, a string or an array of strings,
+// names the schema.org type typ, as a term or an absolute schema.org IRI.
+func hasType(node map[string]any, typ string) bool {
+	switch t := node["@type"].(type) {
+	case string:
+		return isSchemaType(t, typ)
+	case []any:
+		for _, e := range t {
+			if s, ok := e.(string); ok && isSchemaType(s, typ) {
+				return true
+			}
+		}
+	}
+	return false
+}
+
+func isSchemaType(s, typ string) bool {
+	return s == typ || s == "https://schema.org/"+typ || s == "http://schema.org/"+typ
+}
+
+func recipeFrom(n map[string]any, g graph, base *url.URL, ogSiteName string) (Recipe, error) {
+	var r Recipe
+	var err error
+	name, ok := n["name"].(string)
+	if r.Title = cleanText(name); !ok || r.Title == "" {
+		return Recipe{}, errors.New("name is missing")
+	}
+	if r.SiteName, err = siteName(n["publisher"], g, ogSiteName, base); err != nil {
+		return Recipe{}, err
+	}
+	if r.ImageURL, err = imageURL(n["image"], g, base); err != nil {
+		return Recipe{}, err
+	}
+	if r.Ingredients, err = ingredients(n["recipeIngredient"]); err != nil {
+		return Recipe{}, err
+	}
+	if r.Steps, err = steps(n["recipeInstructions"]); err != nil {
+		return Recipe{}, err
+	}
+	if r.PrepMinutes, err = minutes(n, "prepTime"); err != nil {
+		return Recipe{}, err
+	}
+	if r.CookMinutes, err = minutes(n, "cookTime"); err != nil {
+		return Recipe{}, err
+	}
+	if r.TotalMinutes, err = minutes(n, "totalTime"); err != nil {
+		return Recipe{}, err
+	}
+	if r.TotalMinutes == nil && r.PrepMinutes != nil && r.CookMinutes != nil {
+		total := *r.PrepMinutes + *r.CookMinutes
+		r.TotalMinutes = &total
+	}
+	if r.Yield, err = yield(n["recipeYield"]); err != nil {
+		return Recipe{}, err
+	}
+	if r.JSONLD, err = json.Marshal(n); err != nil {
+		return Recipe{}, err
+	}
+	return r, nil
+}
+
+// cleanText decodes HTML entities, which recipe plugins often leave in
+// JSON-LD text, and trims surrounding space.
+func cleanText(s string) string {
+	return strings.TrimSpace(html.UnescapeString(s))
+}
+
+func siteName(publisher any, g graph, ogSiteName string, base *url.URL) (string, error) {
+	pubs, ok := publisher.([]any)
+	if !ok {
+		pubs = []any{publisher}
+	}
+	for _, p := range pubs {
+		p, _, err := g.resolve(p)
+		if err != nil {
+			return "", fmt.Errorf("publisher: %w", err)
+		}
+		if m, ok := p.(map[string]any); ok {
+			if name, ok := m["name"].(string); ok && cleanText(name) != "" {
+				return cleanText(name), nil
+			}
+		}
+	}
+	if ogSiteName != "" {
+		return ogSiteName, nil
+	}
+	return strings.TrimPrefix(base.Hostname(), "www."), nil
+}
+
+// imageURL picks the largest of the recipe's images by width × height;
+// images without both dimensions count as size zero, and the first wins a
+// tie, so a list of bare URLs gives its first.
+func imageURL(v any, g graph, base *url.URL) (string, error) {
+	items, ok := v.([]any)
+	if !ok {
+		if v == nil {
+			return "", nil
+		}
+		items = []any{v}
+	}
+	best, bestArea := "", int64(-1)
+	for _, item := range items {
+		item, found, err := g.resolve(item)
+		if err != nil {
+			return "", fmt.Errorf("image: %w", err)
+		}
+		if !found {
+			return "", fmt.Errorf("image references @id %v, which no node on the page has", item.(map[string]any)["@id"])
+		}
+		var u string
+		var area int64
+		switch img := item.(type) {
+		case string:
+			u = img
+		case map[string]any:
+			if s, ok := img["url"].(string); ok {
+				u = s
+			} else if s, ok := img["contentUrl"].(string); ok {
+				u = s
+			}
+			w, err := dimension(img["width"])
+			if err != nil {
+				return "", fmt.Errorf("image width: %w", err)
+			}
+			h, err := dimension(img["height"])
+			if err != nil {
+				return "", fmt.Errorf("image height: %w", err)
+			}
+			area = w * h
+		default:
+			return "", fmt.Errorf("image is a %T, not a URL or ImageObject", item)
+		}
+		u = strings.TrimSpace(u)
+		if u == "" {
+			return "", errors.New("image has no URL")
+		}
+		if area > bestArea {
+			best, bestArea = u, area
+		}
+	}
+	if best == "" {
+		return "", nil
+	}
+	ref, err := url.Parse(best)
+	if err != nil {
+		return "", fmt.Errorf("image URL %q: %w", best, err)
+	}
+	return base.ResolveReference(ref).String(), nil
+}
+
+// dimension reads an ImageObject width or height: a number or a string of
+// digits; zero when absent.
+func dimension(v any) (int64, error) {
+	switch d := v.(type) {
+	case nil:
+		return 0, nil
+	case json.Number:
+		f, err := d.Float64()
+		if err != nil || f < 0 {
+			return 0, fmt.Errorf("%q is not a size", d)
+		}
+		return int64(f), nil
+	case string:
+		n, err := strconv.ParseInt(strings.TrimSpace(d), 10, 64)
+		if err != nil || n < 0 {
+			return 0, fmt.Errorf("%q is not a size", d)
+		}
+		return n, nil
+	}
+	return 0, fmt.Errorf("a %T is not a size", v)
+}
+
+func ingredients(v any) ([]string, error) {
+	items, ok := v.([]any)
+	if !ok {
+		if v == nil {
+			return nil, nil
+		}
+		items = []any{v}
+	}
+	var out []string
+	for _, item := range items {
+		s, ok := item.(string)
+		if !ok {
+			return nil, fmt.Errorf("recipeIngredient holds a %T, not text", item)
+		}
+		if s = cleanText(s); s != "" {
+			out = append(out, s)
+		}
+	}
+	return out, nil
+}
+
+// steps flattens recipeInstructions: one string of lines, or a list of
+// strings, HowToSteps and HowToSections of those, in order.
+func steps(v any) ([]string, error) {
+	switch in := v.(type) {
+	case nil:
+		return nil, nil
+	case string:
+		var out []string
+		for _, line := range strings.Split(in, "\n") {
+			if line = cleanText(line); line != "" {
+				out = append(out, line)
+			}
+		}
+		return out, nil
+	case map[string]any:
+		return stepList([]any{in})
+	case []any:
+		return stepList(in)
+	}
+	return nil, fmt.Errorf("recipeInstructions is a %T", v)
+}
+
+func stepList(items []any) ([]string, error) {
+	var out []string
+	for _, item := range items {
+		switch s := item.(type) {
+		case string:
+			if text := cleanText(s); text != "" {
+				out = append(out, text)
+			}
+		case map[string]any:
+			switch {
+			case hasType(s, "HowToStep"):
+				text, _ := s["text"].(string)
+				if text = cleanText(text); text == "" {
+					return nil, errors.New("HowToStep has no text")
+				}
+				out = append(out, text)
+			case hasType(s, "HowToSection"):
+				list, ok := s["itemListElement"].([]any)
+				if !ok {
+					return nil, errors.New("HowToSection has no itemListElement list")
+				}
+				section, err := stepList(list)
+				if err != nil {
+					return nil, err
+				}
+				out = append(out, section...)
+			default:
+				return nil, fmt.Errorf("recipeInstructions holds a %v, not a HowToStep or HowToSection", s["@type"])
+			}
+		default:
+			return nil, fmt.Errorf("recipeInstructions holds a %T", item)
+		}
+	}
+	return out, nil
+}
+
+// minutes reads a duration field: nil when absent or null; any string
+// present, empty included, must be an ISO-8601 duration.
+func minutes(n map[string]any, field string) (*int, error) {
+	v, ok := n[field]
+	if !ok || v == nil {
+		return nil, nil
+	}
+	s, ok := v.(string)
+	if !ok {
+		return nil, fmt.Errorf("%s is a %T, not an ISO-8601 duration", field, v)
+	}
+	m, err := isoMinutes(s)
+	if err != nil {
+		return nil, fmt.Errorf("%s: %w", field, err)
+	}
+	return &m, nil
+}
+
+// isoMinutes converts an ISO-8601 duration of weeks, days, hours, minutes
+// and seconds to whole minutes, rounded. Years and months are refused:
+// their length in minutes is not fixed.
+func isoMinutes(s string) (int, error) {
+	bad := fmt.Errorf("%q is not an ISO-8601 duration of days, hours, minutes or seconds", s)
+	rest, ok := strings.CutPrefix(strings.TrimSpace(s), "P")
+	if !ok || rest == "" {
+		return 0, bad
+	}
+	const units = "WDHMS"
+	seconds := map[byte]float64{'W': 7 * 86400, 'D': 86400, 'H': 3600, 'M': 60, 'S': 1}
+	inTime, last, total := false, -1, 0.0
+	for rest != "" {
+		if rest[0] == 'T' {
+			if inTime || len(rest) == 1 {
+				return 0, bad
+			}
+			inTime, rest = true, rest[1:]
+			continue
+		}
+		i := 0
+		for i < len(rest) && (rest[i] >= '0' && rest[i] <= '9' || rest[i] == '.' || rest[i] == ',') {
+			i++
+		}
+		if i == 0 || i == len(rest) {
+			return 0, bad
+		}
+		num, err := strconv.ParseFloat(strings.Replace(rest[:i], ",", ".", 1), 64)
+		if err != nil {
+			return 0, bad
+		}
+		unit := rest[i]
+		pos := strings.IndexByte(units, unit)
+		if pos < 0 || pos <= last || inTime != (pos >= 2) {
+			return 0, bad
+		}
+		last, total, rest = pos, total+num*seconds[unit], rest[i+1:]
+	}
+	if last < 0 {
+		return 0, bad
+	}
+	return int(math.Round(total / 60)), nil
+}
+
+func yield(v any) (string, error) {
+	if list, ok := v.([]any); ok {
+		if len(list) == 0 {
+			return "", nil
+		}
+		v = list[0]
+	}
+	switch y := v.(type) {
+	case nil:
+		return "", nil
+	case string:
+		return cleanText(y), nil
+	case json.Number:
+		return y.String(), nil
+	}
+	return "", fmt.Errorf("recipeYield is a %T", v)
+}
+
+// htmlDoc is what Extract needs from a page's HTML.
+type htmlDoc struct {
+	jsonLD     []string
+	ogSiteName string
+}
+
+// scanHTML is a small targeted scan, not a full HTML parser: it walks the
+// tags, skipping comments and the bodies of scripts and styles, and keeps
+// the bodies of application/ld+json scripts and the first og:site_name.
+func scanHTML(doc string) htmlDoc {
+	var out htmlDoc
+	lower := asciiLower(doc)
+	for pos := 0; ; {
+		lt := strings.IndexByte(doc[pos:], '<')
+		if lt < 0 {
+			return out
+		}
+		pos += lt
+		if strings.HasPrefix(doc[pos:], "<!--") {
+			end := strings.Index(doc[pos+4:], "-->")
+			if end < 0 {
+				return out
+			}
+			pos += 4 + end + 3
+			continue
+		}
+		name := tagName(lower[pos+1:])
+		if name == "" {
+			pos++
+			continue
+		}
+		end := tagEnd(doc, pos+1+len(name))
+		if end < 0 {
+			return out
+		}
+		attrs := parseAttrs(doc[pos+1+len(name) : end])
+		pos = end + 1
+		switch name {
+		case "meta":
+			if out.ogSiteName == "" && attrs["property"] == "og:site_name" {
+				out.ogSiteName = strings.TrimSpace(attrs["content"])
+			}
+		case "script", "style":
+			closeAt := strings.Index(lower[pos:], "</"+name)
+			if closeAt < 0 {
+				return out
+			}
+			typ, _, _ := strings.Cut(attrs["type"], ";")
+			if name == "script" && strings.EqualFold(strings.TrimSpace(typ), "application/ld+json") {
+				out.jsonLD = append(out.jsonLD, doc[pos:pos+closeAt])
+			}
+			pos += closeAt
+		}
+	}
+}
+
+// asciiLower lower-cases ASCII letters only, so byte offsets match doc.
+func asciiLower(s string) string {
+	b := []byte(s)
+	for i, c := range b {
+		if 'A' <= c && c <= 'Z' {
+			b[i] = c + 'a' - 'A'
+		}
+	}
+	return string(b)
+}
+
+// tagName reads the lower-cased name of an opening tag at the start of s,
+// or "" when s does not start one.
+func tagName(s string) string {
+	i := 0
+	for i < len(s) && ('a' <= s[i] && s[i] <= 'z' || i > 0 && '0' <= s[i] && s[i] <= '9') {
+		i++
+	}
+	if i == 0 || i < len(s) && !isSpace(s[i]) && s[i] != '>' && s[i] != '/' {
+		return ""
+	}
+	return s[:i]
+}
+
+// tagEnd finds the '>' closing a tag whose attributes start at i, skipping
+// quoted attribute values.
+func tagEnd(doc string, i int) int {
+	var quote byte
+	for ; i < len(doc); i++ {
+		switch c := doc[i]; {
+		case quote != 0:
+			if c == quote {
+				quote = 0
+			}
+		case c == '"' || c == '\'':
+			quote = c
+		case c == '>':
+			return i
+		}
+	}
+	return -1
+}
+
+// parseAttrs reads a tag's attributes; names are lower-cased, values are
+// entity-decoded, and the first of a repeated name wins.
+func parseAttrs(s string) map[string]string {
+	attrs := map[string]string{}
+	i := 0
+	for i < len(s) {
+		for i < len(s) && (isSpace(s[i]) || s[i] == '/') {
+			i++
+		}
+		start := i
+		for i < len(s) && !isSpace(s[i]) && s[i] != '=' && s[i] != '/' {
+			i++
+		}
+		if i == start {
+			i++
+			continue
+		}
+		name := asciiLower(s[start:i])
+		for i < len(s) && isSpace(s[i]) {
+			i++
+		}
+		value := ""
+		if i < len(s) && s[i] == '=' {
+			i++
+			for i < len(s) && isSpace(s[i]) {
+				i++
+			}
+			if i < len(s) && (s[i] == '"' || s[i] == '\'') {
+				q := s[i]
+				end := strings.IndexByte(s[i+1:], q)
+				if end < 0 {
+					end = len(s) - i - 1
+				}
+				value = s[i+1 : i+1+end]
+				i += end + 2
+			} else {
+				start := i
+				for i < len(s) && !isSpace(s[i]) {
+					i++
+				}
+				value = s[start:i]
+			}
+		}
+		if _, seen := attrs[name]; !seen {
+			attrs[name] = html.UnescapeString(value)
+		}
+	}
+	return attrs
+}
+
+func isSpace(c byte) bool {
+	return c == ' ' || c == '\t' || c == '\n' || c == '\r' || c == '\f'
+}

internal/recipes/extract_test.go

@@ -0,0 +1,408 @@
+package recipes
+
+import (
+	"encoding/json"
+	"errors"
+	"os"
+	"path/filepath"
+	"strings"
+	"testing"
+)
+
+func readPage(t *testing.T, name string) []byte {
+	t.Helper()
+	b, err := os.ReadFile(filepath.Join("testdata", name))
+	if err != nil {
+		t.Fatal(err)
+	}
+	return b
+}
+
+func intp(n int) *int { return &n }
+
+func fmtMinutes(m *int) string {
+	if m == nil {
+		return "nil"
+	}
+	b, _ := json.Marshal(*m)
+	return string(b)
+}
+
+func TestExtractRealPages(t *testing.T) {
+	tests := []struct {
+		file, url               string
+		title, site, image, yld string
+		ingredients, steps      int
+		prep, cook, total       *int
+		firstStep               string
+	}{
+		{
+			// A Recipe script beside VideoObject and BreadcrumbList scripts;
+			// image is an array of one sized ImageObject.
+			file: "bbcgoodfood.html", url: "https://www.bbcgoodfood.com/recipes/easy-chicken-curry",
+			title: "Easy chicken curry", site: "Good Food",
+			image:       "https://images.immediate.co.uk/production/volatile/sites/30/2020/08/easy-chicken-curry-aa22a0b.jpg?resize=440,400",
+			ingredients: 11, steps: 4, prep: intp(5), cook: intp(45), total: intp(50), yld: "4",
+			firstStep: "Heat the oil in a flameproof casserole dish",
+		},
+		{
+			// Yoast @graph; image is a list of bare URLs, the first the
+			// full-size one; no publisher, so og:site_name names the site.
+			file: "budgetbytes.html", url: "https://www.budgetbytes.com/one-pot-creamy-cajun-chicken-pasta/",
+			title: "One Pot Creamy Cajun Chicken Pasta", site: "Budget Bytes",
+			image:       "https://www.budgetbytes.com/wp-content/uploads/2018/10/One-Pot-Creamy-Cajun-Chicken-Pasta-pan.jpg",
+			ingredients: 17, steps: 6, prep: intp(10), cook: intp(20), total: intp(30), yld: "4",
+			firstStep: "Combine the ingredients for the Cajun seasoning",
+		},
+		{
+			// HowToSection groups (ragu, cheese sauce, assembly), flattened.
+			file: "recipetineats.html", url: "https://www.recipetineats.com/lasagna/",
+			title: "Lasagna", site: "RecipeTin Eats",
+			image:       "https://www.recipetineats.com/tachyon/2017/05/Lasagne-recipe-3-main-square.jpg",
+			ingredients: 24, steps: 18, prep: intp(20), cook: intp(240), total: intp(260), yld: "10",
+			firstStep: "Heat oil in a large heavy based pot",
+		},
+		{
+			// @graph of one node; a single ImageObject; instructions as a
+			// list of plain strings; hours and minutes.
+			file: "kingarthur.html", url: "https://www.kingarthurbaking.com/recipes/classic-sandwich-bread-recipe",
+			title: "Classic Sandwich Bread", site: "King Arthur Baking",
+			image:       "https://www.kingarthurbaking.com/sites/default/files/2026-02/Classic-Sandwich-Bread-12.jpg",
+			ingredients: 7, steps: 7, prep: intp(12), cook: intp(40), total: intp(162), yld: "16",
+			firstStep: "Weigh your flour",
+		},
+		{
+			// A top-level array; ImageObjects of different sizes.
+			file: "delish.html", url: "https://www.delish.com/cooking/recipe-ideas/a19636089/creamy-tuscan-chicken-recipe/",
+			title: "Creamy Tuscan Chicken", site: "Delish",
+			image:       "https://hips.hearstapps.com/hmg-prod/images/1d09a24f-5b76-49f4-9ad6-496acdc03933.jpg?crop=0.502xw:1.00xh;0.250xw,0&resize=1200:*",
+			ingredients: 12, steps: 4, prep: intp(10), cook: intp(0), total: intp(45), yld: "4 serving(s)",
+			firstStep: "In a large skillet over medium heat, heat oil.",
+		},
+		{
+			// A bare Recipe object; image is one URL; HTML entities in steps.
+			file: "foodcom.html", url: "https://www.food.com/recipe/best-banana-bread-2886",
+			title: "Best Banana Bread", site: "Food.com",
+			image:       "https://img.sndimg.com/food/image/upload/q_92,fl_progressive,w_1200,c_scale/v1/img/recipes/28/86/wBZxNua1T8yfDwbfo9Fz_0S9A9315.jpg",
+			ingredients: 8, steps: 10, prep: intp(10), cook: intp(60), total: intp(70), yld: "1 loaf, 10 serving(s)",
+			firstStep: "Remove odd pots and pans from oven.",
+		},
+	}
+	for _, tt := range tests {
+		t.Run(tt.file, func(t *testing.T) {
+			r, err := Extract(readPage(t, tt.file), tt.url)
+			if err != nil {
+				t.Fatalf("Extract: %v", err)
+			}
+			if r.Title != tt.title {
+				t.Errorf("Title = %q, want %q", r.Title, tt.title)
+			}
+			if r.SiteName != tt.site {
+				t.Errorf("SiteName = %q, want %q", r.SiteName, tt.site)
+			}
+			if r.ImageURL != tt.image {
+				t.Errorf("ImageURL = %q, want %q", r.ImageURL, tt.image)
+			}
+			if len(r.Ingredients) != tt.ingredients {
+				t.Errorf("%d ingredients, want %d", len(r.Ingredients), tt.ingredients)
+			}
+			if len(r.Steps) != tt.steps {
+				t.Errorf("%d steps, want %d", len(r.Steps), tt.steps)
+			}
+			if len(r.Steps) > 0 && !strings.HasPrefix(r.Steps[0], tt.firstStep) {
+				t.Errorf("first step = %q, want prefix %q", r.Steps[0], tt.firstStep)
+			}
+			for name, got := range map[string][2]*int{
+				"prep": {r.PrepMinutes, tt.prep}, "cook": {r.CookMinutes, tt.cook}, "total": {r.TotalMinutes, tt.total},
+			} {
+				if fmtMinutes(got[0]) != fmtMinutes(got[1]) {
+					t.Errorf("%s minutes = %s, want %s", name, fmtMinutes(got[0]), fmtMinutes(got[1]))
+				}
+			}
+			if r.Yield != tt.yld {
+				t.Errorf("Yield = %q, want %q", r.Yield, tt.yld)
+			}
+			var node map[string]any
+			if err := json.Unmarshal(r.JSONLD, &node); err != nil || node["name"] == nil {
+				t.Errorf("JSONLD is not the Recipe node: %v", err)
+			}
+		})
+	}
+}
+
+func TestExtractHowToSectionOrder(t *testing.T) {
+	r, err := Extract(readPage(t, "recipetineats.html"), "https://www.recipetineats.com/lasagna/")
+	if err != nil {
+		t.Fatal(err)
+	}
+	// Ragu (5 steps), Cheese Sauce (5), Assemble (8): each section's first
+	// step lands right after the previous section's last.
+	for i, prefix := range map[int]string{
+		0:  "Heat oil in a large heavy based pot",
+		4:  "The ragu is ready when the meat",
+		5:  "Warm milk up in a saucepan",
+		10: "Preheat oven to 180°C/350°F.",
+		17: "Stand for 5 to 10 minutes",
+	} {
+		if !strings.HasPrefix(r.Steps[i], prefix) {
+			t.Errorf("step %d = %q, want prefix %q", i+1, r.Steps[i], prefix)
+		}
+	}
+}
+
+func TestExtractNoRecipe(t *testing.T) {
+	_, err := Extract(readPage(t, "budgetbytes-about.html"), "https://www.budgetbytes.com/about/")
+	if !errors.Is(err, ErrNoRecipe) {
+		t.Fatalf("err = %v, want ErrNoRecipe", err)
+	}
+	_, err = Extract([]byte("<html><head><title>x</title></head></html>"), "https://example.com/")
+	if !errors.Is(err, ErrNoRecipe) {
+		t.Fatalf("page without JSON-LD: err = %v, want ErrNoRecipe", err)
+	}
+}
+
+func TestExtractRefusesNonISODuration(t *testing.T) {
+	// Bon Appétit writes cookTime as "20 minutes".
+	_, err := Extract(readPage(t, "bonappetit.html"), "https://www.bonappetit.com/recipe/bas-best-chocolate-chip-cookies")
+	if !errors.Is(err, ErrMalformedRecipe) || !strings.Contains(err.Error(), "cookTime") {
+		t.Fatalf("err = %v, want ErrMalformedRecipe naming cookTime", err)
+	}
+}
+
+// page wraps JSON-LD scripts in a minimal HTML page.
+func page(head string, jsonLD ...string) []byte {
+	var b strings.Builder
+	b.WriteString("<!DOCTYPE html><html><head>" + head)
+	for _, s := range jsonLD {
+		b.WriteString(`<script type="application/ld+json">` + s + "</script>")
+	}
+	b.WriteString("</head><body></body></html>")
+	return []byte(b.String())
+}
+
+func TestExtractShapes(t *testing.T) {
+	const pageURL = "https://www.example.com/recipes/stew/"
+	tests := []struct {
+		name  string
+		html  []byte
+		check func(t *testing.T, r Recipe)
+	}{
+		{
+			name: "type array and relative image",
+			html: page("", `{"@type":["Recipe","NewsArticle"],"name":"Stew","image":"/img/stew.jpg"}`),
+			check: func(t *testing.T, r Recipe) {
+				if r.ImageURL != "https://www.example.com/img/stew.jpg" {
+					t.Errorf("ImageURL = %q", r.ImageURL)
+				}
+			},
+		},
+		{
+			name: "largest image wins regardless of order",
+			html: page("", `{"@type":"Recipe","name":"Stew","image":[
+				"https://cdn.example.com/plain.jpg",
+				{"@type":"ImageObject","url":"small.jpg","width":300,"height":200},
+				{"@type":"ImageObject","contentUrl":"../big.jpg","width":"1600","height":"900"},
+				{"@type":"ImageObject","url":"mid.jpg","width":1200,"height":800}]}`),
+			check: func(t *testing.T, r Recipe) {
+				if r.ImageURL != "https://www.example.com/recipes/big.jpg" {
+					t.Errorf("ImageURL = %q", r.ImageURL)
+				}
+			},
+		},
+		{
+			name: "instructions as one string",
+			html: page("", `{"@type":"Recipe","name":"Stew","recipeInstructions":"Brown the beef.\r\n\r\nAdd the stock &amp; simmer.\nServe."}`),
+			check: func(t *testing.T, r Recipe) {
+				want := []string{"Brown the beef.", "Add the stock & simmer.", "Serve."}
+				if strings.Join(r.Steps, "|") != strings.Join(want, "|") {
+					t.Errorf("Steps = %q, want %q", r.Steps, want)
+				}
+			},
+		},
+		{
+			name: "sections and steps flattened in order",
+			html: page("", `{"@type":"Recipe","name":"Stew","recipeInstructions":[
+				{"@type":"HowToSection","name":"Beef","itemListElement":[
+					{"@type":"HowToStep","text":"one"},{"@type":"HowToStep","text":"two"}]},
+				{"@type":"HowToStep","text":"three"},
+				{"@type":"HowToSection","name":"Finish","itemListElement":["four"]}]}`),
+			check: func(t *testing.T, r Recipe) {
+				if got := strings.Join(r.Steps, " "); got != "one two three four" {
+					t.Errorf("Steps = %q", got)
+				}
+			},
+		},
+		{
+			name: "total derived from prep and cook",
+			html: page("", `{"@type":"Recipe","name":"Stew","prepTime":"PT15M","cookTime":"PT1H30M"}`),
+			check: func(t *testing.T, r Recipe) {
+				if fmtMinutes(r.TotalMinutes) != "105" {
+					t.Errorf("TotalMinutes = %s, want 105", fmtMinutes(r.TotalMinutes))
+				}
+			},
+		},
+		{
+			name: "total not derived from prep alone",
+			html: page("", `{"@type":"Recipe","name":"Stew","prepTime":"PT15M"}`),
+			check: func(t *testing.T, r Recipe) {
+				if r.TotalMinutes != nil || r.CookMinutes != nil {
+					t.Errorf("total %s, cook %s; want nil", fmtMinutes(r.TotalMinutes), fmtMinutes(r.CookMinutes))
+				}
+			},
+		},
+		{
+			name: "site name from host without publisher or og:site_name",
+			html: page("", `{"@type":"Recipe","name":"Stew"}`),
+			check: func(t *testing.T, r Recipe) {
+				if r.SiteName != "example.com" {
+					t.Errorf("SiteName = %q", r.SiteName)
+				}
+			},
+		},
+		{
+			name: "og:site_name with attributes in any order",
+			html: page(`<meta content="Our &amp; Kitchen" property="og:site_name">`, `{"@type":"Recipe","name":"Stew","publisher":{"@id":"#org"}}`),
+			check: func(t *testing.T, r Recipe) {
+				if r.SiteName != "Our & Kitchen" {
+					t.Errorf("SiteName = %q", r.SiteName)
+				}
+			},
+		},
+		{
+			name: "numeric yield and recipe in @graph beside others",
+			html: page("", `{"@context":"https://schema.org","@graph":[{"@type":"WebPage"},{"@type":"Recipe","name":"Stew","recipeYield":6}]}`),
+			check: func(t *testing.T, r Recipe) {
+				if r.Yield != "6" {
+					t.Errorf("Yield = %q", r.Yield)
+				}
+			},
+		},
+		{
+			name: "image and publisher resolved by @id within @graph",
+			html: page(`<meta property="og:site_name" content="Not this">`, `{"@context":"https://schema.org","@graph":[
+				{"@type":"Recipe","name":"Stew",
+					"image":{"@id":"https://www.example.com/#primaryimage"},
+					"publisher":{"@id":"https://www.example.com/#organization"}},
+				{"@type":"ImageObject","@id":"https://www.example.com/#primaryimage","url":"/img/stew.jpg","width":1200,"height":800},
+				{"@type":"Organization","@id":"https://www.example.com/#organization","name":"Our Kitchen"}]}`),
+			check: func(t *testing.T, r Recipe) {
+				if r.ImageURL != "https://www.example.com/img/stew.jpg" {
+					t.Errorf("ImageURL = %q", r.ImageURL)
+				}
+				if r.SiteName != "Our Kitchen" {
+					t.Errorf("SiteName = %q", r.SiteName)
+				}
+			},
+		},
+		{
+			name: "image @id resolved from another script",
+			html: page("", `{"@type":"Recipe","name":"Stew","image":[{"@id":"#img"}]}`,
+				`{"@type":"ImageObject","@id":"#img","contentUrl":"stew.jpg"}`),
+			check: func(t *testing.T, r Recipe) {
+				if r.ImageURL != "https://www.example.com/recipes/stew/stew.jpg" {
+					t.Errorf("ImageURL = %q", r.ImageURL)
+				}
+			},
+		},
+		{
+			name: "absolute schema.org type IRIs",
+			html: page("", `{"@type":"https://schema.org/Recipe","name":"Stew","recipeInstructions":[
+				{"@type":"http://schema.org/HowToStep","text":"one"}]}`),
+			check: func(t *testing.T, r Recipe) {
+				if r.Title != "Stew" || strings.Join(r.Steps, "|") != "one" {
+					t.Errorf("Title = %q, Steps = %q", r.Title, r.Steps)
+				}
+			},
+		},
+		{
+			name: "absolute type IRI in a type array",
+			html: page("", `{"@type":["NewsArticle","http://schema.org/Recipe"],"name":"Stew"}`),
+			check: func(t *testing.T, r Recipe) {
+				if r.Title != "Stew" {
+					t.Errorf("Title = %q", r.Title)
+				}
+			},
+		},
+		{
+			name: "null duration is absent",
+			html: page("", `{"@type":"Recipe","name":"Stew","prepTime":null,"cookTime":"PT10M"}`),
+			check: func(t *testing.T, r Recipe) {
+				if r.PrepMinutes != nil || r.TotalMinutes != nil || fmtMinutes(r.CookMinutes) != "10" {
+					t.Errorf("prep %s, cook %s, total %s", fmtMinutes(r.PrepMinutes), fmtMinutes(r.CookMinutes), fmtMinutes(r.TotalMinutes))
+				}
+			},
+		},
+		{
+			name: "JSON-LD text inside comments and other scripts is ignored",
+			html: []byte(`<html><head><!-- <script type="application/ld+json">{"@type":"Recipe","name":"Old"}</script> -->` +
+				`<script>var s = '<script type="application/ld+json">';</script>` +
+				`<SCRIPT TYPE='application/ld+json; charset=utf-8'>{"@type":"Recipe","name":"Stew"}</SCRIPT></head></html>`),
+			check: func(t *testing.T, r Recipe) {
+				if r.Title != "Stew" {
+					t.Errorf("Title = %q", r.Title)
+				}
+			},
+		},
+	}
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			r, err := Extract(tt.html, pageURL)
+			if err != nil {
+				t.Fatalf("Extract: %v", err)
+			}
+			tt.check(t, r)
+		})
+	}
+}
+
+func TestExtractErrors(t *testing.T) {
+	tests := []struct {
+		name string
+		html []byte
+		url  string
+		want error
+	}{
+		{"two recipes", page("", `{"@type":"Recipe","name":"A"}`, `{"@type":"Recipe","name":"B"}`), "https://example.com/", ErrManyRecipes},
+		{"invalid JSON", page("", `{"@type":"Recipe",`), "https://example.com/", ErrMalformedRecipe},
+		{"no name", page("", `{"@type":"Recipe"}`), "https://example.com/", ErrMalformedRecipe},
+		{"bad step type", page("", `{"@type":"Recipe","name":"A","recipeInstructions":[{"@type":"Thing"}]}`), "https://example.com/", ErrMalformedRecipe},
+		{"step without text", page("", `{"@type":"Recipe","name":"A","recipeInstructions":[{"@type":"HowToStep","name":"x"}]}`), "https://example.com/", ErrMalformedRecipe},
+		{"bad image width", page("", `{"@type":"Recipe","name":"A","image":{"url":"a.jpg","width":"wide"}}`), "https://example.com/", ErrMalformedRecipe},
+		{"empty prepTime", page("", `{"@type":"Recipe","name":"A","prepTime":""}`), "https://example.com/", ErrMalformedRecipe},
+		{"blank cookTime", page("", `{"@type":"Recipe","name":"A","cookTime":"  "}`), "https://example.com/", ErrMalformedRecipe},
+		{"empty totalTime", page("", `{"@type":"Recipe","name":"A","totalTime":""}`), "https://example.com/", ErrMalformedRecipe},
+		{"image @id not on page", page("", `{"@type":"Recipe","name":"A","image":{"@id":"#missing"}}`), "https://example.com/", ErrMalformedRecipe},
+		{"image @id on two nodes", page("", `{"@type":"Recipe","name":"A","image":{"@id":"#img"}}`,
+			`{"@id":"#img","url":"a.jpg"}`, `{"@id":"#img","url":"b.jpg"}`), "https://example.com/", ErrMalformedRecipe},
+		{"non-text ingredient", page("", `{"@type":"Recipe","name":"A","recipeIngredient":[{"name":"salt"}]}`), "https://example.com/", ErrMalformedRecipe},
+	}
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			_, err := Extract(tt.html, tt.url)
+			if !errors.Is(err, tt.want) {
+				t.Fatalf("err = %v, want %v", err, tt.want)
+			}
+		})
+	}
+	if _, err := Extract(page("", `{"@type":"Recipe","name":"A"}`), "/relative"); err == nil {
+		t.Fatal("relative page URL accepted")
+	}
+}
+
+func TestISOMinutes(t *testing.T) {
+	good := map[string]int{
+		"PT0S": 0, "PT45M": 45, "PT1H": 60, "PT2H42M": 162, "PT240M": 240, "P1D": 1440,
+		"P1DT2H": 1560, "PT90S": 2, "PT1.5H": 90, "PT0,5H": 30, "P1W": 10080, " PT5M ": 5,
+	}
+	for in, want := range good {
+		got, err := isoMinutes(in)
+		if err != nil || got != want {
+			t.Errorf("isoMinutes(%q) = %d, %v; want %d", in, got, err, want)
+		}
+	}
+	for _, in := range []string{"", "P", "PT", "20 minutes", "PT5", "P1Y", "P1M", "PT1D", "P5H", "PT5M1H", "PT5M5M", "P1DT", "PTxM", "PT1..5M"} {
+		if got, err := isoMinutes(in); err == nil {
+			t.Errorf("isoMinutes(%q) = %d, want error", in, got)
+		}
+	}
+}

internal/recipes/fetcher.go

@@ -0,0 +1,523 @@
+// Package recipes fetches and holds real recipe pages that ground dinners.
+package recipes
+
+import (
+	"bytes"
+	"context"
+	"crypto/tls"
+	"errors"
+	"fmt"
+	"io"
+	"net"
+	"net/http"
+	"net/netip"
+	"net/url"
+	"strings"
+	"sync"
+	"time"
+)
+
+const (
+	// UserAgent identifies MealQuest to the recipe sites it fetches.
+	UserAgent = "MealQuest/1.0 (household meal planner; recipe fetcher)"
+	// MaxBodyBytes caps a fetched body. A larger page is an error, never
+	// silently truncated: a cut-off recipe page would parse as a wrong one.
+	MaxBodyBytes = 2 << 20
+	// MaxRedirects is how many redirects a fetch follows; each hop is
+	// re-checked for scheme and address.
+	MaxRedirects = 3
+	// Timeout bounds a whole fetch: rate-limit waits, every redirect hop
+	// and reading the body.
+	Timeout = 10 * time.Second
+	// HostInterval is the least time between two requests to one host.
+	HostInterval = time.Second
+	// CacheTTL is how long a fetched page is served from the cache.
+	CacheTTL = time.Hour
+	// CacheSize is how many pages the cache holds; the oldest is evicted
+	// first.
+	CacheSize = 64
+)
+
+var (
+	// ErrNotHTTPS is returned for a URL, or a redirect, that is not
+	// https://.
+	ErrNotHTTPS = errors.New("recipes: only https:// URLs are fetched")
+	// ErrBlockedAddress is returned when a host resolves to an address
+	// that is not public, such as loopback or a private network.
+	ErrBlockedAddress = errors.New("recipes: address is not public")
+	// ErrTooManyRedirects is returned after MaxRedirects redirects.
+	ErrTooManyRedirects = errors.New("recipes: too many redirects")
+	// ErrTooLarge is returned for a body over MaxBodyBytes.
+	ErrTooLarge = errors.New("recipes: body exceeds size cap")
+	// ErrTimeout is returned when a fetch runs past Timeout.
+	ErrTimeout = errors.New("recipes: fetch timed out")
+)
+
+// StatusError is returned for a final response whose status is not 2xx.
+type StatusError struct {
+	URL    string
+	Status int
+}
+
+func (e *StatusError) Error() string {
+	return fmt.Sprintf("recipes: GET %s: HTTP %d", e.URL, e.Status)
+}
+
+// Page is a fetched page.
+type Page struct {
+	// URL is the final URL, after redirects.
+	URL         string
+	ContentType string
+	Body        []byte
+}
+
+// Resolver looks up a host's addresses; *net.Resolver satisfies it.
+type Resolver interface {
+	LookupIPAddr(ctx context.Context, host string) ([]net.IPAddr, error)
+}
+
+// dialFunc dials an address that has already passed the public-address
+// check.
+type dialFunc func(ctx context.Context, network, address string) (net.Conn, error)
+
+// Fetcher retrieves recipe pages without reaching private networks. Its
+// dialer resolves each host itself, refuses the connection unless every
+// answer is a public address, and dials the checked address, so a second DNS
+// answer cannot swap in a private one. Keep-alives are off: every request,
+// redirect hops included, dials and checks afresh. Proxies from the
+// environment are not used, since a proxy would dial on the Fetcher's
+// behalf. Requests are HTTP/1.1, so each connection carries exactly one
+// request and its first write is that request. A Fetcher is safe for
+// concurrent use.
+type Fetcher struct {
+	client    *http.Client
+	resolver  Resolver
+	dial      dialFunc
+	tlsConfig *tls.Config
+	// timeout bounds a fetch; tests shorten it.
+	timeout time.Duration
+	// now is the clock; tests replace it.
+	now func() time.Time
+	// sleep waits d or until ctx is done; tests replace it.
+	sleep func(ctx context.Context, d time.Duration) error
+
+	mu sync.Mutex
+	// lastSent is when each host's latest request began to be written to
+	// its connection; a request whose write fails before any byte is sent
+	// hands its slot back.
+	lastSent map[string]time.Time
+	// released is closed, then replaced, whenever a slot is handed back,
+	// waking requests waiting for their turn.
+	released chan struct{}
+	cache    map[string]cacheEntry
+}
+
+type cacheEntry struct {
+	page    Page
+	fetched time.Time
+}
+
+// NewFetcher returns a Fetcher that resolves through the system resolver.
+func NewFetcher() *Fetcher {
+	return newFetcher(net.DefaultResolver, (&net.Dialer{}).DialContext, nil)
+}
+
+// newFetcher returns a Fetcher over resolver and dial; tlsConfig, when not
+// nil, replaces the default TLS settings so tests can trust their servers.
+func newFetcher(resolver Resolver, dial dialFunc, tlsConfig *tls.Config) *Fetcher {
+	f := &Fetcher{
+		resolver:  resolver,
+		dial:      dial,
+		tlsConfig: tlsConfig,
+		timeout:   Timeout,
+		now:       time.Now,
+		sleep:     sleep,
+		lastSent:  map[string]time.Time{},
+		released:  make(chan struct{}),
+		cache:     map[string]cacheEntry{},
+	}
+	transport := &http.Transport{
+		Proxy:             nil,
+		DialTLSContext:    f.dialTLS,
+		DisableKeepAlives: true,
+	}
+	f.client = &http.Client{
+		Transport:     &requestTransport{next: transport},
+		CheckRedirect: checkRedirect,
+	}
+	return f
+}
+
+// Fetch returns the page at rawURL, from the cache when it was fetched
+// within CacheTTL. Only 2xx responses are returned and cached. The
+// returned Body is the caller's own copy.
+func (f *Fetcher) Fetch(ctx context.Context, rawURL string) (Page, error) {
+	u, err := url.Parse(rawURL)
+	if err != nil {
+		return Page{}, fmt.Errorf("recipes: parse URL: %w", err)
+	}
+	if err := checkURL(u); err != nil {
+		return Page{}, err
+	}
+	key := u.String()
+	if page, ok := f.cached(key); ok {
+		return page, nil
+	}
+
+	fetchCtx, cancel := context.WithTimeout(ctx, f.timeout)
+	defer cancel()
+	page, err := f.get(fetchCtx, key)
+	if err != nil {
+		if ctx.Err() == nil && errors.Is(fetchCtx.Err(), context.DeadlineExceeded) {
+			return Page{}, fmt.Errorf("%w after %s: %s", ErrTimeout, f.timeout, key)
+		}
+		return Page{}, err
+	}
+	f.store(key, page)
+	return page, nil
+}
+
+func (f *Fetcher) get(ctx context.Context, rawURL string) (Page, error) {
+	req, err := http.NewRequestWithContext(ctx, http.MethodGet, rawURL, nil)
+	if err != nil {
+		return Page{}, fmt.Errorf("recipes: build request: %w", err)
+	}
+	req.Header.Set("User-Agent", UserAgent)
+	req.Header.Set("Accept", "text/html,application/xhtml+xml")
+	resp, err := f.client.Do(req)
+	if err != nil {
+		return Page{}, err
+	}
+	defer resp.Body.Close()
+	final := resp.Request.URL.String()
+	if resp.StatusCode < 200 || resp.StatusCode > 299 {
+		return Page{}, &StatusError{URL: final, Status: resp.StatusCode}
+	}
+	if resp.ContentLength > MaxBodyBytes {
+		return Page{}, fmt.Errorf("%w: %s declares %d bytes", ErrTooLarge, final, resp.ContentLength)
+	}
+	body, err := io.ReadAll(io.LimitReader(resp.Body, MaxBodyBytes+1))
+	if err != nil {
+		return Page{}, fmt.Errorf("recipes: read %s: %w", final, err)
+	}
+	if len(body) > MaxBodyBytes {
+		return Page{}, fmt.Errorf("%w: %s is over %d bytes", ErrTooLarge, final, MaxBodyBytes)
+	}
+	return Page{URL: final, ContentType: resp.Header.Get("Content-Type"), Body: body}, nil
+}
+
+// cached returns the page cached under key if it is younger than CacheTTL.
+func (f *Fetcher) cached(key string) (Page, bool) {
+	f.mu.Lock()
+	defer f.mu.Unlock()
+	e, ok := f.cache[key]
+	if !ok {
+		return Page{}, false
+	}
+	if f.now().Sub(e.fetched) >= CacheTTL {
+		delete(f.cache, key)
+		return Page{}, false
+	}
+	page := e.page
+	page.Body = bytes.Clone(page.Body)
+	return page, true
+}
+
+// store caches a copy of page under key, evicting the oldest entry when
+// full.
+func (f *Fetcher) store(key string, page Page) {
+	page.Body = bytes.Clone(page.Body)
+	f.mu.Lock()
+	defer f.mu.Unlock()
+	if _, ok := f.cache[key]; !ok && len(f.cache) >= CacheSize {
+		var oldest string
+		var oldestAt time.Time
+		for k, e := range f.cache {
+			if oldest == "" || e.fetched.Before(oldestAt) {
+				oldest, oldestAt = k, e.fetched
+			}
+		}
+		delete(f.cache, oldest)
+	}
+	f.cache[key] = cacheEntry{page: page, fetched: f.now()}
+}
+
+// waitTurn blocks until a request to host may be sent, keeping requests to
+// one host at least HostInterval apart. A wait holds no reservation, and a
+// request ending with ctx before its turn takes no slot. The request let
+// through takes the host's slot and must be sent at once, or call release
+// if sending fails, so that it delays no other request. Fetch calls it from
+// turnConn's first Write, so the slot is taken after DNS, dialing and the
+// TLS handshake, as the request is written.
+func (f *Fetcher) waitTurn(ctx context.Context, host string) (release func(), err error) {
+	for {
+		if err := ctx.Err(); err != nil {
+			return nil, err
+		}
+		f.mu.Lock()
+		now := f.now()
+		last, sent := f.lastSent[host]
+		wait := last.Add(HostInterval).Sub(now)
+		if !sent || wait <= 0 {
+			f.lastSent[host] = now
+			f.mu.Unlock()
+			return func() { f.release(host, now, last, sent) }, nil
+		}
+		released := f.released
+		f.mu.Unlock()
+		if err := f.waitOrRelease(ctx, wait, released); err != nil {
+			return nil, err
+		}
+	}
+}
+
+// release hands back the slot taken at taken for a request that was not
+// sent, restoring the host's previous send time, and wakes waiting requests.
+func (f *Fetcher) release(host string, taken, prev time.Time, hadPrev bool) {
+	f.mu.Lock()
+	defer f.mu.Unlock()
+	if !f.lastSent[host].Equal(taken) {
+		// A later request already holds the slot.
+		return
+	}
+	if hadPrev {
+		f.lastSent[host] = prev
+	} else {
+		delete(f.lastSent, host)
+	}
+	close(f.released)
+	f.released = make(chan struct{})
+}
+
+// waitOrRelease sleeps for d, returning early without error when a slot is
+// handed back so the caller can look again.
+func (f *Fetcher) waitOrRelease(ctx context.Context, d time.Duration, released <-chan struct{}) error {
+	sleepCtx, cancel := context.WithCancel(ctx)
+	defer cancel()
+	go func() {
+		select {
+		case <-released:
+			cancel()
+		case <-sleepCtx.Done():
+		}
+	}()
+	err := f.sleep(sleepCtx, d)
+	select {
+	case <-released:
+		return nil
+	default:
+		return err
+	}
+}
+
+// dialPublic resolves address's host once, refuses unless every answer is
+// public, then dials the checked addresses in turn.
+func (f *Fetcher) dialPublic(ctx context.Context, network, address string) (net.Conn, error) {
+	host, port, err := net.SplitHostPort(address)
+	if err != nil {
+		return nil, fmt.Errorf("recipes: dial address %q: %w", address, err)
+	}
+	var addrs []netip.Addr
+	if ip, err := netip.ParseAddr(host); err == nil {
+		addrs = []netip.Addr{ip}
+	} else {
+		answers, err := f.resolver.LookupIPAddr(ctx, host)
+		if err != nil {
+			return nil, fmt.Errorf("recipes: resolve %s: %w", host, err)
+		}
+		for _, a := range answers {
+			ip, ok := netip.AddrFromSlice(a.IP)
+			if !ok {
+				return nil, fmt.Errorf("recipes: resolve %s: malformed address %v", host, a.IP)
+			}
+			addrs = append(addrs, ip)
+		}
+	}
+	if len(addrs) == 0 {
+		return nil, fmt.Errorf("recipes: resolve %s: no addresses", host)
+	}
+	for _, ip := range addrs {
+		if !IsPublic(ip) {
+			return nil, fmt.Errorf("%w: %s resolves to %s", ErrBlockedAddress, host, ip)
+		}
+	}
+	var errs []error
+	for _, ip := range addrs {
+		conn, err := f.dial(ctx, network, net.JoinHostPort(ip.Unmap().WithZone("").String(), port))
+		if err == nil {
+			return conn, nil
+		}
+		errs = append(errs, err)
+	}
+	return nil, errors.Join(errs...)
+}
+
+// requestCtxKey carries a request's own context into its dial. The
+// transport dials with a context that keeps the request's values but not
+// its cancellation; the connection and the rate-limit wait must end with
+// the request.
+type requestCtxKey struct{}
+
+// requestTransport hands each request's context, redirect hops included,
+// to dialTLS.
+type requestTransport struct {
+	next http.RoundTripper
+}
+
+func (t *requestTransport) RoundTrip(req *http.Request) (*http.Response, error) {
+	ctx := req.Context()
+	return t.next.RoundTrip(req.WithContext(context.WithValue(ctx, requestCtxKey{}, ctx)))
+}
+
+// dialTLS connects to address through dialPublic and completes the TLS
+// handshake, all under the request's context, and returns a turnConn whose
+// first write waits for the host's rate-limit slot. ALPN offers only
+// HTTP/1.1, so the connection carries this one request.
+func (f *Fetcher) dialTLS(dialCtx context.Context, network, address string) (net.Conn, error) {
+	ctx, ok := dialCtx.Value(requestCtxKey{}).(context.Context)
+	if !ok {
+		return nil, fmt.Errorf("recipes: dial %s outside a request", address)
+	}
+	host, _, err := net.SplitHostPort(address)
+	if err != nil {
+		return nil, fmt.Errorf("recipes: dial address %q: %w", address, err)
+	}
+	conn, err := f.dialPublic(ctx, network, address)
+	if err != nil {
+		return nil, err
+	}
+	cfg := &tls.Config{}
+	if f.tlsConfig != nil {
+		cfg = f.tlsConfig.Clone()
+	}
+	cfg.ServerName = host
+	cfg.NextProtos = []string{"http/1.1"}
+	tc := tls.Client(conn, cfg)
+	if err := tc.HandshakeContext(ctx); err != nil {
+		conn.Close()
+		return nil, fmt.Errorf("recipes: TLS handshake with %s: %w", host, err)
+	}
+	return &turnConn{Conn: tc, f: f, ctx: ctx, host: strings.ToLower(host)}, nil
+}
+
+// turnConn is a connection ready to carry one request. Its first Write —
+// the start of the request — waits for the host's rate-limit slot, so two
+// requests to one host are written at least HostInterval apart however
+// long each spent resolving, dialing and in the TLS handshake. A first
+// Write that sends nothing hands the slot back.
+type turnConn struct {
+	net.Conn
+	f    *Fetcher
+	ctx  context.Context
+	host string
+	// started is set by the first Write; the transport writes a request
+	// from one goroutine, so it needs no lock.
+	started bool
+}
+
+func (c *turnConn) Write(p []byte) (int, error) {
+	if c.started {
+		return c.Conn.Write(p)
+	}
+	release, err := c.f.waitTurn(c.ctx, c.host)
+	if err != nil {
+		return 0, err
+	}
+	c.started = true
+	n, err := c.Conn.Write(p)
+	if n == 0 && err != nil {
+		release()
+	}
+	return n, err
+}
+
+// checkURL refuses anything but an absolute https:// URL with a host.
+func checkURL(u *url.URL) error {
+	if u.Scheme != "https" {
+		return fmt.Errorf("%w: %q", ErrNotHTTPS, u.Redacted())
+	}
+	if u.Hostname() == "" {
+		return fmt.Errorf("recipes: URL %q has no host", u.Redacted())
+	}
+	return nil
+}
+
+// checkRedirect applies checkURL to every hop and stops after
+// MaxRedirects. The hop's address is checked when it is dialed.
+func checkRedirect(req *http.Request, via []*http.Request) error {
+	if len(via) > MaxRedirects {
+		return fmt.Errorf("%w: more than %d", ErrTooManyRedirects, MaxRedirects)
+	}
+	return checkURL(req.URL)
+}
+
+// blockedIPv4 are the IPv4 special-purpose ranges that are not globally
+// reachable (IANA IPv4 Special-Purpose Address Registry). IPv4-mapped IPv6
+// addresses are unmapped before the check, so these cover them too.
+var blockedIPv4 = []netip.Prefix{
+	netip.MustParsePrefix("0.0.0.0/8"),       // "this network"
+	netip.MustParsePrefix("10.0.0.0/8"),      // RFC 1918
+	netip.MustParsePrefix("100.64.0.0/10"),   // carrier-grade NAT
+	netip.MustParsePrefix("127.0.0.0/8"),     // loopback
+	netip.MustParsePrefix("169.254.0.0/16"),  // link-local
+	netip.MustParsePrefix("172.16.0.0/12"),   // RFC 1918
+	netip.MustParsePrefix("192.0.0.0/24"),    // IETF protocol assignments
+	netip.MustParsePrefix("192.0.2.0/24"),    // documentation, TEST-NET-1
+	netip.MustParsePrefix("192.88.99.0/24"),  // 6to4 relay anycast (deprecated)
+	netip.MustParsePrefix("192.168.0.0/16"),  // RFC 1918
+	netip.MustParsePrefix("198.18.0.0/15"),   // benchmarking
+	netip.MustParsePrefix("198.51.100.0/24"), // documentation, TEST-NET-2
+	netip.MustParsePrefix("203.0.113.0/24"),  // documentation, TEST-NET-3
+	netip.MustParsePrefix("224.0.0.0/4"),     // multicast
+	netip.MustParsePrefix("240.0.0.0/4"),     // reserved, broadcast
+}
+
+// globalIPv6 is the only IPv6 range dialed: global unicast. Everything
+// outside it is refused, including ::, ::1, IPv4-compatible and
+// IPv4-translated forms, NAT64 (64:ff9b::/96), discard-only (100::/64),
+// unique local (fc00::/7), link-local (fe80::/10) and multicast (ff00::/8).
+var globalIPv6 = netip.MustParsePrefix("2000::/3")
+
+// blockedIPv6 are the special-purpose ranges inside global unicast that are
+// not public or embed an arbitrary IPv4 address.
+var blockedIPv6 = []netip.Prefix{
+	netip.MustParsePrefix("2001::/23"),     // IETF protocol assignments, Teredo, benchmarking
+	netip.MustParsePrefix("2001:db8::/32"), // documentation
+	netip.MustParsePrefix("2002::/16"),     // 6to4, embeds any IPv4
+	netip.MustParsePrefix("3fff::/20"),     // documentation
+}
+
+// IsPublic reports whether ip is a public unicast address the Fetcher may
+// dial. IPv4 addresses are public unless in a special-purpose range; IPv6
+// addresses must be global unicast and outside its special-purpose ranges.
+func IsPublic(ip netip.Addr) bool {
+	if !ip.IsValid() {
+		return false
+	}
+	ip = ip.Unmap().WithZone("")
+	blocked := blockedIPv4
+	if ip.Is6() {
+		if !globalIPv6.Contains(ip) {
+			return false
+		}
+		blocked = blockedIPv6
+	}
+	for _, p := range blocked {
+		if p.Contains(ip) {
+			return false
+		}
+	}
+	return true
+}
+
+// sleep waits d or until ctx is done, returning ctx's error if it is.
+func sleep(ctx context.Context, d time.Duration) error {
+	t := time.NewTimer(d)
+	defer t.Stop()
+	select {
+	case <-t.C:
+		return nil
+	case <-ctx.Done():
+		return ctx.Err()
+	}
+}

internal/recipes/fetcher_test.go

@@ -0,0 +1,721 @@
+package recipes
+
+import (
+	"context"
+	"crypto/tls"
+	"crypto/x509"
+	"errors"
+	"fmt"
+	"net"
+	"net/http"
+	"net/http/httptest"
+	"net/netip"
+	"strings"
+	"sync"
+	"testing"
+	"time"
+)
+
+// stubResolver answers each host from a list of answers, one per lookup;
+// the last answer repeats.
+type stubResolver struct {
+	mu      sync.Mutex
+	answers map[string][][]string
+	calls   map[string]int
+}
+
+func (r *stubResolver) LookupIPAddr(_ context.Context, host string) ([]net.IPAddr, error) {
+	r.mu.Lock()
+	defer r.mu.Unlock()
+	seq, ok := r.answers[host]
+	if !ok {
+		return nil, fmt.Errorf("stub: no such host %s", host)
+	}
+	n := r.calls[host]
+	r.calls[host]++
+	if n >= len(seq) {
+		n = len(seq) - 1
+	}
+	var out []net.IPAddr
+	for _, s := range seq[n] {
+		out = append(out, net.IPAddr{IP: net.ParseIP(s)})
+	}
+	return out, nil
+}
+
+func (r *stubResolver) lookups(host string) int {
+	r.mu.Lock()
+	defer r.mu.Unlock()
+	return r.calls[host]
+}
+
+// testEnv is a Fetcher whose public test addresses all reach one httptest
+// TLS server, and which records every address it dials.
+type testEnv struct {
+	f        *Fetcher
+	resolver *stubResolver
+	server   *httptest.Server
+
+	mu     sync.Mutex
+	dialed []string
+}
+
+// publicIP, publicIP2 and publicIPv6 are public addresses; the test dialer
+// routes them to the test server and never reaches the real hosts.
+const (
+	publicIP   = "93.184.215.14"
+	publicIP2  = "93.184.215.20"
+	publicIPv6 = "2606:2800:21f::10"
+)
+
+func newTestEnv(t *testing.T, handler http.Handler, answers map[string][][]string) *testEnv {
+	t.Helper()
+	server := httptest.NewTLSServer(handler)
+	t.Cleanup(server.Close)
+	pool := x509.NewCertPool()
+	pool.AddCert(server.Certificate())
+	env := &testEnv{
+		resolver: &stubResolver{answers: answers, calls: map[string]int{}},
+		server:   server,
+	}
+	dial := func(ctx context.Context, network, address string) (net.Conn, error) {
+		env.mu.Lock()
+		env.dialed = append(env.dialed, address)
+		env.mu.Unlock()
+		host, _, err := net.SplitHostPort(address)
+		if err != nil {
+			return nil, err
+		}
+		if host == publicIP || host == publicIP2 || host == publicIPv6 {
+			var d net.Dialer
+			return d.DialContext(ctx, network, server.Listener.Addr().String())
+		}
+		return nil, fmt.Errorf("test dial: no route to %s", address)
+	}
+	env.f = newFetcher(env.resolver, dial, &tls.Config{RootCAs: pool})
+	// Real requests in these tests are not rate-limited unless a test
+	// installs a fake clock.
+	env.f.sleep = func(context.Context, time.Duration) error { return nil }
+	return env
+}
+
+func (e *testEnv) dials() []string {
+	e.mu.Lock()
+	defer e.mu.Unlock()
+	return append([]string(nil), e.dialed...)
+}
+
+func publicAnswers() map[string][][]string {
+	return map[string][][]string{"example.com": {{publicIP}}}
+}
+
+func TestFetchReturnsPage(t *testing.T) {
+	t.Parallel()
+	var gotUA string
+	env := newTestEnv(t, http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		gotUA = r.Header.Get("User-Agent")
+		w.Header().Set("Content-Type", "text/html")
+		fmt.Fprint(w, "<html>tacos</html>")
+	}), publicAnswers())
+
+	page, err := env.f.Fetch(t.Context(), "https://example.com/tacos")
+	if err != nil {
+		t.Fatalf("Fetch: %v", err)
+	}
+	if string(page.Body) != "<html>tacos</html>" || page.ContentType != "text/html" || page.URL != "https://example.com/tacos" {
+		t.Errorf("page = %+v", page)
+	}
+	if gotUA != UserAgent {
+		t.Errorf("User-Agent = %q, want %q", gotUA, UserAgent)
+	}
+	if got := env.dials(); len(got) != 1 || got[0] != publicIP+":443" {
+		t.Errorf("dialed %v, want [%s:443]", got, publicIP)
+	}
+}
+
+func TestFetchDialsPublicIPv6(t *testing.T) {
+	t.Parallel()
+	env := newTestEnv(t, http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		fmt.Fprint(w, "ok")
+	}), map[string][][]string{"example.com": {{publicIPv6}}})
+	if _, err := env.f.Fetch(t.Context(), "https://example.com/"); err != nil {
+		t.Fatalf("Fetch: %v", err)
+	}
+	if got := env.dials(); len(got) != 1 || got[0] != "["+publicIPv6+"]:443" {
+		t.Errorf("dialed %v, want [[%s]:443]", got, publicIPv6)
+	}
+}
+
+func TestFetchRejectsNonHTTPS(t *testing.T) {
+	t.Parallel()
+	env := newTestEnv(t, http.NotFoundHandler(), publicAnswers())
+	for _, raw := range []string{"http://example.com/tacos", "ftp://example.com/", "example.com/tacos", "https:///tacos"} {
+		_, err := env.f.Fetch(t.Context(), raw)
+		if err == nil {
+			t.Errorf("Fetch(%q) succeeded", raw)
+		}
+		if strings.HasPrefix(raw, "http://") && !errors.Is(err, ErrNotHTTPS) {
+			t.Errorf("Fetch(%q) error = %v, want ErrNotHTTPS", raw, err)
+		}
+	}
+	if got := env.dials(); len(got) != 0 {
+		t.Errorf("dialed %v, want nothing", got)
+	}
+	if n := env.resolver.lookups("example.com"); n != 0 {
+		t.Errorf("resolved example.com %d times, want 0", n)
+	}
+}
+
+// blockedIPs has an address in every refused range, including IPv4-mapped
+// forms of the IPv4 ranges.
+var blockedIPs = []string{
+	"127.0.0.1", "127.255.255.254", "::1", "::ffff:127.0.0.1", // loopback
+	"10.0.0.1", "172.16.0.1", "172.31.255.255", "192.168.1.1", // RFC 1918
+	"::ffff:10.1.2.3", "::ffff:172.20.0.1", "::ffff:192.168.0.10",
+	"169.254.169.254", "::ffff:169.254.169.254", "fe80::1", // link-local
+	"100.64.0.1", "100.127.255.254", "::ffff:100.100.0.1", // carrier-grade NAT
+	"224.0.0.1", "239.255.255.250", "ff02::1", "::ffff:224.0.0.251", // multicast
+	"0.0.0.0", "0.1.2.3", "::", "::ffff:0.0.0.0", // unspecified
+	"fc00::1", "fd12:3456:789a::1", // unique local
+	"255.255.255.255", "240.0.0.1", "fec0::1", // reserved, site-local
+	"192.0.0.1", "192.0.0.255", // IETF protocol assignments
+	"192.0.2.1", "198.51.100.7", "203.0.113.10", "::ffff:203.0.113.10", // documentation
+	"192.88.99.1",                                       // 6to4 relay anycast
+	"198.18.0.1", "198.19.255.255", "::ffff:198.18.0.1", // benchmarking
+	"2001:db8::1", "3fff::1", // IPv6 documentation
+	"2001:2::1", "100::1", // IPv6 benchmarking, discard-only
+	"::ffff:0:a00:1", // IPv4-translated
+	"64:ff9b::a00:1", "64:ff9b:1::a00:1", "2002:a00:1::1", "2001:0:a00:1::1", "::a00:1",
+}
+
+func TestIsPublic(t *testing.T) {
+	t.Parallel()
+	for _, s := range blockedIPs {
+		if IsPublic(netip.MustParseAddr(s)) {
+			t.Errorf("IsPublic(%s) = true, want false", s)
+		}
+	}
+	for _, s := range []string{
+		"8.8.8.8", publicIP, "::ffff:8.8.8.8", "100.63.255.255", "100.128.0.1", "172.32.0.1",
+		"192.0.1.255", "192.0.3.1", "192.88.98.255", "198.17.255.255", "198.20.0.1",
+		"198.51.99.255", "203.0.112.255", "203.0.114.1", "223.255.255.254",
+		"2606:4700::1111", publicIPv6, "2001:4860:4860::8888", "2a00:1450::1",
+	} {
+		if !IsPublic(netip.MustParseAddr(s)) {
+			t.Errorf("IsPublic(%s) = false, want true", s)
+		}
+	}
+	if IsPublic(netip.Addr{}) {
+		t.Error("IsPublic(zero Addr) = true, want false")
+	}
+}
+
+func TestFetchRefusesEachBlockedRange(t *testing.T) {
+	t.Parallel()
+	for _, ip := range blockedIPs {
+		t.Run(ip, func(t *testing.T) {
+			t.Parallel()
+			env := newTestEnv(t, http.NotFoundHandler(), map[string][][]string{"example.com": {{ip}}})
+			_, err := env.f.Fetch(t.Context(), "https://example.com/")
+			if !errors.Is(err, ErrBlockedAddress) {
+				t.Fatalf("Fetch error = %v, want ErrBlockedAddress", err)
+			}
+			if got := env.dials(); len(got) != 0 {
+				t.Errorf("dialed %v, want nothing", got)
+			}
+		})
+	}
+}
+
+func TestFetchRefusesBlockedLiteralHost(t *testing.T) {
+	t.Parallel()
+	env := newTestEnv(t, http.NotFoundHandler(), nil)
+	for _, raw := range []string{"https://127.0.0.1/", "https://[::1]/", "https://[::ffff:10.0.0.1]/", "https://169.254.169.254/latest/meta-data/", "https://[fe80::1%25eth0]/"} {
+		if _, err := env.f.Fetch(t.Context(), raw); !errors.Is(err, ErrBlockedAddress) {
+			t.Errorf("Fetch(%q) error = %v, want ErrBlockedAddress", raw, err)
+		}
+	}
+	if got := env.dials(); len(got) != 0 {
+		t.Errorf("dialed %v, want nothing", got)
+	}
+}
+
+func TestFetchRefusesMixedAnswer(t *testing.T) {
+	t.Parallel()
+	env := newTestEnv(t, http.NotFoundHandler(), map[string][][]string{"example.com": {{publicIP, "10.0.0.1"}}})
+	if _, err := env.f.Fetch(t.Context(), "https://example.com/"); !errors.Is(err, ErrBlockedAddress) {
+		t.Fatalf("Fetch error = %v, want ErrBlockedAddress", err)
+	}
+	if got := env.dials(); len(got) != 0 {
+		t.Errorf("dialed %v, want nothing", got)
+	}
+}
+
+func TestFetchRefusesRedirectToPrivateHost(t *testing.T) {
+	t.Parallel()
+	answers := map[string][][]string{
+		"example.com":      {{publicIP}},
+		"evil.example.com": {{"10.0.0.5"}},
+	}
+	for _, target := range []string{"https://evil.example.com/admin", "https://127.0.0.1/admin", "https://[::ffff:192.168.0.1]/"} {
+		t.Run(target, func(t *testing.T) {
+			t.Parallel()
+			env := newTestEnv(t, http.RedirectHandler(target, http.StatusFound), answers)
+			_, err := env.f.Fetch(t.Context(), "https://example.com/tacos")
+			if !errors.Is(err, ErrBlockedAddress) {
+				t.Fatalf("Fetch error = %v, want ErrBlockedAddress", err)
+			}
+			if got := env.dials(); len(got) != 1 || got[0] != publicIP+":443" {
+				t.Errorf("dialed %v, want only the first hop", got)
+			}
+		})
+	}
+}
+
+func TestFetchRefusesRedirectToHTTP(t *testing.T) {
+	t.Parallel()
+	env := newTestEnv(t, http.RedirectHandler("http://example.com/plain", http.StatusMovedPermanently), publicAnswers())
+	if _, err := env.f.Fetch(t.Context(), "https://example.com/"); !errors.Is(err, ErrNotHTTPS) {
+		t.Fatalf("Fetch error = %v, want ErrNotHTTPS", err)
+	}
+}
+
+// chainHandler redirects /hop/N to /hop/N-1 until /hop/0, which serves a
+// page.
+var chainHandler = http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+	var n int
+	if _, err := fmt.Sscanf(r.URL.Path, "/hop/%d", &n); err != nil {
+		http.NotFound(w, r)
+		return
+	}
+	if n == 0 {
+		fmt.Fprint(w, "arrived")
+		return
+	}
+	http.Redirect(w, r, fmt.Sprintf("https://recipes.example.com/hop/%d", n-1), http.StatusFound)
+})
+
+func TestFetchFollowsAtMostThreeRedirects(t *testing.T) {
+	t.Parallel()
+	answers := map[string][][]string{
+		"example.com":         {{publicIP}},
+		"recipes.example.com": {{publicIP2}},
+	}
+	env := newTestEnv(t, chainHandler, answers)
+	page, err := env.f.Fetch(t.Context(), "https://example.com/hop/3")
+	if err != nil {
+		t.Fatalf("Fetch with 3 redirects: %v", err)
+	}
+	if page.URL != "https://recipes.example.com/hop/0" || string(page.Body) != "arrived" {
+		t.Errorf("page = %+v", page)
+	}
+	// Every hop is dialed, so every hop's address is checked.
+	if got := env.dials(); len(got) != 4 || got[0] != publicIP+":443" || got[3] != publicIP2+":443" {
+		t.Errorf("dialed %v, want 4 checked hops", got)
+	}
+
+	if _, err := env.f.Fetch(t.Context(), "https://example.com/hop/4"); !errors.Is(err, ErrTooManyRedirects) {
+		t.Fatalf("Fetch with 4 redirects error = %v, want ErrTooManyRedirects", err)
+	}
+}
+
+// TestFetchRefusesDNSSwitchToPrivate covers DNS rebinding: the host's first
+// answer is public and every later one private. The first fetch dials the
+// address it checked; the next lookup's private answer is refused, and the
+// private address is never dialed.
+func TestFetchRefusesDNSSwitchToPrivate(t *testing.T) {
+	t.Parallel()
+	answers := map[string][][]string{"example.com": {{publicIP}, {"127.0.0.1"}}}
+	env := newTestEnv(t, http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		fmt.Fprint(w, "ok")
+	}), answers)
+
+	if _, err := env.f.Fetch(t.Context(), "https://example.com/one"); err != nil {
+		t.Fatalf("first Fetch: %v", err)
+	}
+	if n := env.resolver.lookups("example.com"); n != 1 {
+		t.Fatalf("first Fetch resolved %d times, want 1 (checked IP must be dialed IP)", n)
+	}
+	_, err := env.f.Fetch(t.Context(), "https://example.com/two")
+	if !errors.Is(err, ErrBlockedAddress) {
+		t.Fatalf("second Fetch error = %v, want ErrBlockedAddress", err)
+	}
+	if got := env.dials(); len(got) != 1 || got[0] != publicIP+":443" {
+		t.Errorf("dialed %v, want only %s:443", got, publicIP)
+	}
+}
+
+func TestFetchSizeCap(t *testing.T) {
+	t.Parallel()
+	env := newTestEnv(t, http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		size := MaxBodyBytes
+		if r.URL.Path != "/exact" {
+			size++
+		}
+		if r.URL.Path == "/declared" {
+			w.Header().Set("Content-Length", fmt.Sprint(size))
+		}
+		w.Write([]byte(strings.Repeat("a", size)))
+	}), publicAnswers())
+
+	page, err := env.f.Fetch(t.Context(), "https://example.com/exact")
+	if err != nil {
+		t.Fatalf("Fetch at the cap: %v", err)
+	}
+	if len(page.Body) != MaxBodyBytes {
+		t.Errorf("body is %d bytes, want %d", len(page.Body), MaxBodyBytes)
+	}
+	for _, path := range []string{"/declared", "/chunked"} {
+		if _, err := env.f.Fetch(t.Context(), "https://example.com"+path); !errors.Is(err, ErrTooLarge) {
+			t.Errorf("Fetch %s error = %v, want ErrTooLarge", path, err)
+		}
+	}
+}
+
+func TestFetchTimeout(t *testing.T) {
+	t.Parallel()
+	env := newTestEnv(t, http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		if r.URL.Path == "/slow-body" {
+			w.Write([]byte("start"))
+			w.(http.Flusher).Flush()
+		}
+		<-r.Context().Done()
+	}), publicAnswers())
+	env.f.timeout = 100 * time.Millisecond
+
+	for _, path := range []string{"/slow-headers", "/slow-body"} {
+		start := time.Now()
+		_, err := env.f.Fetch(t.Context(), "https://example.com"+path)
+		if !errors.Is(err, ErrTimeout) {
+			t.Errorf("Fetch %s error = %v, want ErrTimeout", path, err)
+		}
+		if elapsed := time.Since(start); elapsed > 5*time.Second {
+			t.Errorf("Fetch %s took %s", path, elapsed)
+		}
+	}
+	if Timeout != 10*time.Second {
+		t.Errorf("Timeout = %s, want 10s", Timeout)
+	}
+}
+
+func TestFetchCallerCancelIsNotTimeout(t *testing.T) {
+	t.Parallel()
+	env := newTestEnv(t, http.NotFoundHandler(), publicAnswers())
+	ctx, cancel := context.WithCancel(t.Context())
+	cancel()
+	_, err := env.f.Fetch(ctx, "https://example.com/")
+	if !errors.Is(err, context.Canceled) || errors.Is(err, ErrTimeout) {
+		t.Fatalf("Fetch error = %v, want context.Canceled", err)
+	}
+}
+
+// fakeClock is a clock whose sleep advances it and records the wait.
+type fakeClock struct {
+	mu     sync.Mutex
+	t      time.Time
+	sleeps []time.Duration
+}
+
+func (c *fakeClock) now() time.Time {
+	c.mu.Lock()
+	defer c.mu.Unlock()
+	return c.t
+}
+
+func (c *fakeClock) sleep(_ context.Context, d time.Duration) error {
+	c.mu.Lock()
+	defer c.mu.Unlock()
+	c.sleeps = append(c.sleeps, d)
+	c.t = c.t.Add(d)
+	return nil
+}
+
+func (c *fakeClock) advance(d time.Duration) {
+	c.mu.Lock()
+	defer c.mu.Unlock()
+	c.t = c.t.Add(d)
+}
+
+func (c *fakeClock) waits() []time.Duration {
+	c.mu.Lock()
+	defer c.mu.Unlock()
+	return append([]time.Duration(nil), c.sleeps...)
+}
+
+func installClock(f *Fetcher) *fakeClock {
+	c := &fakeClock{t: time.Date(2026, 10, 5, 18, 0, 0, 0, time.UTC)}
+	f.now = c.now
+	f.sleep = c.sleep
+	return c
+}
+
+func TestFetchCachesByURL(t *testing.T) {
+	t.Parallel()
+	var mu sync.Mutex
+	hits := map[string]int{}
+	env := newTestEnv(t, http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		mu.Lock()
+		hits[r.URL.Path]++
+		n := hits[r.URL.Path]
+		mu.Unlock()
+		if r.URL.Path == "/missing" {
+			http.NotFound(w, r)
+			return
+		}
+		fmt.Fprintf(w, "%s #%d", r.URL.Path, n)
+	}), publicAnswers())
+	clock := installClock(env.f)
+
+	fetch := func(path string) string {
+		t.Helper()
+		page, err := env.f.Fetch(t.Context(), "https://example.com"+path)
+		if err != nil {
+			t.Fatalf("Fetch %s: %v", path, err)
+		}
+		return string(page.Body)
+	}
+	if got := fetch("/a"); got != "/a #1" {
+		t.Errorf("first /a = %q", got)
+	}
+	if got := fetch("/a"); got != "/a #1" {
+		t.Errorf("cached /a = %q, want the first response", got)
+	}
+	if got := fetch("/b"); got != "/b #1" {
+		t.Errorf("/b = %q, want its own response", got)
+	}
+	clock.advance(CacheTTL)
+	if got := fetch("/a"); got != "/a #2" {
+		t.Errorf("/a after CacheTTL = %q, want a refetch", got)
+	}
+
+	for range 2 {
+		var se *StatusError
+		if _, err := env.f.Fetch(t.Context(), "https://example.com/missing"); !errors.As(err, &se) || se.Status != http.StatusNotFound {
+			t.Fatalf("Fetch /missing error = %v, want StatusError 404", err)
+		}
+	}
+	mu.Lock()
+	defer mu.Unlock()
+	if hits["/missing"] != 2 {
+		t.Errorf("/missing was requested %d times, want 2 (errors are not cached)", hits["/missing"])
+	}
+}
+
+func TestCacheEvictsOldest(t *testing.T) {
+	t.Parallel()
+	f := newFetcher(&stubResolver{}, nil, nil)
+	clock := installClock(f)
+	for i := range CacheSize + 1 {
+		f.store(fmt.Sprintf("https://example.com/%d", i), Page{})
+		clock.advance(time.Second)
+	}
+	if len(f.cache) != CacheSize {
+		t.Errorf("cache holds %d pages, want %d", len(f.cache), CacheSize)
+	}
+	if _, ok := f.cached("https://example.com/0"); ok {
+		t.Error("oldest page was not evicted")
+	}
+	if _, ok := f.cached(fmt.Sprintf("https://example.com/%d", CacheSize)); !ok {
+		t.Error("newest page is missing")
+	}
+}
+
+func TestFetchRateLimitsPerHost(t *testing.T) {
+	t.Parallel()
+	answers := map[string][][]string{
+		"example.com":         {{publicIP}},
+		"recipes.example.com": {{publicIP2}},
+	}
+	env := newTestEnv(t, http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		fmt.Fprint(w, "ok")
+	}), answers)
+	clock := installClock(env.f)
+
+	for _, raw := range []string{"https://example.com/1", "https://recipes.example.com/1", "https://example.com/2", "https://example.com/3"} {
+		if _, err := env.f.Fetch(t.Context(), raw); err != nil {
+			t.Fatalf("Fetch %s: %v", raw, err)
+		}
+	}
+	// example.com/2 waits a full interval; example.com/3 waits another.
+	// recipes.example.com has its own slot and never waits.
+	got := clock.waits()
+	if len(got) != 2 || got[0] != HostInterval || got[1] != HostInterval {
+		t.Errorf("waits = %v, want [%s %s]", got, HostInterval, HostInterval)
+	}
+}
+
+func TestFetchCacheIsNotSharedWithCallers(t *testing.T) {
+	t.Parallel()
+	env := newTestEnv(t, http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		fmt.Fprint(w, "tacos")
+	}), publicAnswers())
+
+	for range 2 {
+		page, err := env.f.Fetch(t.Context(), "https://example.com/tacos")
+		if err != nil {
+			t.Fatalf("Fetch: %v", err)
+		}
+		if string(page.Body) != "tacos" {
+			t.Fatalf("body = %q, want the cached page unchanged", page.Body)
+		}
+		// Overwrite the returned body: neither the fetched nor the cached
+		// copy may reach the cache.
+		copy(page.Body, "XXXXX")
+	}
+}
+
+// TestFetchCanceledWaitHoldsNoSlot covers a burst of fetches whose
+// rate-limit waits end with their context: they must not push back the
+// next request that is allowed through.
+func TestFetchCanceledWaitHoldsNoSlot(t *testing.T) {
+	t.Parallel()
+	env := newTestEnv(t, http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		fmt.Fprint(w, "ok")
+	}), publicAnswers())
+	clock := installClock(env.f)
+
+	if _, err := env.f.Fetch(t.Context(), "https://example.com/first"); err != nil {
+		t.Fatalf("first Fetch: %v", err)
+	}
+	env.f.sleep = func(context.Context, time.Duration) error { return context.DeadlineExceeded }
+	for i := range 5 {
+		if _, err := env.f.Fetch(t.Context(), fmt.Sprintf("https://example.com/burst/%d", i)); !errors.Is(err, context.DeadlineExceeded) {
+			t.Fatalf("burst Fetch %d error = %v, want context.DeadlineExceeded", i, err)
+		}
+	}
+	env.f.sleep = clock.sleep
+	clock.advance(HostInterval)
+	if _, err := env.f.Fetch(t.Context(), "https://example.com/after"); err != nil {
+		t.Fatalf("Fetch after the burst: %v", err)
+	}
+	if got := clock.waits(); len(got) != 0 {
+		t.Errorf("waits = %v, want none: canceled waits held slots", got)
+	}
+}
+
+// TestFetchUnsentRequestHoldsNoSlot covers requests to a host with no
+// previous request that are never sent — canceled before Fetch, or failing
+// to connect — and checks that the next request is not delayed.
+func TestFetchUnsentRequestHoldsNoSlot(t *testing.T) {
+	t.Parallel()
+	const unroutableIP = "93.184.215.99"
+	answers := map[string][][]string{"example.com": {{unroutableIP}, {publicIP}}}
+	env := newTestEnv(t, http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		fmt.Fprint(w, "ok")
+	}), answers)
+	clock := installClock(env.f)
+
+	ctx, cancel := context.WithCancel(t.Context())
+	cancel()
+	if _, err := env.f.Fetch(ctx, "https://example.com/canceled"); !errors.Is(err, context.Canceled) {
+		t.Fatalf("canceled Fetch error = %v, want context.Canceled", err)
+	}
+	if _, err := env.f.Fetch(t.Context(), "https://example.com/unroutable"); err == nil || !strings.Contains(err.Error(), "no route") {
+		t.Fatalf("unroutable Fetch error = %v, want a dial failure", err)
+	}
+	if _, err := env.f.Fetch(t.Context(), "https://example.com/next"); err != nil {
+		t.Fatalf("next Fetch: %v", err)
+	}
+	if got := clock.waits(); len(got) != 0 {
+		t.Errorf("waits = %v, want none: unsent requests held slots", got)
+	}
+	if got := env.dials(); len(got) != 2 {
+		t.Errorf("dials = %v, want the unroutable and the next request only", got)
+	}
+}
+
+// TestReleaseWakesWaiter checks that a request waiting behind a slot that
+// is handed back goes ahead at once instead of sleeping out the interval.
+func TestReleaseWakesWaiter(t *testing.T) {
+	t.Parallel()
+	f := newFetcher(&stubResolver{}, nil, nil)
+	release, err := f.waitTurn(t.Context(), "example.com")
+	if err != nil {
+		t.Fatalf("first waitTurn: %v", err)
+	}
+	done := make(chan error, 1)
+	start := time.Now()
+	go func() {
+		_, err := f.waitTurn(t.Context(), "example.com")
+		done <- err
+	}()
+	time.Sleep(50 * time.Millisecond)
+	release()
+	select {
+	case err := <-done:
+		if err != nil {
+			t.Fatalf("waiting waitTurn: %v", err)
+		}
+		if waited := time.Since(start); waited >= HostInterval {
+			t.Errorf("waiter took %s, want less than %s", waited, HostInterval)
+		}
+	case <-time.After(HostInterval / 2):
+		t.Fatal("waiter still waiting after the slot was handed back")
+	}
+}
+
+// stallingResolver holds its first lookup until unstall is closed, then
+// answers from the stub.
+type stallingResolver struct {
+	*stubResolver
+	once    sync.Once
+	stalled chan struct{}
+	unstall chan struct{}
+}
+
+func (r *stallingResolver) LookupIPAddr(ctx context.Context, host string) ([]net.IPAddr, error) {
+	first := false
+	r.once.Do(func() { first = true })
+	if first {
+		close(r.stalled)
+		<-r.unstall
+	}
+	return r.stubResolver.LookupIPAddr(ctx, host)
+}
+
+// TestFetchRateLimitsSendsNotSetup covers a request that stalls in DNS
+// while a later request to the same host is sent: the stalled request must
+// still be sent a full HostInterval after the other one, because the slot
+// is taken when the request is written, not when it starts.
+func TestFetchRateLimitsSendsNotSetup(t *testing.T) {
+	t.Parallel()
+	var clock *fakeClock
+	var mu sync.Mutex
+	sentAt := map[string]time.Time{}
+	env := newTestEnv(t, http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		mu.Lock()
+		sentAt[r.URL.Path] = clock.now()
+		mu.Unlock()
+		fmt.Fprint(w, "ok")
+	}), publicAnswers())
+	resolver := &stallingResolver{
+		stubResolver: env.resolver,
+		stalled:      make(chan struct{}),
+		unstall:      make(chan struct{}),
+	}
+	env.f.resolver = resolver
+	clock = installClock(env.f)
+
+	slow := make(chan error, 1)
+	go func() {
+		_, err := env.f.Fetch(t.Context(), "https://example.com/slow")
+		slow <- err
+	}()
+	<-resolver.stalled
+	if _, err := env.f.Fetch(t.Context(), "https://example.com/fast"); err != nil {
+		t.Fatalf("fast Fetch: %v", err)
+	}
+	close(resolver.unstall)
+	if err := <-slow; err != nil {
+		t.Fatalf("slow Fetch: %v", err)
+	}
+
+	mu.Lock()
+	defer mu.Unlock()
+	if gap := sentAt["/slow"].Sub(sentAt["/fast"]); gap < HostInterval {
+		t.Errorf("slow request sent %s after fast, want at least %s", gap, HostInterval)
+	}
+	if got := clock.waits(); len(got) != 1 || got[0] != HostInterval {
+		t.Errorf("waits = %v, want [%s]: only the stalled request waits", got, HostInterval)
+	}
+}

internal/recipes/rank.go

@@ -0,0 +1,130 @@
+package recipes
+
+import (
+	"context"
+	"errors"
+	"fmt"
+	"sort"
+	"sync"
+
+	"code.cubecraftlabs.com/Maniacs_Mansion/Meal-Quest/internal/ai/harness"
+)
+
+// RankDepth is how many of the top search candidates Rank fetches and
+// extracts.
+const RankDepth = 5
+
+var (
+	// ErrNoIngredients rejects a candidate whose recipe lists no
+	// ingredients.
+	ErrNoIngredients = errors.New("recipes: recipe has no ingredients")
+	// ErrNoImage rejects a candidate whose recipe has no image.
+	ErrNoImage = errors.New("recipes: recipe has no image")
+	// ErrNoUsableRecipe is returned when Rank ranks no candidate; it is
+	// joined with each candidate's reason.
+	ErrNoUsableRecipe = errors.New("recipes: no candidate holds a usable recipe")
+	// ErrCandidatesUnranked is returned, alongside the ranked candidates,
+	// when Rank ranks some candidates but not all; it is joined with each
+	// unranked candidate's reason.
+	ErrCandidatesUnranked = errors.New("recipes: some candidates were not ranked")
+)
+
+// PageFetcher fetches a candidate's page; *Fetcher satisfies it.
+type PageFetcher interface {
+	Fetch(ctx context.Context, rawURL string) (Page, error)
+}
+
+// Ranked is a candidate whose page held a usable recipe, with its score.
+type Ranked struct {
+	Candidate Candidate
+	Recipe    Recipe
+	// Score is the token similarity of the recipe's title to the dish name
+	// (0 to 1, harness.Similarity) plus a quarter for each of ingredients,
+	// image, steps and total minutes present (0 to 1).
+	Score float64
+}
+
+// Rank fetches and extracts the first RankDepth candidates concurrently and
+// returns those whose recipe has ingredients and an image, highest Score
+// first; equal scores keep search order.
+//
+// Every candidate that is not ranked — its fetch failed, its page holds no
+// valid recipe, or the recipe lacks ingredients or an image — is reported,
+// never dropped silently: when some are ranked the error is
+// ErrCandidatesUnranked, returned together with the ranked ones; when none
+// are it is ErrNoUsableRecipe with no results. Either is joined with each
+// unranked candidate's URL and reason. A cancelled ctx returns only its
+// error.
+func Rank(ctx context.Context, f PageFetcher, dish string, cands []Candidate) ([]Ranked, error) {
+	if len(cands) > RankDepth {
+		cands = cands[:RankDepth]
+	}
+	type outcome struct {
+		ranked Ranked
+		err    error
+	}
+	outcomes := make([]outcome, len(cands))
+	var wg sync.WaitGroup
+	for i, c := range cands {
+		wg.Add(1)
+		go func() {
+			defer wg.Done()
+			r, err := rankOne(ctx, f, dish, c)
+			outcomes[i] = outcome{r, err}
+		}()
+	}
+	wg.Wait()
+	if err := ctx.Err(); err != nil {
+		return nil, err
+	}
+
+	var ranked []Ranked
+	var reasons []error
+	for i, o := range outcomes {
+		if o.err != nil {
+			reasons = append(reasons, fmt.Errorf("%s: %w", cands[i].URL, o.err))
+			continue
+		}
+		ranked = append(ranked, o.ranked)
+	}
+	if len(ranked) == 0 {
+		return nil, errors.Join(append([]error{ErrNoUsableRecipe}, reasons...)...)
+	}
+	sort.SliceStable(ranked, func(a, b int) bool { return ranked[a].Score > ranked[b].Score })
+	if len(reasons) > 0 {
+		return ranked, errors.Join(append([]error{ErrCandidatesUnranked}, reasons...)...)
+	}
+	return ranked, nil
+}
+
+// rankOne fetches, extracts and scores one candidate, or says why it is
+// rejected.
+func rankOne(ctx context.Context, f PageFetcher, dish string, c Candidate) (Ranked, error) {
+	page, err := f.Fetch(ctx, c.URL)
+	if err != nil {
+		return Ranked{}, err
+	}
+	r, err := Extract(page.Body, page.URL)
+	if err != nil {
+		return Ranked{}, err
+	}
+	if len(r.Ingredients) == 0 {
+		return Ranked{}, ErrNoIngredients
+	}
+	if r.ImageURL == "" {
+		return Ranked{}, ErrNoImage
+	}
+	return Ranked{Candidate: c, Recipe: r, Score: harness.Similarity(r.Title, dish) + completeness(r)}, nil
+}
+
+// completeness is a quarter for each of ingredients, image, steps and total
+// minutes the recipe has.
+func completeness(r Recipe) float64 {
+	score := 0.0
+	for _, present := range []bool{len(r.Ingredients) > 0, r.ImageURL != "", len(r.Steps) > 0, r.TotalMinutes != nil} {
+		if present {
+			score += 0.25
+		}
+	}
+	return score
+}

internal/recipes/rank_test.go

@@ -0,0 +1,255 @@
+package recipes
+
+import (
+	"context"
+	"errors"
+	"fmt"
+	"net/http"
+	"strings"
+	"sync"
+	"testing"
+)
+
+var _ PageFetcher = (*Fetcher)(nil)
+
+// fakeFetcher serves fixture pages by URL and records what was fetched.
+type fakeFetcher struct {
+	pages map[string][]byte
+
+	mu      sync.Mutex
+	fetched []string
+}
+
+func (f *fakeFetcher) Fetch(_ context.Context, rawURL string) (Page, error) {
+	f.mu.Lock()
+	f.fetched = append(f.fetched, rawURL)
+	f.mu.Unlock()
+	body, ok := f.pages[rawURL]
+	if !ok {
+		return Page{}, &StatusError{URL: rawURL, Status: 404}
+	}
+	return Page{URL: rawURL, ContentType: "text/html", Body: body}, nil
+}
+
+// recipePage returns a page whose Recipe has the given name and, as asked,
+// ingredients, an image, steps and a total time.
+func recipePage(name string, ingredients, image, steps, minutes bool) []byte {
+	ld := fmt.Sprintf(`{"@context":"https://schema.org","@type":"Recipe","name":%q`, name)
+	if ingredients {
+		ld += `,"recipeIngredient":["1 onion","2 cloves garlic"]`
+	}
+	if image {
+		ld += `,"image":"https://img.example.com/dish.jpg"`
+	}
+	if steps {
+		ld += `,"recipeInstructions":["Chop.","Cook."]`
+	}
+	if minutes {
+		ld += `,"totalTime":"PT40M"`
+	}
+	return page("", ld+"}")
+}
+
+func TestRankOrder(t *testing.T) {
+	f := &fakeFetcher{pages: map[string][]byte{
+		"https://a.example/thai":    recipePage("Thai Green Curry", true, true, true, true),
+		"https://b.example/noimg":   recipePage("Chicken Curry", true, false, true, true),
+		"https://c.example/easy":    recipePage("Easy Chicken Curry", true, true, true, false),
+		"https://d.example/noingr":  recipePage("Chicken Curry", false, true, true, true),
+		"https://e.example/best":    recipePage("Chicken Curry", true, true, true, true),
+		"https://f.example/toodeep": recipePage("Chicken Curry", true, true, true, true),
+	}}
+	cands := []Candidate{
+		{URL: "https://a.example/thai", Title: "Thai green curry"},
+		{URL: "https://b.example/noimg"},
+		{URL: "https://c.example/easy"},
+		{URL: "https://d.example/noingr"},
+		{URL: "https://e.example/best"},
+		{URL: "https://f.example/toodeep"},
+	}
+	got, err := Rank(context.Background(), f, "Chicken curry", cands)
+	if !errors.Is(err, ErrCandidatesUnranked) || !errors.Is(err, ErrNoImage) || !errors.Is(err, ErrNoIngredients) {
+		t.Errorf("err = %v, want ErrCandidatesUnranked with ErrNoImage and ErrNoIngredients", err)
+	}
+	if errors.Is(err, ErrNoUsableRecipe) {
+		t.Errorf("err = %v, want not ErrNoUsableRecipe when some are ranked", err)
+	}
+	want := []struct {
+		url   string
+		score float64
+	}{
+		// Same words, every field: 1 + 1.
+		{"https://e.example/best", 2},
+		// "Easy" is a filler word, so the title matches; no total time.
+		{"https://c.example/easy", 1.75},
+		// Shares one of four words: 0.25 + 1.
+		{"https://a.example/thai", 1.25},
+	}
+	if len(got) != len(want) {
+		t.Fatalf("got %d ranked, want %d: %+v", len(got), len(want), got)
+	}
+	for i, w := range want {
+		if got[i].Candidate.URL != w.url || got[i].Score != w.score {
+			t.Errorf("rank %d = %s (%.2f), want %s (%.2f)", i, got[i].Candidate.URL, got[i].Score, w.url, w.score)
+		}
+	}
+	if got[2].Recipe.Title != "Thai Green Curry" || got[2].Candidate.Title != "Thai green curry" {
+		t.Errorf("ranked entry lost its recipe or candidate: %+v", got[2])
+	}
+	for _, u := range f.fetched {
+		if u == "https://f.example/toodeep" {
+			t.Errorf("fetched a candidate past RankDepth")
+		}
+	}
+	if len(f.fetched) != RankDepth {
+		t.Errorf("fetched %d pages, want %d", len(f.fetched), RankDepth)
+	}
+}
+
+func TestRankTiesKeepSearchOrder(t *testing.T) {
+	f := &fakeFetcher{pages: map[string][]byte{
+		"https://a.example/1": recipePage("Beef Stew", true, true, true, true),
+		"https://b.example/2": recipePage("Beef Stew", true, true, true, true),
+		"https://c.example/3": recipePage("Beef Stew", true, true, true, true),
+	}}
+	cands := []Candidate{{URL: "https://a.example/1"}, {URL: "https://b.example/2"}, {URL: "https://c.example/3"}}
+	got, err := Rank(context.Background(), f, "beef stew", cands)
+	if err != nil {
+		t.Fatal(err)
+	}
+	for i, c := range cands {
+		if got[i].Candidate.URL != c.URL {
+			t.Errorf("rank %d = %s, want %s", i, got[i].Candidate.URL, c.URL)
+		}
+	}
+}
+
+// TestRankReportsFetchFailure checks a failed fetch is reported with the
+// candidates that were ranked, never swallowed.
+func TestRankReportsFetchFailure(t *testing.T) {
+	f := &fakeFetcher{pages: map[string][]byte{
+		"https://a.example/1": recipePage("Beef Stew", true, true, true, true),
+	}}
+	cands := []Candidate{{URL: "https://a.example/1"}, {URL: "https://b.example/missing"}}
+	got, err := Rank(context.Background(), f, "beef stew", cands)
+	if len(got) != 1 || got[0].Candidate.URL != "https://a.example/1" {
+		t.Errorf("ranked = %+v, want only https://a.example/1", got)
+	}
+	if !errors.Is(err, ErrCandidatesUnranked) {
+		t.Errorf("err = %v, want ErrCandidatesUnranked", err)
+	}
+	var se *StatusError
+	if !errors.As(err, &se) || se.Status != 404 || se.URL != "https://b.example/missing" {
+		t.Errorf("err = %v, want the 404 StatusError for https://b.example/missing", err)
+	}
+	if err != nil && !strings.Contains(err.Error(), "https://b.example/missing") {
+		t.Errorf("err = %v, want it to name the failed candidate", err)
+	}
+}
+
+func TestRankAllRejected(t *testing.T) {
+	f := &fakeFetcher{pages: map[string][]byte{
+		"https://a.example/noimg":  recipePage("Chicken Curry", true, false, true, true),
+		"https://b.example/noingr": recipePage("Chicken Curry", false, true, true, true),
+		"https://c.example/blog":   page(""),
+	}}
+	cands := []Candidate{
+		{URL: "https://a.example/noimg"},
+		{URL: "https://b.example/noingr"},
+		{URL: "https://c.example/blog"},
+		{URL: "https://d.example/missing"},
+	}
+	_, err := Rank(context.Background(), f, "chicken curry", cands)
+	for _, want := range []error{ErrNoUsableRecipe, ErrNoImage, ErrNoIngredients, ErrNoRecipe} {
+		if !errors.Is(err, want) {
+			t.Errorf("err = %v, want it to wrap %v", err, want)
+		}
+	}
+	var se *StatusError
+	if !errors.As(err, &se) || se.Status != 404 {
+		t.Errorf("err = %v, want the 404 StatusError", err)
+	}
+}
+
+func TestRankNoCandidates(t *testing.T) {
+	if _, err := Rank(context.Background(), &fakeFetcher{}, "chicken curry", nil); !errors.Is(err, ErrNoUsableRecipe) {
+		t.Errorf("err = %v, want ErrNoUsableRecipe", err)
+	}
+}
+
+func TestRankContextCancelled(t *testing.T) {
+	f := &fakeFetcher{pages: map[string][]byte{"https://a.example/1": recipePage("Beef Stew", true, true, true, true)}}
+	ctx, cancel := context.WithCancel(context.Background())
+	cancel()
+	if _, err := Rank(ctx, f, "beef stew", []Candidate{{URL: "https://a.example/1"}}); !errors.Is(err, context.Canceled) {
+		t.Errorf("err = %v, want context.Canceled", err)
+	}
+}
+
+// TestRankOverHTTP ranks fixture candidates served over HTTPS through the
+// SSRF-safe Fetcher: every host, a subdomain the test server's certificate
+// covers, resolves to a public test address the test
+// dialer routes to one httptest server, which serves pages by host and path.
+func TestRankOverHTTP(t *testing.T) {
+	t.Parallel()
+	pages := map[string][]byte{
+		"best.example.com/curry":  recipePage("Chicken Curry", true, true, true, true),
+		"easy.example.com/curry":  recipePage("Easy Chicken Curry", true, true, true, false),
+		"thai.example.com/curry":  recipePage("Thai Green Curry", true, true, true, true),
+		"noimg.example.com/curry": recipePage("Chicken Curry", true, false, true, true),
+	}
+	env := newTestEnv(t, http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		body, ok := pages[r.Host+r.URL.Path]
+		if !ok {
+			http.Error(w, "broken", http.StatusInternalServerError)
+			return
+		}
+		w.Header().Set("Content-Type", "text/html")
+		w.Write(body)
+	}), map[string][][]string{
+		"best.example.com":   {{publicIP}},
+		"easy.example.com":   {{publicIP}},
+		"thai.example.com":   {{publicIP}},
+		"noimg.example.com":  {{publicIP}},
+		"broken.example.com": {{publicIP}},
+	})
+	cands := []Candidate{
+		{URL: "https://thai.example.com/curry"},
+		{URL: "https://broken.example.com/curry"},
+		{URL: "https://easy.example.com/curry"},
+		{URL: "https://noimg.example.com/curry"},
+		{URL: "https://best.example.com/curry"},
+	}
+
+	got, err := Rank(t.Context(), env.f, "Chicken curry", cands)
+
+	want := []struct {
+		url   string
+		score float64
+	}{
+		{"https://best.example.com/curry", 2},
+		{"https://easy.example.com/curry", 1.75},
+		{"https://thai.example.com/curry", 1.25},
+	}
+	if len(got) != len(want) {
+		t.Fatalf("got %d ranked, want %d: %+v (err %v)", len(got), len(want), got, err)
+	}
+	for i, w := range want {
+		if got[i].Candidate.URL != w.url || got[i].Score != w.score {
+			t.Errorf("rank %d = %s (%.2f), want %s (%.2f)", i, got[i].Candidate.URL, got[i].Score, w.url, w.score)
+		}
+	}
+	if got[0].Recipe.Title != "Chicken Curry" || got[0].Recipe.ImageURL != "https://img.example.com/dish.jpg" {
+		t.Errorf("top recipe = %+v", got[0].Recipe)
+	}
+	if !errors.Is(err, ErrCandidatesUnranked) || !errors.Is(err, ErrNoImage) {
+		t.Errorf("err = %v, want ErrCandidatesUnranked with ErrNoImage", err)
+	}
+	var se *StatusError
+	if !errors.As(err, &se) || se.Status != http.StatusInternalServerError {
+		t.Errorf("err = %v, want the 500 StatusError for broken.example.com", err)
+	}
+	if n := len(env.dials()); n != len(cands) {
+		t.Errorf("dialed %d times, want %d", n, len(cands))
+	}
+}

internal/recipes/search.go

@@ -0,0 +1,314 @@
+package recipes
+
+import (
+	"context"
+	"encoding/json"
+	"errors"
+	"fmt"
+	"io"
+	"net/http"
+	"net/url"
+	"strconv"
+	"strings"
+	"time"
+
+	"code.cubecraftlabs.com/Maniacs_Mansion/Meal-Quest/internal/db"
+	"code.cubecraftlabs.com/Maniacs_Mansion/Meal-Quest/internal/secretbox"
+)
+
+const (
+	// MaxSearchResults is the most candidates one search may ask for; it is
+	// Brave's largest page.
+	MaxSearchResults = 20
+	// SearchTimeout bounds one search request, reading the body included.
+	SearchTimeout = 10 * time.Second
+	// MaxSearchBodyBytes caps a search response body. A larger one is an
+	// error, never truncated.
+	MaxSearchBodyBytes = 1 << 20
+	// BraveEndpoint is the Brave Web Search API.
+	BraveEndpoint = "https://api.search.brave.com/res/v1/web/search"
+)
+
+var (
+	// ErrSearchRefused is returned for HTTP 401 or 403: Brave rejected the
+	// API key, or SearXNG does not allow format=json.
+	ErrSearchRefused = errors.New("recipes: search provider refused the request")
+	// ErrSearchRateLimited is returned for HTTP 429.
+	ErrSearchRateLimited = errors.New("recipes: search provider rate limit reached")
+	// ErrSearchResponse is returned for a response body that is not the
+	// provider's JSON shape — SearXNG without a results array, Brave without
+	// "type": "search" or with a web object lacking a results array — is
+	// over MaxSearchBodyBytes, or holds a result without an absolute http(s)
+	// URL.
+	ErrSearchResponse = errors.New("recipes: malformed search response")
+)
+
+// SearchStatusError is returned for any other non-2xx search response,
+// redirects included: a search endpoint is called directly and never
+// followed elsewhere.
+type SearchStatusError struct {
+	Provider string
+	Status   int
+}
+
+func (e *SearchStatusError) Error() string {
+	return fmt.Sprintf("recipes: %s search: HTTP %d", e.Provider, e.Status)
+}
+
+// Candidate is one search result: a page that may hold a recipe.
+type Candidate struct {
+	URL     string
+	Title   string
+	Snippet string
+}
+
+// SearchProvider finds recipe pages on the web.
+type SearchProvider interface {
+	// Search returns up to n candidates for query, best first. The query
+	// gets " recipe" appended unless it already has the word. n must be
+	// 1 to MaxSearchResults.
+	Search(ctx context.Context, query string, n int) ([]Candidate, error)
+}
+
+// NewSearchProvider returns the provider s names. SearXNG is called at
+// searxngURL, the configured MEALQUEST_SEARXNG_URL; Brave is called with the
+// API key in s.ApiKeyCt, decrypted by box.
+func NewSearchProvider(s db.RecipeSearchSetting, box *secretbox.Box, searxngURL string) (SearchProvider, error) {
+	switch s.Provider {
+	case "searxng":
+		return NewSearXNG(searxngURL)
+	case "brave":
+		if len(s.ApiKeyCt) == 0 {
+			return nil, errors.New("recipes: Brave search has no API key")
+		}
+		key, err := box.Decrypt(s.ApiKeyCt)
+		if err != nil {
+			return nil, fmt.Errorf("recipes: Brave API key: %w", err)
+		}
+		return NewBrave(string(key))
+	default:
+		return nil, fmt.Errorf("recipes: unknown search provider %q", s.Provider)
+	}
+}
+
+// SearXNG searches through a SearXNG instance. Its base URL is a configured
+// endpoint and may be on a private network, so it is called directly rather
+// than through the public-only Fetcher.
+type SearXNG struct {
+	endpoint *url.URL
+	client   *http.Client
+}
+
+// NewSearXNG returns a SearXNG provider for the instance at baseURL, an
+// absolute http:// or https:// URL without credentials, query or fragment.
+func NewSearXNG(baseURL string) (*SearXNG, error) {
+	if baseURL == "" {
+		return nil, errors.New("recipes: SearXNG URL is not set")
+	}
+	u, err := url.Parse(baseURL)
+	if err != nil || (u.Scheme != "http" && u.Scheme != "https") || u.Host == "" {
+		return nil, errors.New("recipes: SearXNG URL must be an absolute http:// or https:// URL")
+	}
+	if u.User != nil || u.RawQuery != "" || u.Fragment != "" {
+		return nil, errors.New("recipes: SearXNG URL must not have credentials, a query or a fragment")
+	}
+	return &SearXNG{endpoint: u.JoinPath("search"), client: newSearchClient()}, nil
+}
+
+// Search implements SearchProvider with
+// {base}/search?q=…&format=json&categories=general.
+func (s *SearXNG) Search(ctx context.Context, query string, n int) ([]Candidate, error) {
+	q, err := searchQuery(query, n)
+	if err != nil {
+		return nil, err
+	}
+	u := *s.endpoint
+	u.RawQuery = url.Values{"q": {q}, "format": {"json"}, "categories": {"general"}}.Encode()
+	req, err := http.NewRequestWithContext(ctx, http.MethodGet, u.String(), nil)
+	if err != nil {
+		return nil, fmt.Errorf("recipes: build SearXNG request: %w", err)
+	}
+	// Results is a pointer so a body without the array is told apart from
+	// an empty search.
+	var body struct {
+		Results *[]struct {
+			URL     string `json:"url"`
+			Title   string `json:"title"`
+			Content string `json:"content"`
+		} `json:"results"`
+	}
+	if err := doSearch(s.client, req, "SearXNG", &body); err != nil {
+		return nil, err
+	}
+	if body.Results == nil {
+		return nil, fmt.Errorf("%w: SearXNG body has no results array", ErrSearchResponse)
+	}
+	var out []Candidate
+	for _, r := range *body.Results {
+		out = append(out, Candidate{URL: r.URL, Title: r.Title, Snippet: r.Content})
+	}
+	return candidates("SearXNG", out, n)
+}
+
+// Brave searches through the Brave Web Search API.
+type Brave struct {
+	key      string
+	endpoint string
+	client   *http.Client
+}
+
+// NewBrave returns a Brave provider that authenticates with key, the
+// decrypted API key.
+func NewBrave(key string) (*Brave, error) {
+	if key == "" {
+		return nil, errors.New("recipes: Brave API key is empty")
+	}
+	for i := 0; i < len(key); i++ {
+		if key[i] < 0x21 || key[i] > 0x7e {
+			// The key is never echoed.
+			return nil, errors.New("recipes: Brave API key has a character not allowed in a header")
+		}
+	}
+	return &Brave{key: key, endpoint: BraveEndpoint, client: newSearchClient()}, nil
+}
+
+// Search implements SearchProvider with GET BraveEndpoint?q=…&count=n.
+func (b *Brave) Search(ctx context.Context, query string, n int) ([]Candidate, error) {
+	q, err := searchQuery(query, n)
+	if err != nil {
+		return nil, err
+	}
+	u := b.endpoint + "?" + url.Values{"q": {q}, "count": {strconv.Itoa(n)}}.Encode()
+	req, err := http.NewRequestWithContext(ctx, http.MethodGet, u, nil)
+	if err != nil {
+		return nil, fmt.Errorf("recipes: build Brave request: %w", err)
+	}
+	req.Header.Set("X-Subscription-Token", b.key)
+	// Brave leaves out "web" when nothing matched, so a search response is
+	// recognised by its "type"; a web object present must hold the results
+	// array.
+	var body struct {
+		Type string `json:"type"`
+		Web  *struct {
+			Results *[]struct {
+				URL         string `json:"url"`
+				Title       string `json:"title"`
+				Description string `json:"description"`
+			} `json:"results"`
+		} `json:"web"`
+	}
+	if err := doSearch(b.client, req, "Brave", &body); err != nil {
+		return nil, err
+	}
+	if body.Type != "search" {
+		return nil, fmt.Errorf("%w: Brave body type is %q, not \"search\"", ErrSearchResponse, body.Type)
+	}
+	if body.Web == nil {
+		return nil, nil
+	}
+	if body.Web.Results == nil {
+		return nil, fmt.Errorf("%w: Brave web object has no results array", ErrSearchResponse)
+	}
+	var out []Candidate
+	for _, r := range *body.Web.Results {
+		out = append(out, Candidate{URL: r.URL, Title: r.Title, Snippet: r.Description})
+	}
+	return candidates("Brave", out, n)
+}
+
+func newSearchClient() *http.Client {
+	return &http.Client{
+		Timeout: SearchTimeout,
+		CheckRedirect: func(*http.Request, []*http.Request) error {
+			return http.ErrUseLastResponse
+		},
+	}
+}
+
+// searchQuery validates n and returns query with " recipe" appended unless
+// one of its words already is "recipe", in any case.
+func searchQuery(query string, n int) (string, error) {
+	if n < 1 || n > MaxSearchResults {
+		return "", fmt.Errorf("recipes: search result count %d is outside 1-%d", n, MaxSearchResults)
+	}
+	q := strings.TrimSpace(query)
+	if q == "" {
+		return "", errors.New("recipes: search query is empty")
+	}
+	for _, w := range strings.Fields(q) {
+		if strings.EqualFold(w, "recipe") {
+			return q, nil
+		}
+	}
+	return q + " recipe", nil
+}
+
+// doSearch sends req and decodes a 2xx JSON body into v, mapping failures to
+// the search errors.
+func doSearch(client *http.Client, req *http.Request, provider string, v any) error {
+	req.Header.Set("Accept", "application/json")
+	req.Header.Set("User-Agent", UserAgent)
+	resp, err := client.Do(req)
+	if err != nil {
+		if ctxErr := req.Context().Err(); ctxErr != nil {
+			return ctxErr
+		}
+		return fmt.Errorf("recipes: %s search: %w", provider, err)
+	}
+	defer resp.Body.Close()
+	switch {
+	case resp.StatusCode == http.StatusUnauthorized || resp.StatusCode == http.StatusForbidden:
+		return fmt.Errorf("%w: %s HTTP %d", ErrSearchRefused, provider, resp.StatusCode)
+	case resp.StatusCode == http.StatusTooManyRequests:
+		return fmt.Errorf("%w: %s", ErrSearchRateLimited, provider)
+	case resp.StatusCode < 200 || resp.StatusCode > 299:
+		return &SearchStatusError{Provider: provider, Status: resp.StatusCode}
+	}
+	data, err := io.ReadAll(io.LimitReader(resp.Body, MaxSearchBodyBytes+1))
+	if err != nil {
+		return fmt.Errorf("recipes: %s search: read body: %w", provider, err)
+	}
+	if len(data) > MaxSearchBodyBytes {
+		return fmt.Errorf("%w: %s body is over %d bytes", ErrSearchResponse, provider, MaxSearchBodyBytes)
+	}
+	if err := json.Unmarshal(data, v); err != nil {
+		return fmt.Errorf("%w: %s: %v", ErrSearchResponse, provider, err)
+	}
+	return nil
+}
+
+// candidates cleans results — HTML tags and entities, which both providers
+// leave in titles and snippets, are removed — checks every URL is absolute
+// http(s) and keeps the first n.
+func candidates(provider string, results []Candidate, n int) ([]Candidate, error) {
+	if len(results) > n {
+		results = results[:n]
+	}
+	out := make([]Candidate, 0, len(results))
+	for i, r := range results {
+		u, err := url.Parse(r.URL)
+		if err != nil || (u.Scheme != "http" && u.Scheme != "https") || u.Host == "" {
+			return nil, fmt.Errorf("%w: %s result %d has no absolute http(s) URL", ErrSearchResponse, provider, i)
+		}
+		out = append(out, Candidate{URL: r.URL, Title: stripTags(r.Title), Snippet: stripTags(r.Snippet)})
+	}
+	return out, nil
+}
+
+// stripTags drops HTML tags such as Brave's <strong> highlights, then
+// decodes entities and trims space.
+func stripTags(s string) string {
+	var b strings.Builder
+	inTag := false
+	for _, r := range s {
+		switch {
+		case r == '<':
+			inTag = true
+		case r == '>' && inTag:
+			inTag = false
+		case !inTag:
+			b.WriteRune(r)
+		}
+	}
+	return cleanText(b.String())
+}

internal/recipes/search_test.go

@@ -0,0 +1,294 @@
+package recipes
+
+import (
+	"context"
+	"crypto/rand"
+	"encoding/base64"
+	"errors"
+	"net/http"
+	"net/http/httptest"
+	"reflect"
+	"strings"
+	"testing"
+
+	"code.cubecraftlabs.com/Maniacs_Mansion/Meal-Quest/internal/db"
+	"code.cubecraftlabs.com/Maniacs_Mansion/Meal-Quest/internal/secretbox"
+)
+
+const searxngBody = `{
+	"query": "chicken curry recipe",
+	"results": [
+		{"url": "https://www.bbcgoodfood.com/recipes/easy-chicken-curry", "title": "Easy chicken curry recipe | BBC Good Food", "content": "Make our easy chicken curry & rice.", "engine": "duckduckgo"},
+		{"url": "https://www.recipetineats.com/chicken-curry/", "title": "Chicken Curry", "content": "A proper curry."},
+		{"url": "https://example.com/third", "title": "Third", "content": ""}
+	],
+	"number_of_results": 0
+}`
+
+const braveBody = `{
+	"type": "search",
+	"web": {
+		"type": "search",
+		"results": [
+			{"url": "https://www.bbcgoodfood.com/recipes/easy-chicken-curry", "title": "Easy <strong>chicken curry</strong> recipe", "description": "Make our <strong>easy</strong> chicken curry &amp; rice.", "language": "en"},
+			{"url": "https://www.recipetineats.com/chicken-curry/", "title": "Chicken Curry", "description": "A proper curry."}
+		]
+	}
+}`
+
+func newSearXNGServer(t *testing.T, status int, body string, got *http.Request) *SearXNG {
+	t.Helper()
+	srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		if got != nil {
+			*got = *r.Clone(context.Background())
+		}
+		w.Header().Set("Content-Type", "application/json")
+		w.WriteHeader(status)
+		w.Write([]byte(body))
+	}))
+	t.Cleanup(srv.Close)
+	s, err := NewSearXNG(srv.URL + "/searx")
+	if err != nil {
+		t.Fatal(err)
+	}
+	return s
+}
+
+func newBraveServer(t *testing.T, key string, status int, body string, got *http.Request) *Brave {
+	t.Helper()
+	srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		if got != nil {
+			*got = *r.Clone(context.Background())
+		}
+		w.WriteHeader(status)
+		w.Write([]byte(body))
+	}))
+	t.Cleanup(srv.Close)
+	b, err := NewBrave(key)
+	if err != nil {
+		t.Fatal(err)
+	}
+	b.endpoint = srv.URL + "/res/v1/web/search"
+	return b
+}
+
+func TestSearXNGSearch(t *testing.T) {
+	var req http.Request
+	s := newSearXNGServer(t, http.StatusOK, searxngBody, &req)
+	got, err := s.Search(context.Background(), "chicken curry", 2)
+	if err != nil {
+		t.Fatal(err)
+	}
+	if req.URL.Path != "/searx/search" {
+		t.Errorf("path = %q, want /searx/search", req.URL.Path)
+	}
+	q := req.URL.Query()
+	if q.Get("q") != "chicken curry recipe" || q.Get("format") != "json" || q.Get("categories") != "general" {
+		t.Errorf("query = %v", q)
+	}
+	if req.Header.Get("User-Agent") != UserAgent {
+		t.Errorf("User-Agent = %q", req.Header.Get("User-Agent"))
+	}
+	want := []Candidate{
+		{URL: "https://www.bbcgoodfood.com/recipes/easy-chicken-curry", Title: "Easy chicken curry recipe | BBC Good Food", Snippet: "Make our easy chicken curry & rice."},
+		{URL: "https://www.recipetineats.com/chicken-curry/", Title: "Chicken Curry", Snippet: "A proper curry."},
+	}
+	if !reflect.DeepEqual(got, want) {
+		t.Errorf("candidates =\n%#v\nwant\n%#v", got, want)
+	}
+}
+
+func TestBraveSearch(t *testing.T) {
+	var req http.Request
+	b := newBraveServer(t, "brave-key-123", http.StatusOK, braveBody, &req)
+	got, err := b.Search(context.Background(), "Chicken curry RECIPE", 5)
+	if err != nil {
+		t.Fatal(err)
+	}
+	if req.Header.Get("X-Subscription-Token") != "brave-key-123" {
+		t.Errorf("X-Subscription-Token = %q", req.Header.Get("X-Subscription-Token"))
+	}
+	if req.Header.Get("Accept") != "application/json" {
+		t.Errorf("Accept = %q", req.Header.Get("Accept"))
+	}
+	q := req.URL.Query()
+	if q.Get("q") != "Chicken curry RECIPE" || q.Get("count") != "5" {
+		t.Errorf("query = %v", q)
+	}
+	want := []Candidate{
+		{URL: "https://www.bbcgoodfood.com/recipes/easy-chicken-curry", Title: "Easy chicken curry recipe", Snippet: "Make our easy chicken curry & rice."},
+		{URL: "https://www.recipetineats.com/chicken-curry/", Title: "Chicken Curry", Snippet: "A proper curry."},
+	}
+	if !reflect.DeepEqual(got, want) {
+		t.Errorf("candidates =\n%#v\nwant\n%#v", got, want)
+	}
+}
+
+func TestSearchQuery(t *testing.T) {
+	for _, tc := range []struct{ in, want string }{
+		{"chicken curry", "chicken curry recipe"},
+		{"  chicken curry  ", "chicken curry recipe"},
+		{"chicken curry recipe", "chicken curry recipe"},
+		{"Recipe for chicken curry", "Recipe for chicken curry"},
+		{"recipes for curry", "recipes for curry recipe"},
+	} {
+		got, err := searchQuery(tc.in, 3)
+		if err != nil || got != tc.want {
+			t.Errorf("searchQuery(%q) = %q, %v; want %q", tc.in, got, err, tc.want)
+		}
+	}
+	for _, tc := range []struct {
+		q string
+		n int
+	}{{"", 3}, {"   ", 3}, {"curry", 0}, {"curry", MaxSearchResults + 1}} {
+		if _, err := searchQuery(tc.q, tc.n); err == nil {
+			t.Errorf("searchQuery(%q, %d) succeeded, want error", tc.q, tc.n)
+		}
+	}
+}
+
+func TestSearchErrors(t *testing.T) {
+	tests := []struct {
+		name   string
+		status int
+		body   string
+		check  func(error) bool
+	}{
+		{"unauthorized", http.StatusUnauthorized, `{}`, func(err error) bool { return errors.Is(err, ErrSearchRefused) }},
+		{"forbidden", http.StatusForbidden, `{}`, func(err error) bool { return errors.Is(err, ErrSearchRefused) }},
+		{"rate limited", http.StatusTooManyRequests, `{}`, func(err error) bool { return errors.Is(err, ErrSearchRateLimited) }},
+		{"server error", http.StatusBadGateway, `{}`, func(err error) bool {
+			var se *SearchStatusError
+			return errors.As(err, &se) && se.Status == http.StatusBadGateway
+		}},
+		{"redirect not followed", http.StatusFound, ``, func(err error) bool {
+			var se *SearchStatusError
+			return errors.As(err, &se) && se.Status == http.StatusFound
+		}},
+		{"not JSON", http.StatusOK, `<html>`, func(err error) bool { return errors.Is(err, ErrSearchResponse) }},
+		{"result without URL", http.StatusOK, `{"type":"search","results":[{"title":"x"}],"web":{"results":[{"title":"x"}]}}`, func(err error) bool { return errors.Is(err, ErrSearchResponse) }},
+		{"relative URL", http.StatusOK, `{"type":"search","results":[{"url":"/x"}],"web":{"results":[{"url":"/x"}]}}`, func(err error) bool { return errors.Is(err, ErrSearchResponse) }},
+		{"empty object", http.StatusOK, `{}`, func(err error) bool { return errors.Is(err, ErrSearchResponse) }},
+		{"null results", http.StatusOK, `{"type":"search","results":null,"web":{"results":null}}`, func(err error) bool { return errors.Is(err, ErrSearchResponse) }},
+		{"results not an array", http.StatusOK, `{"type":"search","results":{},"web":{"results":{}}}`, func(err error) bool { return errors.Is(err, ErrSearchResponse) }},
+		{"body too large", http.StatusOK, `{"results":[],"pad":"` + strings.Repeat("x", MaxSearchBodyBytes) + `"}`, func(err error) bool { return errors.Is(err, ErrSearchResponse) }},
+	}
+	for _, tc := range tests {
+		t.Run(tc.name, func(t *testing.T) {
+			providers := map[string]SearchProvider{
+				"SearXNG": newSearXNGServer(t, tc.status, tc.body, nil),
+				"Brave":   newBraveServer(t, "secret-key", tc.status, tc.body, nil),
+			}
+			for name, p := range providers {
+				_, err := p.Search(context.Background(), "curry", 3)
+				if err == nil || !tc.check(err) {
+					t.Errorf("%s: err = %v", name, err)
+				}
+				if err != nil && strings.Contains(err.Error(), "secret-key") {
+					t.Errorf("%s: error leaks the API key: %v", name, err)
+				}
+			}
+		})
+	}
+}
+
+// TestSearchNoMatches checks a well-formed response with nothing found is an
+// empty search, not an error.
+func TestSearchNoMatches(t *testing.T) {
+	providers := map[string]SearchProvider{
+		"SearXNG":           newSearXNGServer(t, http.StatusOK, `{"query":"x recipe","results":[]}`, nil),
+		"Brave without web": newBraveServer(t, "secret-key", http.StatusOK, `{"type":"search","query":{"original":"x recipe"}}`, nil),
+		"Brave empty web":   newBraveServer(t, "secret-key", http.StatusOK, `{"type":"search","web":{"type":"search","results":[]}}`, nil),
+	}
+	for name, p := range providers {
+		got, err := p.Search(context.Background(), "x", 3)
+		if err != nil || len(got) != 0 {
+			t.Errorf("%s: got %v, %v; want no candidates and no error", name, got, err)
+		}
+	}
+}
+
+func TestSearchContextCancelled(t *testing.T) {
+	s := newSearXNGServer(t, http.StatusOK, searxngBody, nil)
+	ctx, cancel := context.WithCancel(context.Background())
+	cancel()
+	if _, err := s.Search(ctx, "curry", 3); !errors.Is(err, context.Canceled) {
+		t.Errorf("err = %v, want context.Canceled", err)
+	}
+}
+
+func TestSearchUnreachable(t *testing.T) {
+	srv := httptest.NewServer(http.NotFoundHandler())
+	url := srv.URL
+	srv.Close()
+	s, err := NewSearXNG(url)
+	if err != nil {
+		t.Fatal(err)
+	}
+	_, err = s.Search(context.Background(), "curry", 3)
+	if err == nil || !strings.Contains(err.Error(), "SearXNG search") {
+		t.Errorf("err = %v", err)
+	}
+}
+
+func TestNewSearXNGRejects(t *testing.T) {
+	for _, u := range []string{"", "searxng:8080", "ftp://searxng", "http://", "http://u:p@searxng", "http://searxng?x=1", "http://searxng#f"} {
+		if _, err := NewSearXNG(u); err == nil {
+			t.Errorf("NewSearXNG(%q) succeeded, want error", u)
+		}
+	}
+	if _, err := NewSearXNG("http://searxng:8080"); err != nil {
+		t.Errorf("internal URL rejected: %v", err)
+	}
+}
+
+func TestNewBraveRejects(t *testing.T) {
+	for _, k := range []string{"", "with space", "line\nbreak"} {
+		if _, err := NewBrave(k); err == nil {
+			t.Errorf("NewBrave(%q) succeeded, want error", k)
+		}
+	}
+}
+
+func TestNewSearchProvider(t *testing.T) {
+	raw := make([]byte, secretbox.KeySize)
+	rand.Read(raw)
+	box, err := secretbox.New(base64.StdEncoding.EncodeToString(raw))
+	if err != nil {
+		t.Fatal(err)
+	}
+	ct, err := box.Encrypt([]byte("brave-key"))
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	p, err := NewSearchProvider(db.RecipeSearchSetting{Provider: "brave", ApiKeyCt: ct}, box, "")
+	if err != nil {
+		t.Fatal(err)
+	}
+	if b, ok := p.(*Brave); !ok || b.key != "brave-key" || b.endpoint != BraveEndpoint {
+		t.Errorf("provider = %#v, want Brave with decrypted key", p)
+	}
+
+	p, err = NewSearchProvider(db.RecipeSearchSetting{Provider: "searxng"}, box, "http://searxng:8080")
+	if err != nil {
+		t.Fatal(err)
+	}
+	if s, ok := p.(*SearXNG); !ok || s.endpoint.String() != "http://searxng:8080/search" {
+		t.Errorf("provider = %#v, want SearXNG at http://searxng:8080/search", p)
+	}
+
+	for name, tc := range map[string]struct {
+		s   db.RecipeSearchSetting
+		url string
+	}{
+		"brave without key":   {db.RecipeSearchSetting{Provider: "brave"}, ""},
+		"brave bad key":       {db.RecipeSearchSetting{Provider: "brave", ApiKeyCt: []byte("not a token at all, far too plain")}, ""},
+		"searxng without URL": {db.RecipeSearchSetting{Provider: "searxng"}, ""},
+		"unknown provider":    {db.RecipeSearchSetting{Provider: "google"}, ""},
+	} {
+		if _, err := NewSearchProvider(tc.s, box, tc.url); err == nil {
+			t.Errorf("%s: succeeded, want error", name)
+		}
+	}
+}

internal/recipes/testdata/bbcgoodfood.html

@@ -0,0 +1,13 @@
+<!DOCTYPE html>
+<!-- Trimmed from https://www.bbcgoodfood.com/recipes/easy-chicken-curry (fetched 2026-10-05): head metadata and JSON-LD only. -->
+<html><head>
+<title>Easy chicken curry recipe | Good Food</title>
+<meta property="og:title" content="Easy chicken curry" data-testid="meta-og-title"/>
+<meta property="og:site_name" content="Good Food" data-testid="meta-og-site-name"/>
+<meta property="og:url" content="https://www.bbcgoodfood.com/recipes/easy-chicken-curry" data-testid="meta-og-url"/>
+<meta property="og:image" content="https://images.immediate.co.uk/production/volatile/sites/30/2020/08/easy-chicken-curry-aa22a0b.jpg?resize=440%2C230" data-testid="meta-og-image"/>
+<script data-testid="schema" type="application/ld+json">{"@context":"https://schema.org","@type":"VideoObject","name":"Easy Chicken Curry","contentUrl":"https://media.delight.video/0c18e53b610964fac175fe0e757adc30557e79b7/cb0e0397ffd102279027b8d892036a85e98afa8b/MEDIA/v0/HD/media.mp4","description":"This easy staple chicken curry is a fantastic recipe for family dinners. It's made with just a handful of ingredients and is enriched with creamy yogurt","embedUrl":"https://www.bbcgoodfood.com/recipes/easy-chicken-curry","thumbnailUrl":"https://media.delight.video/0c18e53b610964fac175fe0e757adc30557e79b7/cb0e0397ffd102279027b8d892036a85e98afa8b/MEDIA/v0/THUMBS/media.0000005.jpg","uploadDate":"2022-07-29T10:34:28.279Z","duration":"PT0M57S","customPosterUrl":"https://media.delight.video/0c18e53b610964fac175fe0e757adc30557e79b7/cb0e0397ffd102279027b8d892036a85e98afa8b/POSTER_USER/v0/cb0e0397ffd102279027b8d892036a85e98afa8b.jpeg"}</script>
+<script data-testid="page-schema" type="application/ld+json">{"@context":"https://schema.org","@id":"https://www.bbcgoodfood.com/recipes/easy-chicken-curry#Recipe","@type":"Recipe","description":"This easy staple chicken curry is a fantastic recipe for family dinners. It's made with just a handful of ingredients and is enriched with creamy yogurt","image":[{"@type":"ImageObject","url":"https://images.immediate.co.uk/production/volatile/sites/30/2020/08/easy-chicken-curry-aa22a0b.jpg?resize=440,400","width":440,"height":400}],"mainEntityOfPage":{"@type":"WebPage","@id":"https://www.bbcgoodfood.com/recipes/easy-chicken-curry"},"name":"Easy chicken curry","url":"https://www.bbcgoodfood.com/recipes/easy-chicken-curry","author":[{"@type":"Person","name":"Esther Clark","url":"https://www.bbcgoodfood.com/author/estherclark"}],"dateModified":"2026-08-13T14:40:21+01:00","datePublished":"2019-01-25T15:26:05+00:00","headline":"Easy chicken curry","keywords":"1 of 5-a-day, Chicken, Curry, Esther Clark, Family, Family friendly, Gluten free","publisher":{"@type":"Organization","name":"Good Food","url":"https://www.bbcgoodfood.com","logo":{"@type":"ImageObject","height":325,"url":"https://images.immediate.co.uk/production/volatile/sites/30/2026/02/goodfood.logo-f37096c.png?resize=1312,325","width":1312}},"cookTime":"PT45M","nutrition":{"@type":"NutritionInformation","calories":"354 calories","fatContent":"23 grams fat","saturatedFatContent":"4 grams saturated fat","carbohydrateContent":"10 grams carbohydrates","sugarContent":"8 grams sugar","fiberContent":"3 grams fiber","proteinContent":"24 grams protein","sodiumContent":"0.6 milligram of sodium"},"prepTime":"PT5M","recipeCategory":"Dinner, Main course","recipeIngredient":["2 tbsp sunflower oil","1 onion thinly sliced","2 garlic cloves crushed","thumb-sized piece of ginger grated","6 chicken thighs boneless and skinless","3 tbsp medium spice paste (tikka works well)","400g can chopped tomatoes","100g Greek yogurt","1 small bunch of coriander leaves chopped","50g ground almonds","naan breads or cooked basmati rice, to serve"],"recipeInstructions":[{"@type":"HowToStep","text":"Heat the oil in a flameproof casserole dish or large frying pan over a medium heat. Add the onion and a generous pinch of salt and fry for 8-10 mins, or until the onion has turned golden brown and sticky."},{"@type":"HowToStep","text":"Add the garlic and ginger, cooking for a further minute."},{"@type":"HowToStep","text":"Chop the chicken into chunky 3cm pieces, add to the pan and fry for 5 mins before stirring through the spice paste and tomatoes, along with 250ml water. Bring to the boil, lower to a simmer and cook on a gentle heat uncovered for 25-30 mins or until rich and slightly reduced."},{"@type":"HowToStep","text":"Stir though the yogurt, coriander and ground almonds, season and serve with warm naan or fluffy basmati rice."}],"recipeYield":4,"suitableForDiet":"https://schema.org/GlutenFreeDiet","totalTime":"PT50M"}</script>
+<script type="application/ld+json">{"@context":"https://schema.org/","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://www.bbcgoodfood.com/"},{"@type":"ListItem","position":2,"name":"Recipes","item":"https://www.bbcgoodfood.com/recipes"},{"@type":"ListItem","position":3,"name":"Easy chicken curry"}]}</script>
+<script type="application/ld+json">{"@context":"https://schema.org/","@id":"https://www.bbcgoodfood.com/recipes/easy-chicken-curry#Recipe","aggregateRating":{"@type":"AggregateRating","ratingValue":5,"reviewCount":363,"bestRating":5,"worstRating":1}}</script>
+</head><body><main>(page body trimmed)</main></body></html>

internal/recipes/testdata/budgetbytes-about.html

@@ -0,0 +1,10 @@
+<!DOCTYPE html>
+<!-- Trimmed from https://www.budgetbytes.com/about/ (fetched 2026-10-05): head metadata and JSON-LD only. -->
+<html><head>
+<title>The Budget Bytes Story - BudgetBytes.com</title>
+<meta property="og:title" content="The Budget Bytes Story - BudgetBytes.com" />
+<meta property="og:url" content="https://www.budgetbytes.com/about/" />
+<meta property="og:site_name" content="Budget Bytes" />
+<meta property="og:image" content="https://www.budgetbytes.com/wp-content/uploads/2025/04/Budget-Bytes-Group-Shot_Web.jpg" />
+<script type="application/ld+json" class="yoast-schema-graph">{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.budgetbytes.com\/about\/","url":"https:\/\/www.budgetbytes.com\/about\/","name":"The Budget Bytes Story - BudgetBytes.com","isPartOf":{"@id":"https:\/\/www.budgetbytes.com\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.budgetbytes.com\/about\/#primaryimage"},"image":{"@id":"https:\/\/www.budgetbytes.com\/about\/#primaryimage"},"thumbnailUrl":"https:\/\/www.budgetbytes.com\/wp-content\/uploads\/2025\/04\/Budget-Bytes-Group-Shot_Web.jpg","datePublished":"2013-05-02T18:19:55+00:00","dateModified":"2026-07-20T15:36:17+00:00","description":"Beth Moncel created Budget Bytes in 2009 to help people learn how to cook and create delicious recipes designed for small budgets.","inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.budgetbytes.com\/about\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.budgetbytes.com\/about\/#primaryimage","url":"https:\/\/www.budgetbytes.com\/wp-content\/uploads\/2025\/04\/Budget-Bytes-Group-Shot_Web.jpg","contentUrl":"https:\/\/www.budgetbytes.com\/wp-content\/uploads\/2025\/04\/Budget-Bytes-Group-Shot_Web.jpg","width":1200,"height":800,"caption":"The Budget Bytes team that develops recipes out of our Nashville studio. Left to right: Rebecca Denton (Photographer), Jess Rice (Recipe Development Manager), Jennie Alley (Senior Recipe Developer), and Melissa Nolan (Senior Recipe Developer). We develop, test, and photograph each and every recipe on Budget Bytes from Nashville Tennessee."},{"@type":"WebSite","@id":"https:\/\/www.budgetbytes.com\/#website","url":"https:\/\/www.budgetbytes.com\/","name":"Budget Bytes","description":"Delicious Recipes Designed for Small Budgets","publisher":{"@id":"https:\/\/www.budgetbytes.com\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.budgetbytes.com\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.budgetbytes.com\/#organization","name":"Budget Bytes","url":"https:\/\/www.budgetbytes.com\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.budgetbytes.com\/#\/schema\/logo\/image\/","url":"https:\/\/www.budgetbytes.com\/wp-content\/uploads\/2024\/11\/budgetbytes_square_icon_transparent-300x300-1.png","contentUrl":"https:\/\/www.budgetbytes.com\/wp-content\/uploads\/2024\/11\/budgetbytes_square_icon_transparent-300x300-1.png","width":300,"height":300,"caption":"Budget Bytes"},"image":{"@id":"https:\/\/www.budgetbytes.com\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/budgetbytes1\/?ref=hl","https:\/\/x.com\/Budget_Bytes","https:\/\/instagram.com\/budgetbytes\/","https:\/\/www.pinterest.com\/budgetbytes\/","https:\/\/www.youtube.com\/channel\/UC17vdVOZBVxTSDcldUBBlRg"]}]}</script>
+</head><body><main>(page body trimmed)</main></body></html>

internal/recipes/testdata/foodcom.html

@@ -0,0 +1,9 @@
+<!DOCTYPE html>
+<!-- Trimmed from https://www.food.com/recipe/best-banana-bread-2886 (fetched 2026-10-05): head metadata and JSON-LD only; the 80 KB review array removed from the Recipe node. -->
+<html><head>
+<title>Best Banana Bread Recipe  - Food.com</title>
+<meta name="og:title" content="Best Banana Bread Recipe  - Food.com"/>
+<meta name="og:image" content="https://img.sndimg.com/food/image/upload/w_555,h_416,c_fit,fl_progressive,q_95/v1/img/recipes/28/86/wBZxNua1T8yfDwbfo9Fz_0S9A9315.jpg"/>
+<meta name="og:url" content=""/>
+<script type="application/ld+json">{"@context":"http://schema.org","@type":"Recipe","mainEntityOfPage":"true","name":"Best Banana Bread","author":"lkadlec","cookTime":"PT1H","prepTime":"PT10M","totalTime":"PT1H10M","datePublished":"1999-09-26T20:49Z","description":"Make and share this Best Banana Bread recipe from Food.com.","image":"https://img.sndimg.com/food/image/upload/q_92,fl_progressive,w_1200,c_scale/v1/img/recipes/28/86/wBZxNua1T8yfDwbfo9Fz_0S9A9315.jpg","recipeCategory":"Quick Breads","keywords":"Breads,Fruit,Oven,< 4 Hours","recipeIngredient":["1/2  cup    butter, softened ","1   cup    granulated sugar","2       eggs, beaten ","3       bananas, finely crushed  (for serious and extreme moist and delicious, try 4 bananas)","1 1/2  cups    all-purpose flour","1   teaspoon    baking soda","1/2  teaspoon    salt","1/2  teaspoon    vanilla (optional)"],"aggregateRating":{"@type":"AggregateRating","ratingValue":"5.0","reviewCount":"2355"},"nutrition":{"@type":"NutritionInformation","calories":"272.8","fatContent":"10.5","saturatedFatContent":"6.2","cholesterolContent":"61.6","sodiumContent":"338.3","carbohydrateContent":"42.5","fiberContent":"1.4","sugarContent":"24.4","proteinContent":"3.7"},"recipeInstructions":[{"@type":"HowToStep","text":"Remove odd pots and pans from oven."},{"@type":"HowToStep","text":"Preheat oven to 350&ordm; / 180&ordm;."},{"@type":"HowToStep","text":"Cream together butter and sugar."},{"@type":"HowToStep","text":"Add eggs and crushed bananas."},{"@type":"HowToStep","text":"Combine well."},{"@type":"HowToStep","text":"Sift together flour, soda and salt. Add to creamed mixture. Add vanilla."},{"@type":"HowToStep","text":"Mix just until combined. Do not overmix."},{"@type":"HowToStep","text":"Pour into greased and floured loaf pan."},{"@type":"HowToStep","text":"Bake at 350&ordm; / 180&ordm; for 55 minutes."},{"@type":"HowToStep","text":"Keeps well, refrigerated."}],"recipeYield":"1 loaf, 10 serving(s)","publisher":{"@type":"Organization","name":"Food.com","logo":{"@type":"ImageObject","url":"https://geniuskitchen.sndimg.com/fdc-new/img/FDC-Logo.png"},"url":"https://www.food.com"},"video":[{"@type":"VideoObject","name":"Best Banana Bread","duration":"42S","thumbnailUrl":"https://img.sndimg.com/food/image/upload/w_1024,h_576,c_fit/v1/img/mpx/5458761_16x9.jpg","contentUrl":"https://www.food.com/recipe/best-banana-bread-2886"}]}</script>
+</head><body><main>(page body trimmed)</main></body></html>

internal/recipes/testdata/kingarthur.html

@@ -0,0 +1,79 @@
+<!DOCTYPE html>
+<!-- Trimmed from https://www.kingarthurbaking.com/recipes/classic-sandwich-bread-recipe (fetched 2026-10-05): head metadata and JSON-LD only. -->
+<html><head>
+<title>Classic Sandwich Bread Recipe | King Arthur Baking</title>
+<meta property="og:site_name" content="King Arthur Baking" />
+<meta property="og:url" content="https://www.kingarthurbaking.com/recipes/classic-sandwich-bread-recipe" />
+<meta property="og:title" content="Classic Sandwich Bread" />
+<meta property="og:image" content="https://www.kingarthurbaking.com/sites/default/files/2026-02/Classic-Sandwich-Bread-12.jpg" />
+<script type="application/ld+json">{
+    "@context": "https://schema.org",
+    "@graph": [
+        {
+            "@type": "Recipe",
+            "keywords": "Sandwich bread;;Tangzhong",
+            "name": "Classic Sandwich Bread",
+            "description": "Made with all of the ingredients that make homemade white bread superior to storebought — butter, milk, and high-quality flour — this sandwich bread recipe is our version of a classic, perfected. The soft and tender interior makes this loaf ideal for sandwiches, toast, or even French toast. It\u0027s a timeless recipe you\u0027ll come back to time and time again.",
+            "recipeYield": [
+                "16",
+                "1 loaf"
+            ],
+            "datePublished": "December 31, 2007 at 7:00pm",
+            "prepTime": "PT12M",
+            "cookTime": "PT40M",
+            "totalTime": "PT2H42M",
+            "recipeInstructions": [
+                "Weigh your flour; or measure it by gently spooning it into a cup, then sweeping off any excess.",
+                "In a large bowl, combine all of the ingredients. Mix and knead everything together — by hand, mixer, or bread machine set on the dough cycle — until you\u0027ve made a smooth dough. If you\u0027re kneading in a stand mixer, it should take 5 to 7 minutes at medium-low speed, and the dough should barely clean the sides of the bowl, perhaps sticking a bit at the bottom. In a bread machine (or by hand), knead the dough for about 6 to 8 minutes, until it forms a smooth ball. ",
+                "Transfer the dough to a lightly greased bowl, cover the bowl, and allow the dough to rise until puffy though not necessarily doubled in bulk, about 1 to 2 hours, depending on the warmth of your kitchen. If you\u0027re using a bread machine, allow the machine to complete its cycle, then leave the dough in the machine until it\u0027s doubled in bulk, perhaps an additional 30 minutes or so.",
+                "Gently deflate the dough and transfer it to a lightly oiled work surface. Shape the dough into an 8\u0022 log.",
+                "Place the log in a lightly greased 8 1/2\u0022 x 4 1/2\u0022 loaf pan, cover the pan loosely with lightly greased plastic wrap, and allow the bread to rise for about 60 minutes, until it\u0027s domed about 1\u0022 above the edge of the pan. A finger pressed into the dough should leave a mark that rebounds slowly. Towards the end of the rise, preheat your oven to 350°F.",
+                "Bake the bread for 30 to 35 minutes, until it\u0027s light golden brown. Test it for doneness by removing it from the pan and thumping it on the bottom (it should sound hollow), or by measuring its interior temperature with a digital thermometer (it should register 190°F at the center of the loaf).",
+                "Remove the bread from the oven, and cool it on a rack before slicing. Store the bread in a plastic bag at room temperature for several days; freeze for longer storage."
+            ],
+            "recipeIngredient": [
+                "3 cups (360g) King Arthur Unbleached All-Purpose Flour",
+                "1/2 cup (113g) milk, (skim, 1%, 2% or whole, your choice)*",
+                "1/2 to 2/3 cup (113g to 152g) hot water, enough to make a soft, smooth dough* (see \u0022tips,\u0022 below)",
+                "4 tablespoons (57g) melted butter or 1/4 cup (50g) vegetable oil",
+                "2 tablespoons (25g) granulated sugar",
+                "1 1/4 teaspoons (8g) table salt",
+                "2 teaspoons instant yeast"
+            ],
+            "publisher": {
+                "@type": "Organization",
+                "name": "King Arthur Baking",
+                "url": "https://www.kingarthurbaking.com/"
+            },
+            "author": {
+                "@type": "Person",
+                "name": "PJ Hamel",
+                "url": "https://www.kingarthurbaking.com/author/pj-hamel"
+            },
+            "image": {
+                "@type": "ImageObject",
+                "url": "https://www.kingarthurbaking.com/sites/default/files/2026-02/Classic-Sandwich-Bread-12.jpg"
+            },
+            "nutrition": {
+                "@type": "NutritionInformation",
+                "servingSize": "43g",
+                "calories": "120 calories",
+                "carbohydrateContent": "19g",
+                "cholesterolContent": "10mg",
+                "fiberContent": "1g",
+                "proteinContent": "3g",
+                "sodiumContent": "190mg",
+                "sugarContent": "2g",
+                "fatContent": "3g",
+                "saturatedFatContent": "2g",
+                "transFatContent": "0g"
+            },
+            "aggregateRating": {
+                "@type": "AggregateRating",
+                "ratingValue": "4.60",
+                "ratingCount": "681"
+            }
+        }
+    ]
+}</script>
+</head><body><main>(page body trimmed)</main></body></html>