Fail empty ideas jobs, clear ideas on failure, show each idea as it grounds
CI / ci (push) Successful in 5m24s
CI / ci (pull_request) Successful in 6m11s

- A run that keeps no idea fails with no_ideas_found instead of ending
  done with an empty list.
- Before an ideas job fails it republishes its result with no ideas, so
  GET /api/v1/jobs/{id} never shows ideas on a failed job, even ones a
  previous grounding round already published.
- Each idea is published as soon as its grounding finishes, placed in the
  model's order. recipes.GroundEach reports items as they finish;
  GroundMany is rebuilt on it with unchanged semantics.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-08 19:22:51 -04:00
co-authored by Claude Opus 5.5
parent b566be7ddc
commit 4146f31275
6 changed files with 325 additions and 68 deletions
+11 -6
View File
@@ -317,7 +317,7 @@ automatic** (D-021).
job (service-only: `POST /jobs` refuses the kind) and answers 202 job (service-only: `POST /jobs` refuses the kind) and answers 202
`{jobId}` without waiting. The job runs `ideas_from_ingredients` (scope: `{jobId}` without waiting. The job runs `ideas_from_ingredients` (scope:
`skills.IdeasScope`, the flag and the normalized list in order), then `skills.IdeasScope`, the flag and the normalized list in order), then
grounds its ideas in the model's order with `Grounder.GroundMany`, as grounds its ideas in the model's order with `Grounder.GroundEach`, as
many as are still wanted at a time, until 5 are kept or none are left. many as are still wanted at a time, until 5 are kept or none are left.
An idea no recipe grounds is dropped; with `quick_only`, so is one whose An idea no recipe grounds is dropped; with `quick_only`, so is one whose
grounded recipe is not a Quick dinner (`ideas.IsQuick`: method grounded recipe is not a Quick dinner (`ideas.IsQuick`: method
@@ -326,10 +326,14 @@ automatic** (D-021).
`{stage, ingredients, quick_only, ideas: [{name, uses, missing, summary, `{stage, ingredients, quick_only, ideas: [{name, uses, missing, summary,
method, minutes_total, difficulty, quick, recipe_id, photo_url, method, minutes_total, difficulty, quick, recipe_id, photo_url,
source_url, site_name}], dropped}` at each stage (`suggesting`, source_url, site_name}], dropped}` at each stage (`suggesting`,
`grounding`, `done`) and again each time an idea is kept, so the result `grounding`, `done`) and again as each idea is kept, the moment its
fills as the ideas ground; it may end `done` with no ideas. It fails grounding finishes, so the result fills idea by idea; the ideas shown
with `ai_failed` (the skill or `normalize_recipe` failed; no idea is are always in the model's order, an earlier idea that grounds later
made up instead, D-017), `search_not_configured` or `search_failed`. taking its place before those shown. It fails with `no_ideas_found`
(every idea was dropped), `ai_failed` (the skill or `normalize_recipe`
failed; no idea is made up instead, D-017), `search_not_configured` or
`search_failed`. Before it fails it publishes its result again with no
ideas, so a failed job shows none, even those kept before the failure.
`GET /api/v1/suggestions/deck` returns `{suggestions: [{id, dinner_id, `GET /api/v1/suggestions/deck` returns `{suggestions: [{id, dinner_id,
name, photo_url, difficulty, method, minutes_total, cuisine, reason, name, photo_url, difficulty, method, minutes_total, cuisine, reason,
cited_regular_ids, status, dinner_status, decided_at, created_at}], cited_regular_ids, status, dinner_status, decided_at, created_at}],
@@ -663,7 +667,8 @@ untitled (the job fails `title_failed`), and stores nothing. The
candidates `Ground` would try, without grounding any. Search failures candidates `Ground` would try, without grounding any. Search failures
wrap `recipes.ErrSearchFailed` and model provider failures wrap `recipes.ErrSearchFailed` and model provider failures
`recipes.ErrNormalizeFailed`. `GroundMany` grounds items 3 at a time and returns one result per `recipes.ErrNormalizeFailed`. `GroundMany` grounds items 3 at a time and returns one result per
item, so one failure never affects its siblings. A recipe is usable only item, so one failure never affects its siblings; `GroundEach` grounds
the same way but reports each item, one at a time, as it finishes. A recipe is usable only
with ingredients, an image and, if given, a positive total time. with ingredients, an image and, if given, a positive total time.
## Scheduler ## Scheduler
+85 -47
View File
@@ -2,8 +2,9 @@
// of ingredients on hand, ideas_from_ingredients proposes dinners, each is // of ingredients on hand, ideas_from_ingredients proposes dinners, each is
// grounded with the recipe grounding pipeline (dropping those no recipe // grounded with the recipe grounding pipeline (dropping those no recipe
// grounds), and the ideas kept are reported as a job's result, which fills // grounds), and the ideas kept are reported as a job's result, which fills
// as each idea grounds. Ideas are not stored as dinners: the household // as each idea grounds. A job that keeps no idea, or fails, shows no idea:
// acts on one later. // nothing is made up instead (D-017). Ideas are not stored as dinners: the
// household acts on one later.
package ideas package ideas
import ( import (
@@ -12,6 +13,7 @@ import (
"errors" "errors"
"fmt" "fmt"
"log/slog" "log/slog"
"slices"
"code.cubecraftlabs.com/Maniacs_Mansion/Meal-Quest/internal/ai/skills" "code.cubecraftlabs.com/Maniacs_Mansion/Meal-Quest/internal/ai/skills"
"code.cubecraftlabs.com/Maniacs_Mansion/Meal-Quest/internal/dinners" "code.cubecraftlabs.com/Maniacs_Mansion/Meal-Quest/internal/dinners"
@@ -43,10 +45,15 @@ const (
ReasonSearchNotConfigured = "search_not_configured" ReasonSearchNotConfigured = "search_not_configured"
// ReasonSearchFailed: the search provider failed. // ReasonSearchFailed: the search provider failed.
ReasonSearchFailed = "search_failed" ReasonSearchFailed = "search_failed"
// ReasonNoIdeasFound: every idea was dropped, by
// ideas_from_ingredients' rules, because no recipe grounds it, or, for
// a Quick only request, because its grounded recipe is not a Quick
// dinner.
ReasonNoIdeasFound = "no_ideas_found"
) )
// Reasons are every failure reason of the job. // Reasons are every failure reason of the job.
var Reasons = []string{ReasonAIFailed, ReasonSearchNotConfigured, ReasonSearchFailed} var Reasons = []string{ReasonAIFailed, ReasonSearchNotConfigured, ReasonSearchFailed, ReasonNoIdeasFound}
// InvalidError is a request the household must correct, such as an empty // InvalidError is a request the household must correct, such as an empty
// ingredient list. // ingredient list.
@@ -153,7 +160,9 @@ type Idea struct {
} }
// Result is the KindFromIngredients result, published as the job goes: // Result is the KindFromIngredients result, published as the job goes:
// its stage, the request, the ideas grounded so far in the model's order, // its stage, the request, the ideas grounded so far in the model's order
// (an idea that grounds sooner than one before it is shown first, and the
// earlier one is put back in its place when it grounds),
// and how many ideas were dropped, by ideas_from_ingredients' rules, // and how many ideas were dropped, by ideas_from_ingredients' rules,
// because no recipe grounds them, or, for a Quick only request, because // because no recipe grounds them, or, for a Quick only request, because
// the grounded recipe is not a Quick dinner. // the grounded recipe is not a Quick dinner.
@@ -179,103 +188,132 @@ func IsQuick(method string, minutes *int) bool {
// most the number still wanted at a time, until skills.IdeasKept are kept // most the number still wanted at a time, until skills.IdeasKept are kept
// or none are left. An idea no recipe grounds is dropped, and so, for a // or none are left. An idea no recipe grounds is dropped, and so, for a
// Quick only request, is one whose grounded recipe is not a Quick dinner; // Quick only request, is one whose grounded recipe is not a Quick dinner;
// any other grounding failure fails the job, as does // when every idea is dropped the job fails with ReasonNoIdeasFound. Any
// ideas_from_ingredients failing, with no idea made up instead. The // other grounding failure fails the job, as does ideas_from_ingredients
// result is published at each stage and again as each idea is kept, so it // failing, with no idea made up instead. The result is published at each
// fills as the ideas ground. The job may end done with no ideas, when // stage and again as each idea is kept, so it fills as the ideas ground.
// every one was dropped. // Before the job fails, its result is published again with no ideas, so a
// failed job never shows ideas kept before the failure.
func (s *Service) run(ctx context.Context, p FromIngredientsJob, publish func(Result) error) (Result, error) { func (s *Service) run(ctx context.Context, p FromIngredientsJob, publish func(Result) error) (Result, error) {
res, err := s.fill(ctx, p, publish) res, err := s.fill(ctx, p, publish)
if err != nil { if err != nil {
if ctx.Err() != nil {
return Result{}, err
}
var failure *jobs.Failure var failure *jobs.Failure
if ctx.Err() == nil && errors.As(err, &failure) { if errors.As(err, &failure) {
// Only the code is logged: the error may quote a model's or // Only the code is logged: the error may quote a model's or
// search provider's response. // search provider's response.
s.logger.InfoContext(ctx, "ideas from ingredients failed", "reason", failure.Code) s.logger.InfoContext(ctx, "ideas from ingredients failed", "reason", failure.Code)
} }
if len(res.Ideas) > 0 {
res.Ideas = []Idea{}
if perr := publish(res); perr != nil {
return Result{}, fmt.Errorf("ideas: clear failed job's ideas: %w", perr)
}
}
return Result{}, err return Result{}, err
} }
return res, nil return res, nil
} }
// fill does run's work, returning a *jobs.Failure for a failure the // fill does run's work, returning a *jobs.Failure for a failure the
// household may see. // household may see. On failure it returns the result last published with
// the error, so run can clear its ideas.
func (s *Service) fill(ctx context.Context, p FromIngredientsJob, publish func(Result) error) (Result, error) { func (s *Service) fill(ctx context.Context, p FromIngredientsJob, publish func(Result) error) (Result, error) {
res := Result{Stage: StageSuggesting, Ingredients: p.Ingredients, QuickOnly: p.QuickOnly, Ideas: []Idea{}} res := Result{Stage: StageSuggesting, Ingredients: p.Ingredients, QuickOnly: p.QuickOnly, Ideas: []Idea{}}
if err := publish(res); err != nil { if err := publish(res); err != nil {
return Result{}, err return res, err
} }
in := skills.IdeasRequest{Ingredients: p.Ingredients, QuickOnly: p.QuickOnly} in := skills.IdeasRequest{Ingredients: p.Ingredients, QuickOnly: p.QuickOnly}
out, err := s.generate(ctx, skills.IdeasScope(in), in) out, err := s.generate(ctx, skills.IdeasScope(in), in)
if err != nil { if err != nil {
if ctx.Err() != nil { if ctx.Err() != nil {
return Result{}, ctx.Err() return res, ctx.Err()
} }
return Result{}, &jobs.Failure{Code: ReasonAIFailed} return res, &jobs.Failure{Code: ReasonAIFailed}
} }
res.Stage = StageGrounding res.Stage = StageGrounding
res.Dropped = len(out.Dropped) res.Dropped = len(out.Dropped)
if err := publish(res); err != nil { if err := publish(res); err != nil {
return Result{}, err return res, err
} }
left := out.Ideas // order[k] is the index in out.Ideas of res.Ideas[k], so each idea is
for len(res.Ideas) < skills.IdeasKept && len(left) > 0 { // put in its place in the model's order as it grounds.
round := left[:min(skills.IdeasKept-len(res.Ideas), len(left))] var order []int
left = left[len(round):] for next := 0; len(res.Ideas) < skills.IdeasKept && next < len(out.Ideas); {
results, err := s.ground(ctx, round) first := next
if err != nil { round := out.Ideas[first:min(first+skills.IdeasKept-len(res.Ideas), len(out.Ideas))]
return Result{}, err next += len(round)
} err := s.ground(ctx, round, func(i int, gr *recipes.Grounded) error {
for i, r := range results { if gr == nil {
if r == nil {
res.Dropped++ res.Dropped++
continue return nil
} }
idea := ideaOf(round[i], *r) idea := ideaOf(round[i], *gr)
if p.QuickOnly && !idea.Quick { if p.QuickOnly && !idea.Quick {
res.Dropped++ res.Dropped++
continue return nil
} }
res.Ideas = append(res.Ideas, idea) at := len(order)
if err := publish(res); err != nil { for at > 0 && order[at-1] > first+i {
return Result{}, err at--
} }
order = slices.Insert(order, at, first+i)
res.Ideas = slices.Insert(res.Ideas, at, idea)
return publish(res)
})
if err != nil {
return res, err
} }
} }
if len(res.Ideas) == 0 {
return res, &jobs.Failure{Code: ReasonNoIdeasFound}
}
res.Stage = StageDone res.Stage = StageDone
return res, nil return res, nil
} }
// ground grounds ideas by their search queries and returns one result per // ground grounds ideas by their search queries and calls kept, one call at
// idea, in order: its grounding, or nil when no recipe grounds it. Any // a time, as each idea grounds, with its index in ideas and its
// other failure fails the job. // grounding, or nil when no recipe grounds it. Any other grounding
func (s *Service) ground(ctx context.Context, ideas []skills.Idea) ([]*recipes.Grounded, error) { // failure, or kept failing, stops the groundings still running and is
// returned, as a *jobs.Failure when the household may see it.
func (s *Service) ground(ctx context.Context, ideas []skills.Idea, kept func(i int, gr *recipes.Grounded) error) error {
groundCtx, cancel := context.WithCancel(ctx)
defer cancel()
reqs := make([]recipes.GroundRequest, len(ideas)) reqs := make([]recipes.GroundRequest, len(ideas))
for i, idea := range ideas { for i, idea := range ideas {
reqs[i] = recipes.GroundRequest{DishName: idea.Name, Query: idea.SearchQuery} reqs[i] = recipes.GroundRequest{DishName: idea.Name, Query: idea.SearchQuery}
} }
results := s.grounder.GroundMany(ctx, reqs) var failed error
if err := ctx.Err(); err != nil { s.grounder.GroundEach(groundCtx, reqs, func(i int, r recipes.GroundResult) {
return nil, err if failed != nil {
} return
out := make([]*recipes.Grounded, len(results)) }
for i, r := range results {
var ungroundable *recipes.ErrUngroundable var ungroundable *recipes.ErrUngroundable
switch { switch {
case r.Err == nil: case r.Err == nil:
out[i] = &r.Grounded failed = kept(i, &r.Grounded)
case errors.As(r.Err, &ungroundable): case errors.As(r.Err, &ungroundable):
failed = kept(i, nil)
case errors.Is(r.Err, recipes.ErrSearchNotConfigured): case errors.Is(r.Err, recipes.ErrSearchNotConfigured):
return nil, &jobs.Failure{Code: ReasonSearchNotConfigured} failed = &jobs.Failure{Code: ReasonSearchNotConfigured}
case errors.Is(r.Err, recipes.ErrSearchFailed): case errors.Is(r.Err, recipes.ErrSearchFailed):
return nil, &jobs.Failure{Code: ReasonSearchFailed} failed = &jobs.Failure{Code: ReasonSearchFailed}
case errors.Is(r.Err, recipes.ErrNormalizeFailed): case errors.Is(r.Err, recipes.ErrNormalizeFailed):
return nil, &jobs.Failure{Code: ReasonAIFailed} failed = &jobs.Failure{Code: ReasonAIFailed}
default: default:
return nil, fmt.Errorf("ideas: ground %q: %w", ideas[i].Name, r.Err) failed = fmt.Errorf("ideas: ground %q: %w", ideas[i].Name, r.Err)
} }
if failed != nil {
cancel()
}
})
if err := ctx.Err(); err != nil {
return err
} }
return out, nil return failed
} }
// ideaOf is idea as grounded by gr. // ideaOf is idea as grounded by gr.
+139 -7
View File
@@ -131,6 +131,16 @@ type recorder struct {
published []Result published []Result
} }
// latest returns the last result published.
func (r *recorder) latest() (Result, bool) {
r.mu.Lock()
defer r.mu.Unlock()
if len(r.published) == 0 {
return Result{}, false
}
return r.published[len(r.published)-1], true
}
func (r *recorder) publish(res Result) error { func (r *recorder) publish(res Result) error {
r.mu.Lock() r.mu.Lock()
defer r.mu.Unlock() defer r.mu.Unlock()
@@ -154,6 +164,17 @@ func (e *env) run(t *testing.T, ingredients []string, quickOnly bool) (Result, [
return res, rec.published, err return res, rec.published, err
} }
// within reports whether every name in sub appears in list, in list's
// order.
func within(sub, list []string) bool {
for _, name := range list {
if len(sub) > 0 && sub[0] == name {
sub = sub[1:]
}
}
return len(sub) == 0
}
func names(list []Idea) []string { func names(list []Idea) []string {
out := []string{} out := []string{}
for _, i := range list { for _, i := range list {
@@ -214,9 +235,13 @@ func TestResultFillsAsEachIdeaGrounds(t *testing.T) {
if !reflect.DeepEqual(stages, wantStages) || !reflect.DeepEqual(counts, []int{0, 0, 1, 2, 3, 4, 5}) { if !reflect.DeepEqual(stages, wantStages) || !reflect.DeepEqual(counts, []int{0, 0, 1, 2, 3, 4, 5}) {
t.Fatalf("published stages %q with %v ideas, want %q with 0, 0, 1, 2, 3, 4, 5", stages, counts, wantStages) t.Fatalf("published stages %q with %v ideas, want %q with 0, 0, 1, 2, 3, 4, 5", stages, counts, wantStages)
} }
// Ideas ground concurrently, so one may appear before an earlier one,
// but each publish keeps every idea already shown, in the model's
// order.
for i := 3; i < len(published); i++ { for i := 3; i < len(published); i++ {
if !reflect.DeepEqual(published[i].Ideas[:i-2], published[i-1].Ideas) { prev, cur := names(published[i-1].Ideas), names(published[i].Ideas)
t.Errorf("publish %d changed ideas already published", i+1) if !within(prev, cur) || !within(cur, want) {
t.Errorf("publish %d shows %q after %q, want earlier ideas kept in the model's order %q", i+1, cur, prev, want)
} }
} }
@@ -342,15 +367,122 @@ func TestFailures(t *testing.T) {
} }
} }
func TestEveryIdeaUngroundableEndsDoneEmpty(t *testing.T) { func TestEachIdeaShowsAsSoonAsItGrounds(t *testing.T) {
e := newEnv(t) e := newEnv(t)
e.reply(idea("Mystery Stew", "stovetop", 30, "rice")) for _, name := range []string{"Burrito Bowls", "Fried Rice", "Black Bean Soup"} {
res, _, err := e.run(t, pantry, false) e.recipe(t, name, name, 35)
}
// The first idea's search waits, so the two after it ground first.
gate := make(chan struct{})
e.search.Gates = map[string]chan struct{}{"Burrito Bowls": gate}
e.reply(
idea("Burrito Bowls", "stovetop", 35, "rice", "black beans"),
idea("Fried Rice", "stovetop", 20, "rice"),
idea("Black Bean Soup", "stovetop", 40, "black beans"),
)
ings, err := skills.CleanIngredients(pantry)
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
} }
if res.Stage != StageDone || len(res.Ideas) != 0 || res.Dropped != 1 { var rec recorder
t.Fatalf("result = %+v, want done with none and 1 dropped", res) type outcome struct {
res Result
err error
}
done := make(chan outcome, 1)
go func() {
res, err := e.svc.run(t.Context(), FromIngredientsJob{Ingredients: ings}, rec.publish)
done <- outcome{res, err}
}()
deadline := time.Now().Add(10 * time.Second)
for {
if latest, ok := rec.latest(); ok && len(latest.Ideas) == 2 {
if got := names(latest.Ideas); !reflect.DeepEqual(got, []string{"Fried Rice", "Black Bean Soup"}) {
t.Fatalf("while the first idea grounds, ideas = %q, want the two grounded", got)
}
if latest.Stage != StageGrounding {
t.Fatalf("stage = %s, want grounding", latest.Stage)
}
break
}
if time.Now().After(deadline) {
t.Fatal("the ideas that grounded were not shown while the first idea was still grounding")
}
time.Sleep(5 * time.Millisecond)
}
close(gate)
out := <-done
if out.err != nil {
t.Fatal(out.err)
}
want := []string{"Burrito Bowls", "Fried Rice", "Black Bean Soup"}
if got := names(out.res.Ideas); !reflect.DeepEqual(got, want) || out.res.Stage != StageDone {
t.Fatalf("result = %s %q, want done with %q in the model's order", out.res.Stage, got, want)
}
}
func TestNoIdeaKeptFailsTheJob(t *testing.T) {
for _, tc := range []struct {
name string
quickOnly bool
setup func(t *testing.T, e *env)
}{
{"no recipe grounds any", false, func(t *testing.T, e *env) {
e.reply(idea("Mystery Stew", "stovetop", 30, "rice"), idea("Unknown Casserole", "oven", 50, "rice"))
}},
{"no grounded recipe is quick", true, func(t *testing.T, e *env) {
e.recipe(t, "Roast Chicken", "Roast Chicken", 90)
e.reply(idea("Roast Chicken", "oven", 25, "Chicken thighs"))
}},
} {
t.Run(tc.name, func(t *testing.T) {
e := newEnv(t)
tc.setup(t, e)
_, published, err := e.run(t, pantry, tc.quickOnly)
var failure *jobs.Failure
if !errors.As(err, &failure) || failure.Code != ReasonNoIdeasFound {
t.Fatalf("err = %v, want failure %s", err, ReasonNoIdeasFound)
}
for _, p := range published {
if len(p.Ideas) != 0 || p.Stage == StageDone {
t.Fatalf("a job that kept no idea published %+v", p)
}
}
})
}
}
func TestLaterRoundFailureClearsPublishedIdeas(t *testing.T) {
e := newEnv(t)
kept := []string{"Burrito Bowls", "Fried Rice", "Black Bean Soup", "Chicken Tacos"}
for _, name := range kept {
e.recipe(t, name, name, 35)
}
// The first round keeps four ideas; the second round's search fails.
e.search.Fail = map[string]error{"Rice Pilaf": errors.New("boom")}
e.reply(
idea("Burrito Bowls", "stovetop", 35, "rice"),
idea("Mystery Stew", "stovetop", 35, "rice"),
idea("Fried Rice", "stovetop", 20, "rice"),
idea("Black Bean Soup", "stovetop", 40, "black beans"),
idea("Chicken Tacos", "stovetop", 25, "Chicken thighs"),
idea("Rice Pilaf", "stovetop", 30, "rice"),
)
_, published, err := e.run(t, pantry, false)
var failure *jobs.Failure
if !errors.As(err, &failure) || failure.Code != ReasonSearchFailed {
t.Fatalf("err = %v, want failure %s", err, ReasonSearchFailed)
}
most := 0
for _, p := range published {
most = max(most, len(p.Ideas))
}
if most != len(kept) {
t.Fatalf("at most %d ideas published, want the first round's %d before the failure", most, len(kept))
}
if last := published[len(published)-1]; len(last.Ideas) != 0 {
t.Fatalf("last publish before the failure shows %q, want no ideas", names(last.Ideas))
} }
} }
+18 -4
View File
@@ -292,7 +292,22 @@ type GroundResult struct {
// affect the others. // affect the others.
func (g *Grounder) GroundMany(ctx context.Context, reqs []GroundRequest) []GroundResult { func (g *Grounder) GroundMany(ctx context.Context, reqs []GroundRequest) []GroundResult {
results := make([]GroundResult, len(reqs)) results := make([]GroundResult, len(reqs))
g.GroundEach(ctx, reqs, func(i int, r GroundResult) { results[i] = r })
return results
}
// GroundEach grounds each request as GroundMany does, but calls done with
// each item's index and result as soon as that item finishes, one call at
// a time. It returns once every item has finished and every call has
// returned.
func (g *Grounder) GroundEach(ctx context.Context, reqs []GroundRequest, done func(i int, r GroundResult)) {
slots := make(chan struct{}, GroundConcurrency) slots := make(chan struct{}, GroundConcurrency)
var mu sync.Mutex
report := func(i int, r GroundResult) {
mu.Lock()
defer mu.Unlock()
done(i, r)
}
var wg sync.WaitGroup var wg sync.WaitGroup
for i, req := range reqs { for i, req := range reqs {
wg.Add(1) wg.Add(1)
@@ -301,16 +316,15 @@ func (g *Grounder) GroundMany(ctx context.Context, reqs []GroundRequest) []Groun
select { select {
case slots <- struct{}{}: case slots <- struct{}{}:
case <-ctx.Done(): case <-ctx.Done():
results[i] = GroundResult{Err: ctx.Err()} report(i, GroundResult{Err: ctx.Err()})
return return
} }
defer func() { <-slots }()
gr, err := g.Ground(ctx, req.DishName, req.Query) gr, err := g.Ground(ctx, req.DishName, req.Query)
results[i] = GroundResult{Grounded: gr, Err: err} <-slots
report(i, GroundResult{Grounded: gr, Err: err})
}() }()
} }
wg.Wait() wg.Wait()
return results
} }
// stored returns the grounding of the recipe stored for sourceURL, and // stored returns the grounding of the recipe stored for sourceURL, and
+14 -4
View File
@@ -23,12 +23,16 @@ import (
) )
// Search is a recipes.SearchSource and recipes.SearchProvider answering // Search is a recipes.SearchSource and recipes.SearchProvider answering
// each query from Results. Err, when set, fails every search. Gate, when // each query from Results. Err, when set, fails every search, and Fail
// set, makes every search wait for it to close. // fails the searches for its queries. Gate, when set, makes every search
// wait for it to close, and Gates makes the searches for its queries wait
// for theirs.
type Search struct { type Search struct {
Results map[string][]recipes.Candidate Results map[string][]recipes.Candidate
Err error Err error
Fail map[string]error
Gate chan struct{} Gate chan struct{}
Gates map[string]chan struct{}
mu sync.Mutex mu sync.Mutex
queries []string queries []string
@@ -40,9 +44,12 @@ func (s *Search) Search(ctx context.Context, query string, n int) ([]recipes.Can
s.mu.Lock() s.mu.Lock()
s.queries = append(s.queries, query) s.queries = append(s.queries, query)
s.mu.Unlock() s.mu.Unlock()
if s.Gate != nil { for _, gate := range []chan struct{}{s.Gate, s.Gates[query]} {
if gate == nil {
continue
}
select { select {
case <-s.Gate: case <-gate:
case <-ctx.Done(): case <-ctx.Done():
return nil, ctx.Err() return nil, ctx.Err()
} }
@@ -50,6 +57,9 @@ func (s *Search) Search(ctx context.Context, query string, n int) ([]recipes.Can
if s.Err != nil { if s.Err != nil {
return nil, s.Err return nil, s.Err
} }
if err := s.Fail[query]; err != nil {
return nil, err
}
c := s.Results[query] c := s.Results[query]
return c[:min(n, len(c))], nil return c[:min(n, len(c))], nil
} }
+58
View File
@@ -3,9 +3,12 @@ package server
import ( import (
"context" "context"
"encoding/json" "encoding/json"
"errors"
"fmt"
"io" "io"
"log/slog" "log/slog"
"net/http" "net/http"
"strings"
"testing" "testing"
"testing/fstest" "testing/fstest"
"time" "time"
@@ -164,6 +167,61 @@ func TestPostIdeasFromIngredientsRunsTheJob(t *testing.T) {
} }
} }
func TestFailedIdeasJobShowsNoIdeas(t *testing.T) {
t.Parallel()
idea := func(name string) string {
return fmt.Sprintf(`{"name":%q,"uses":["rice"],"missing":[],"est_minutes":30,"method":"stovetop","difficulty":"easy","search_query":%q}`, name, name)
}
for _, tc := range []struct {
name string
// grounds are the ideas a recipe grounds; the others ground none.
grounds []string
ideas []string
fail map[string]error
code string
}{
{"no idea kept", nil, []string{"Mystery Stew"}, nil, "no_ideas_found"},
{"a later round fails after ideas were shown",
[]string{"Fried Rice", "Rice Bowls", "Rice Soup", "Rice Salad"},
[]string{"Fried Rice", "Mystery Stew", "Rice Bowls", "Rice Soup", "Rice Salad", "Rice Pilaf"},
map[string]error{"Rice Pilaf": errors.New("boom")}, "search_failed"},
} {
t.Run(tc.name, func(t *testing.T) {
t.Parallel()
model, normalize := &llm.Fake{}, &llm.Fake{}
search := &recipestest.Search{Results: map[string][]recipes.Candidate{}, Fail: tc.fail}
pages := &recipestest.Pages{Bodies: map[string][]byte{}}
images := &recipestest.Images{JPEGs: map[string][]byte{}}
for i, name := range tc.grounds {
page, photo := fmt.Sprintf("https://r.example/%d", i), fmt.Sprintf("https://r.example/%d.jpg", i)
search.Results[name] = []recipes.Candidate{{URL: page}}
pages.Bodies[page] = recipestest.RecipePage(name, "R Kitchen", photo, 25)
images.JPEGs[photo] = recipestest.JPEG(t, 400, 320, uint8(i+1))
normalize.QueueGenerate(recipestest.NormalizedReply("stovetop", 25))
}
list := make([]string, len(tc.ideas))
for i, name := range tc.ideas {
list[i] = idea(name)
}
js := `{"ideas":[` + strings.Join(list, ",") + `]}`
model.QueueGenerate(llm.GenerateResult{Response: llm.Response{Text: js, JSON: json.RawMessage(js)}})
h, _ := newIdeasRouter(t, model, normalize, search, pages, images, true)
rec, body := callDinners(t, h, http.MethodPost, ideasPath, `{"ingredients": ["rice"], "quick_only": false}`)
if rec.Code != http.StatusAccepted {
t.Fatalf("POST = %d: %s", rec.Code, rec.Body)
}
job := waitJob(t, h, body["jobId"].(string))
if job["status"] != "failed" || job["error"] != tc.code {
t.Fatalf("GET job = %v, want failed with %s", job, tc.code)
}
if res, _ := job["result"].(map[string]any); res == nil || len(res["ideas"].([]any)) != 0 {
t.Fatalf("failed job result = %v, want no ideas", job["result"])
}
})
}
}
func TestJobsEndpointRefusesIdeasKind(t *testing.T) { func TestJobsEndpointRefusesIdeasKind(t *testing.T) {
t.Parallel() t.Parallel()
h, _ := newIdeasRouter(t, &llm.Fake{}, &llm.Fake{}, &recipestest.Search{}, &recipestest.Pages{}, &recipestest.Images{}, false) h, _ := newIdeasRouter(t, &llm.Fake{}, &llm.Fake{}, &recipestest.Search{}, &recipestest.Pages{}, &recipestest.Images{}, false)