Add suggest_side_quests skill v1
The skill asks for twice the Side quests needed, using the household context plus the pending Side quests, through an embedded, checksummed prompt template. A malformed reply gets the harness's one repair turn. A well-formed Side quest is dropped (and listed in Dropped) when it repeats a Regular, Quest log dinner, pending or passed name or an earlier Side quest in the batch, cites no Regular or one that is not a Regular, or has a reason over 180 characters or naming none of its cited Regulars. Golden tests use llm.Fake. Maestro task 85a898e1-f1f5-4414-92f3-3277d86afe90. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -322,6 +322,18 @@ automatic** (D-021).
|
||||
dinner when the token-set (Jaccard) similarity of their words is at least
|
||||
0.85. A `Deduper` over the Regulars, Quest log, pending suggestions and
|
||||
passed names rejects a suggestion that repeats any of them.
|
||||
- **Side quest suggestions** (`skills.SuggestSideQuests`, skill
|
||||
`suggest_side_quests` v1): asks for twice the Side quests needed, at the
|
||||
household temperature, with the household context plus the pending Side
|
||||
quests. Malformed replies (bad JSON, a missing field, a bad enum, an
|
||||
empty name) get the one repair turn. Well-formed Side quests that break a
|
||||
rule are dropped, not repaired, and listed in `Dropped` with the rule:
|
||||
one that repeats a Regular, Quest log dinner, pending Side quest, any
|
||||
passed name (the whole history, not only the 60 in the context) or an
|
||||
earlier Side quest of the batch; one citing no Regular or a name that is
|
||||
not a Regular; and one whose reason is over 180 characters or names none
|
||||
of its cited Regulars. Cited names are rewritten as the household writes
|
||||
them. Grounding each survivor in a recipe is the caller's next step.
|
||||
|
||||
## Recipe grounding
|
||||
|
||||
|
||||
@@ -3,3 +3,4 @@
|
||||
# version, rename its template to match and add a line for the new version.
|
||||
echo v1 ab5d7884f5c7a2261c06b0c9353bb9597070c001420bd4cbb6ec6272c093fdce
|
||||
normalize_recipe v1 7174e5479fc020f3b2b39953d85411381aea26defb9943145e7864c5b752d7ee
|
||||
suggest_side_quests v1 bd54d7bf9aea0980059a346e5f40ff4c866b80ef9bb9f74e7cbd413be0911c8a
|
||||
|
||||
@@ -0,0 +1,22 @@
|
||||
{{/* suggest_side_quests drafts Side quests: new dinners for the household to try, each based on its Regulars. */}}
|
||||
{{define "instructions" -}}
|
||||
You suggest Side quests: new dinners for the household to try, based on the dinners it already makes.
|
||||
|
||||
Every Side quest must be a dinner the household has not had: never one of its Regulars, its Quest log, its pending Side quests or the dinners it has passed, and never one of those under another name. Base every Side quest on one or more of the household's Regulars, such as a familiar protein in a new cuisine or familiar flavours cooked a new way. Prefer the favorite Regulars, and vary the cuisines across the batch.
|
||||
|
||||
Give for each Side quest:
|
||||
- name: the dinner's usual name, as a recipe site would title it.
|
||||
- cuisine: its cuisine, e.g. "Thai" or "Tex-Mex".
|
||||
- method: how it is mainly cooked: "stovetop", "oven", "crockpot" (slow cooker or crock pot), "grill" or "other".
|
||||
- est_minutes: your estimate of the whole minutes from starting prep to the table.
|
||||
- difficulty: "easy", "medium" or "hard" for a home cook.
|
||||
- reason: one sentence of at most 180 characters saying why the household would like it. It must name at least one of the Regulars in cited_regulars.
|
||||
- cited_regulars: the Regulars the reason relies on, at least one, each written exactly as its name appears in the Regulars list of the household context, without the details in parentheses. Cite only Regulars from that list.
|
||||
- search_query: a web search query that finds a recipe for this dinner.
|
||||
|
||||
One valid Side quest, for a household with a Regular named "Chicken Fajitas" (an example only, not about this household):
|
||||
{"name": "Chicken Shawarma Bowls", "cuisine": "Middle Eastern", "method": "oven", "est_minutes": 40, "difficulty": "easy", "reason": "Spiced roast chicken and crisp toppings, like your Chicken Fajitas with a Middle Eastern twist.", "cited_regulars": ["Chicken Fajitas"], "search_query": "chicken shawarma bowls recipe"}
|
||||
{{- end}}
|
||||
{{define "user" -}}
|
||||
Suggest {{.Count}} Side quests.
|
||||
{{- end}}
|
||||
@@ -0,0 +1,274 @@
|
||||
package skills
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"slices"
|
||||
"strings"
|
||||
"time"
|
||||
"unicode/utf8"
|
||||
|
||||
"code.cubecraftlabs.com/Maniacs_Mansion/Meal-Quest/internal/ai/harness"
|
||||
"code.cubecraftlabs.com/Maniacs_Mansion/Meal-Quest/internal/ai/llm"
|
||||
)
|
||||
|
||||
// ReasonLimit is the most characters a Side quest's reason, its "Why
|
||||
// this?", may have.
|
||||
const ReasonLimit = 180
|
||||
|
||||
// SideQuestsRequest is what suggest_side_quests reads.
|
||||
type SideQuestsRequest struct {
|
||||
// Context is the household context (see harness.BuildContext). It must
|
||||
// hold at least one Regular, since every Side quest cites one.
|
||||
Context harness.HouseholdContext
|
||||
// Needed is how many Side quests the caller wants to keep. The skill
|
||||
// asks the model for twice as many, since validation and grounding
|
||||
// drop some. It must be positive.
|
||||
Needed int
|
||||
// Pending holds the names of the Side quests not yet tried or passed.
|
||||
Pending []string
|
||||
// Declined holds the name of every passed Side quest, not only the
|
||||
// recent ones Context holds.
|
||||
Declined []string
|
||||
}
|
||||
|
||||
// SideQuest is one suggested new dinner. Method, EstMinutes and Difficulty
|
||||
// are the model's estimates until the dinner is grounded in a recipe.
|
||||
type SideQuest struct {
|
||||
Name string `json:"name"`
|
||||
Cuisine string `json:"cuisine"`
|
||||
// Method is one of stovetop, oven, crockpot, grill and other.
|
||||
Method string `json:"method"`
|
||||
EstMinutes int `json:"est_minutes"`
|
||||
// Difficulty is easy, medium or hard.
|
||||
Difficulty string `json:"difficulty"`
|
||||
// Reason is the "Why this?": at most ReasonLimit characters, naming at
|
||||
// least one of CitedRegulars.
|
||||
Reason string `json:"reason"`
|
||||
// CitedRegulars are names of the household's Regulars, as the
|
||||
// household writes them.
|
||||
CitedRegulars []string `json:"cited_regulars"`
|
||||
// SearchQuery is a web search query that finds a recipe for the dinner.
|
||||
SearchQuery string `json:"search_query"`
|
||||
}
|
||||
|
||||
// SideQuests is suggest_side_quests' output.
|
||||
type SideQuests struct {
|
||||
// Suggestions are the Side quests that passed every rule, in the
|
||||
// model's order. There may be more or fewer than Needed.
|
||||
Suggestions []SideQuest `json:"suggestions"`
|
||||
// Dropped are the Side quests the model gave that broke a rule, in the
|
||||
// model's order. It is never part of the model's reply.
|
||||
Dropped []DroppedSideQuest `json:"-"`
|
||||
}
|
||||
|
||||
// DroppedSideQuest is a Side quest dropped by PostProcess.
|
||||
type DroppedSideQuest struct {
|
||||
Name string
|
||||
// Why says which rule it broke.
|
||||
Why string
|
||||
}
|
||||
|
||||
// SuggestSideQuests runs suggest_side_quests v1 on in and returns the Side
|
||||
// quests that cite only real Regulars, give a short reason naming one of
|
||||
// them, and repeat no Regular, Quest log dinner, pending or passed Side
|
||||
// quest, nor each other. The rest are in Dropped. scope names the batch
|
||||
// (see harness.Run).
|
||||
func SuggestSideQuests(ctx context.Context, cfg harness.Config, scope string, in SideQuestsRequest) (SideQuests, error) {
|
||||
return harness.Run(ctx, cfg, suggestSideQuests{in: in}, scope, in)
|
||||
}
|
||||
|
||||
// suggestSideQuests is the skill. It holds its input because validation
|
||||
// and post-processing compare the output with it.
|
||||
type suggestSideQuests struct {
|
||||
in SideQuestsRequest
|
||||
}
|
||||
|
||||
var _ harness.Skill[SideQuestsRequest, SideQuests] = suggestSideQuests{}
|
||||
|
||||
func (suggestSideQuests) Name() string { return "suggest_side_quests" }
|
||||
|
||||
func (suggestSideQuests) Version() int { return 1 }
|
||||
|
||||
func (suggestSideQuests) Timeout() time.Duration { return harness.DefaultTimeout }
|
||||
|
||||
// sideQuestsPrompt is the template data of suggest_side_quests.v1.tmpl.
|
||||
type sideQuestsPrompt struct {
|
||||
// Count is how many Side quests to ask for.
|
||||
Count int
|
||||
}
|
||||
|
||||
// requested is how many Side quests the model is asked for.
|
||||
func (s suggestSideQuests) requested() int { return 2 * s.in.Needed }
|
||||
|
||||
func (suggestSideQuests) PromptInput(in SideQuestsRequest) (harness.PromptInput, error) {
|
||||
if in.Needed <= 0 {
|
||||
return harness.PromptInput{}, fmt.Errorf("%d Side quests needed; the number must be positive", in.Needed)
|
||||
}
|
||||
if len(in.Context.Regulars) == 0 {
|
||||
return harness.PromptInput{}, errors.New("household has no Regulars for Side quests to be based on")
|
||||
}
|
||||
sections := append(in.Context.Sections(), harness.ContextSection{Title: "Pending Side quests", Lines: in.Pending})
|
||||
return harness.PromptInput{Data: sideQuestsPrompt{Count: suggestSideQuests{in: in}.requested()}, Context: sections}, nil
|
||||
}
|
||||
|
||||
func (suggestSideQuests) Schema() llm.JSONSchema {
|
||||
return llm.JSONSchema{Name: "suggest_side_quests", Schema: json.RawMessage(`{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"suggestions": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"name": {"type": "string"},
|
||||
"cuisine": {"type": "string"},
|
||||
"method": {"type": "string", "enum": ["stovetop", "oven", "crockpot", "grill", "other"]},
|
||||
"est_minutes": {"type": "integer"},
|
||||
"difficulty": {"type": "string", "enum": ["easy", "medium", "hard"]},
|
||||
"reason": {"type": "string"},
|
||||
"cited_regulars": {"type": "array", "items": {"type": "string"}},
|
||||
"search_query": {"type": "string"}
|
||||
},
|
||||
"required": ["name", "cuisine", "method", "est_minutes", "difficulty", "reason", "cited_regulars", "search_query"],
|
||||
"additionalProperties": false
|
||||
}
|
||||
}
|
||||
},
|
||||
"required": ["suggestions"],
|
||||
"additionalProperties": false
|
||||
}`)}
|
||||
}
|
||||
|
||||
// UnmarshalJSON decodes a Side quest, requiring each of its fields as the
|
||||
// schema does: encoding/json would otherwise read a missing or null one as
|
||||
// empty.
|
||||
func (q *SideQuest) UnmarshalJSON(data []byte) error {
|
||||
if err := requireFields(data, nil, "name", "cuisine", "method", "est_minutes", "difficulty", "reason", "cited_regulars", "search_query"); err != nil {
|
||||
return fmt.Errorf("suggestion %s", err)
|
||||
}
|
||||
type plain SideQuest
|
||||
return json.Unmarshal(data, (*plain)(q))
|
||||
}
|
||||
|
||||
// UnmarshalJSON decodes the output, requiring its one field as the schema
|
||||
// does.
|
||||
func (out *SideQuests) UnmarshalJSON(data []byte) error {
|
||||
if err := requireFields(data, nil, "suggestions"); err != nil {
|
||||
return fmt.Errorf("output %s", err)
|
||||
}
|
||||
type plain SideQuests
|
||||
return json.Unmarshal(data, (*plain)(out))
|
||||
}
|
||||
|
||||
// Validate finds what makes the reply unusable as a whole: a wrong number
|
||||
// of Side quests or one that is malformed. A well-formed Side quest that
|
||||
// breaks a rule is dropped by PostProcess instead.
|
||||
func (s suggestSideQuests) Validate(out SideQuests) []string {
|
||||
var problems []string
|
||||
switch n := len(out.Suggestions); {
|
||||
case n == 0 && len(out.Dropped) == 0:
|
||||
problems = append(problems, fmt.Sprintf("suggestions is empty; give %d Side quests", s.requested()))
|
||||
case n > s.requested():
|
||||
problems = append(problems, fmt.Sprintf("suggestions has %d entries; give at most %d Side quests", n, s.requested()))
|
||||
}
|
||||
for i, q := range out.Suggestions {
|
||||
for _, f := range []struct{ field, value string }{
|
||||
{"name", q.Name}, {"cuisine", q.Cuisine}, {"reason", q.Reason}, {"search_query", q.SearchQuery},
|
||||
} {
|
||||
if strings.TrimSpace(f.value) == "" {
|
||||
problems = append(problems, fmt.Sprintf("suggestion %d has an empty %s; give one", i+1, f.field))
|
||||
}
|
||||
}
|
||||
if !slices.Contains(methods, q.Method) {
|
||||
problems = append(problems, fmt.Sprintf("suggestion %d method %q is not one of %s", i+1, q.Method, strings.Join(methods, ", ")))
|
||||
}
|
||||
if q.EstMinutes <= 0 {
|
||||
problems = append(problems, fmt.Sprintf("suggestion %d est_minutes %d is not positive; give the minutes from prep to table", i+1, q.EstMinutes))
|
||||
}
|
||||
if !slices.Contains(difficulties, q.Difficulty) {
|
||||
problems = append(problems, fmt.Sprintf("suggestion %d difficulty %q is not one of %s", i+1, q.Difficulty, strings.Join(difficulties, ", ")))
|
||||
}
|
||||
}
|
||||
return problems
|
||||
}
|
||||
|
||||
// PostProcess tidies every Side quest's text, writes its cited Regulars as
|
||||
// the household does, and moves each one that breaks a rule to Dropped
|
||||
// (see drop).
|
||||
func (s suggestSideQuests) PostProcess(out SideQuests) (SideQuests, error) {
|
||||
regulars := map[string]string{} // normalized name -> name
|
||||
var names []string
|
||||
for _, r := range s.in.Context.Regulars {
|
||||
if n := harness.Normalize(r.Name); n != "" {
|
||||
regulars[n] = r.Name
|
||||
}
|
||||
names = append(names, r.Name)
|
||||
}
|
||||
known := harness.NewDeduper(harness.DedupeSources{
|
||||
Regulars: names,
|
||||
QuestLog: s.in.Context.QuestLog,
|
||||
Pending: s.in.Pending,
|
||||
Declined: s.in.Declined,
|
||||
})
|
||||
result := SideQuests{Suggestions: []SideQuest{}}
|
||||
for _, q := range out.Suggestions {
|
||||
q = SideQuest{
|
||||
Name: oneLine(q.Name),
|
||||
Cuisine: oneLine(q.Cuisine),
|
||||
Method: q.Method,
|
||||
EstMinutes: q.EstMinutes,
|
||||
Difficulty: q.Difficulty,
|
||||
Reason: oneLine(q.Reason),
|
||||
CitedRegulars: q.CitedRegulars,
|
||||
SearchQuery: oneLine(q.SearchQuery),
|
||||
}
|
||||
cited, why := drop(q, regulars, known, result.Suggestions)
|
||||
if why != "" {
|
||||
result.Dropped = append(result.Dropped, DroppedSideQuest{Name: q.Name, Why: why})
|
||||
continue
|
||||
}
|
||||
q.CitedRegulars = cited
|
||||
result.Suggestions = append(result.Suggestions, q)
|
||||
}
|
||||
return result, nil
|
||||
}
|
||||
|
||||
// drop returns why q must be dropped, or "" and its cited Regulars as the
|
||||
// household writes them when it may be kept. q is dropped when it repeats
|
||||
// a known dinner or a Side quest already kept, cites no Regular or one not
|
||||
// in regulars (keyed by normalized name), or has a reason over ReasonLimit
|
||||
// characters or naming none of its cited Regulars.
|
||||
func drop(q SideQuest, regulars map[string]string, known *harness.Deduper, kept []SideQuest) ([]string, string) {
|
||||
if known.IsDuplicate(q.Name) {
|
||||
return nil, "repeats a Regular, Quest log dinner, pending or passed Side quest"
|
||||
}
|
||||
for _, k := range kept {
|
||||
if harness.Similar(q.Name, k.Name) {
|
||||
return nil, fmt.Sprintf("repeats the Side quest %q in the same batch", k.Name)
|
||||
}
|
||||
}
|
||||
if len(q.CitedRegulars) == 0 {
|
||||
return nil, "cites no Regular"
|
||||
}
|
||||
var cited []string
|
||||
for _, c := range q.CitedRegulars {
|
||||
name, ok := regulars[harness.Normalize(c)]
|
||||
if !ok {
|
||||
return nil, fmt.Sprintf("cites %q, which is not a Regular", c)
|
||||
}
|
||||
cited = append(cited, name)
|
||||
}
|
||||
if n := utf8.RuneCountInString(q.Reason); n > ReasonLimit {
|
||||
return nil, fmt.Sprintf("reason has %d characters, over %d", n, ReasonLimit)
|
||||
}
|
||||
reason := " " + harness.Normalize(q.Reason) + " "
|
||||
if !slices.ContainsFunc(cited, func(name string) bool {
|
||||
return strings.Contains(reason, " "+harness.Normalize(name)+" ")
|
||||
}) {
|
||||
return nil, "reason names none of its cited Regulars"
|
||||
}
|
||||
return cited, ""
|
||||
}
|
||||
@@ -0,0 +1,352 @@
|
||||
package skills
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"reflect"
|
||||
"slices"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"code.cubecraftlabs.com/Maniacs_Mansion/Meal-Quest/internal/ai/harness"
|
||||
"code.cubecraftlabs.com/Maniacs_Mansion/Meal-Quest/internal/ai/llm"
|
||||
"code.cubecraftlabs.com/Maniacs_Mansion/Meal-Quest/internal/store"
|
||||
)
|
||||
|
||||
func date(y int, m time.Month, d int) store.LocalDate {
|
||||
return store.LocalDateOf(time.Date(y, m, d, 0, 0, 0, 0, time.UTC))
|
||||
}
|
||||
|
||||
// household asks for two Side quests, so four are requested.
|
||||
var household = SideQuestsRequest{
|
||||
Context: harness.HouseholdContext{
|
||||
Today: date(2026, 10, 6),
|
||||
Timezone: "America/New_York",
|
||||
Regulars: []harness.RegularDinner{
|
||||
{Name: "Beef Chili", Cuisine: "Tex-Mex", Method: "crockpot", Minutes: 240},
|
||||
{Name: "Chicken Fajitas", Cuisine: "Tex-Mex", Method: "stovetop", Minutes: 30, Favorite: true},
|
||||
{Name: "Spaghetti Bolognese", Cuisine: "Italian", Method: "stovetop", Minutes: 45},
|
||||
},
|
||||
QuestLog: []string{"Pad Thai"},
|
||||
Passed: []string{"Chicken Tikka Masala"},
|
||||
Plans: []harness.PlannedDinner{
|
||||
{Date: date(2026, 10, 5), Name: "Chicken Fajitas"},
|
||||
{Date: date(2026, 10, 6), Name: "Beef Chili"},
|
||||
},
|
||||
},
|
||||
Needed: 2,
|
||||
Pending: []string{"Shrimp Scampi"},
|
||||
Declined: []string{"Chicken Tikka Masala", "Fish Tacos"},
|
||||
}
|
||||
|
||||
// sideQuest renders one Side quest of a reply.
|
||||
func sideQuest(name, reason string, cited ...string) string {
|
||||
c, _ := json.Marshal(append([]string{}, cited...))
|
||||
r, _ := json.Marshal(reason)
|
||||
return fmt.Sprintf(`{"name":%q,"cuisine":"Fusion","method":"stovetop","est_minutes":35,"difficulty":"easy","reason":%s,"cited_regulars":%s,"search_query":%q}`,
|
||||
name, r, c, strings.ToLower(name)+" recipe")
|
||||
}
|
||||
|
||||
// batch renders a reply holding the Side quests given.
|
||||
func batch(quests ...string) string {
|
||||
return `{"suggestions":[` + strings.Join(quests, ",") + `]}`
|
||||
}
|
||||
|
||||
var (
|
||||
shawarma = sideQuest("Chicken Shawarma Bowls", "Spiced chicken and crisp toppings, like your Chicken Fajitas.", "Chicken Fajitas")
|
||||
ragu = sideQuest("Beef Ragu Pappardelle", "A slow beef sauce like your Spaghetti Bolognese, on wide noodles.", "Spaghetti Bolognese")
|
||||
verde = sideQuest("Chili Verde", "Your Beef Chili gone green, with pork and tomatillos.", "beef chili")
|
||||
bulgogi = sideQuest("Beef Bulgogi", "Sweet, quick-seared beef for fans of Chicken Fajitas and Beef Chili.", "Chicken Fajitas", "Beef Chili")
|
||||
)
|
||||
|
||||
// want returns the Side quest that sideQuest renders, with cited as the
|
||||
// household writes them.
|
||||
func want(name, reason string, cited ...string) SideQuest {
|
||||
return SideQuest{
|
||||
Name: name, Cuisine: "Fusion", Method: "stovetop", EstMinutes: 35, Difficulty: "easy",
|
||||
Reason: reason, CitedRegulars: cited, SearchQuery: strings.ToLower(name) + " recipe",
|
||||
}
|
||||
}
|
||||
|
||||
var (
|
||||
wantShawarma = want("Chicken Shawarma Bowls", "Spiced chicken and crisp toppings, like your Chicken Fajitas.", "Chicken Fajitas")
|
||||
wantRagu = want("Beef Ragu Pappardelle", "A slow beef sauce like your Spaghetti Bolognese, on wide noodles.", "Spaghetti Bolognese")
|
||||
wantVerde = want("Chili Verde", "Your Beef Chili gone green, with pork and tomatillos.", "Beef Chili")
|
||||
wantBulgogi = want("Beef Bulgogi", "Sweet, quick-seared beef for fans of Chicken Fajitas and Beef Chili.", "Chicken Fajitas", "Beef Chili")
|
||||
)
|
||||
|
||||
// suggest runs SuggestSideQuests on household with f and checks f was
|
||||
// called exactly calls times, each at the household temperature.
|
||||
func suggest(t *testing.T, f *llm.Fake, in SideQuestsRequest, calls int) (SideQuests, []llm.Request, error) {
|
||||
t.Helper()
|
||||
out, err := SuggestSideQuests(context.Background(), testConfig(t, f), "2026-W41", in)
|
||||
reqs := f.Requests()
|
||||
if len(reqs) != calls {
|
||||
t.Fatalf("Generate called %d times, want %d", len(reqs), calls)
|
||||
}
|
||||
if g, _, _ := f.Remaining(); g != 0 {
|
||||
t.Fatalf("%d scripted Generate results unused", g)
|
||||
}
|
||||
for i, req := range reqs {
|
||||
if req.Temperature == nil || *req.Temperature != 0.7 {
|
||||
t.Fatalf("request %d temperature = %v, want 0.7", i+1, req.Temperature)
|
||||
}
|
||||
if req.Schema == nil || req.Schema.Name != "suggest_side_quests" {
|
||||
t.Fatalf("request %d schema = %v, want suggest_side_quests", i+1, req.Schema)
|
||||
}
|
||||
}
|
||||
return out, reqs, err
|
||||
}
|
||||
|
||||
func assertSideQuests(t *testing.T, got, want SideQuests) {
|
||||
t.Helper()
|
||||
if !reflect.DeepEqual(got, want) {
|
||||
t.Fatalf("output = %+v\nwant %+v", got, want)
|
||||
}
|
||||
}
|
||||
|
||||
func TestSuggestSideQuestsCleanBatch(t *testing.T) {
|
||||
var f llm.Fake
|
||||
f.QueueGenerate(reply(batch(shawarma, ragu, verde, bulgogi)))
|
||||
out, reqs, err := suggest(t, &f, household, 1)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
assertSideQuests(t, out, SideQuests{Suggestions: []SideQuest{wantShawarma, wantRagu, wantVerde, wantBulgogi}})
|
||||
|
||||
system, user := reqs[0].System, reqs[0].Messages[0].Content
|
||||
golden(t, "suggest_side_quests_clean.system.golden", system)
|
||||
golden(t, "suggest_side_quests_clean.user.golden", user)
|
||||
for _, part := range []string{
|
||||
"## Household context",
|
||||
"- Chicken Fajitas (Tex-Mex, stovetop, 30 min, favorite)",
|
||||
"### Quest log\n- Pad Thai",
|
||||
"### Passed, oldest first\n- Chicken Tikka Masala",
|
||||
"- 2026-10-06 Tue: Beef Chili",
|
||||
"### Pending Side quests\n- Shrimp Scampi",
|
||||
"## Output contract",
|
||||
`"cited_regulars"`,
|
||||
`"search_query"`,
|
||||
} {
|
||||
if !strings.Contains(system, part) {
|
||||
t.Errorf("system prompt lacks %q", part)
|
||||
}
|
||||
}
|
||||
if strings.Index(system, "## Household context") > strings.Index(system, "## Output contract") {
|
||||
t.Error("the output contract is not after the household context")
|
||||
}
|
||||
if user != "Suggest 4 Side quests." {
|
||||
t.Errorf("user message = %q, want twice the 2 needed", user)
|
||||
}
|
||||
}
|
||||
|
||||
func TestSuggestSideQuestsPersonaFirst(t *testing.T) {
|
||||
var f llm.Fake
|
||||
f.QueueGenerate(reply(batch(shawarma)))
|
||||
cfg := testConfig(t, &f)
|
||||
cfg.Persona = "You are the household's cheerful kitchen helper."
|
||||
if _, err := SuggestSideQuests(context.Background(), cfg, "2026-W41", household); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if system := f.Requests()[0].System; !strings.HasPrefix(system, cfg.Persona+"\n\n## Instructions\n") {
|
||||
t.Fatalf("system prompt does not start with the persona then the instructions:\n%s", system)
|
||||
}
|
||||
}
|
||||
|
||||
func TestSuggestSideQuestsInventedRegularDropped(t *testing.T) {
|
||||
invented := sideQuest("Meatloaf Wellington", "Your Grandma's Meatloaf, wrapped in pastry.", "Grandma's Meatloaf")
|
||||
partly := sideQuest("Fajita Meatballs", "Chicken Fajitas meets your Grandma's Meatloaf.", "Chicken Fajitas", "Grandma's Meatloaf")
|
||||
none := sideQuest("Mushroom Risotto", "Creamy and comforting.")
|
||||
var f llm.Fake
|
||||
f.QueueGenerate(reply(batch(invented, shawarma, partly, none)))
|
||||
out, _, err := suggest(t, &f, household, 1)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
assertSideQuests(t, out, SideQuests{
|
||||
Suggestions: []SideQuest{wantShawarma},
|
||||
Dropped: []DroppedSideQuest{
|
||||
{Name: "Meatloaf Wellington", Why: `cites "Grandma's Meatloaf", which is not a Regular`},
|
||||
{Name: "Fajita Meatballs", Why: `cites "Grandma's Meatloaf", which is not a Regular`},
|
||||
{Name: "Mushroom Risotto", Why: "cites no Regular"},
|
||||
},
|
||||
})
|
||||
}
|
||||
|
||||
func TestSuggestSideQuestsDuplicatesDropped(t *testing.T) {
|
||||
declined := sideQuest("The Best Homemade Chicken Tikka-Masala", "Spiced like your Chicken Fajitas.", "Chicken Fajitas")
|
||||
oldDeclined := sideQuest("Fish Tacos!", "Tortillas like your Chicken Fajitas.", "Chicken Fajitas")
|
||||
regular := sideQuest("Easy Beef Chili", "Just like your Beef Chili.", "Beef Chili")
|
||||
questLog := sideQuest("Pad Thai", "Noodles like your Spaghetti Bolognese.", "Spaghetti Bolognese")
|
||||
pending := sideQuest("Shrimp Scampi", "Pasta like your Spaghetti Bolognese.", "Spaghetti Bolognese")
|
||||
again := sideQuest("Chicken Shawarma Bowls", "Chicken Fajitas, but Middle Eastern.", "Chicken Fajitas")
|
||||
var f llm.Fake
|
||||
f.QueueGenerate(reply(batch(declined, oldDeclined, regular, questLog, pending, shawarma, again)))
|
||||
in := household
|
||||
in.Needed = 4
|
||||
out, _, err := suggest(t, &f, in, 1)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
known := "repeats a Regular, Quest log dinner, pending or passed Side quest"
|
||||
assertSideQuests(t, out, SideQuests{
|
||||
Suggestions: []SideQuest{wantShawarma},
|
||||
Dropped: []DroppedSideQuest{
|
||||
{Name: "The Best Homemade Chicken Tikka-Masala", Why: known},
|
||||
{Name: "Fish Tacos!", Why: known},
|
||||
{Name: "Easy Beef Chili", Why: known},
|
||||
{Name: "Pad Thai", Why: known},
|
||||
{Name: "Shrimp Scampi", Why: known},
|
||||
{Name: "Chicken Shawarma Bowls", Why: `repeats the Side quest "Chicken Shawarma Bowls" in the same batch`},
|
||||
},
|
||||
})
|
||||
}
|
||||
|
||||
func TestSuggestSideQuestsReasonRulesDropped(t *testing.T) {
|
||||
long := sideQuest("Beef Stroganoff", "Your Beef Chili"+strings.Repeat(" and more", 20)+".", "Beef Chili")
|
||||
unnamed := sideQuest("Chicken Satay", "Grilled skewers with a peanut sauce.", "Chicken Fajitas")
|
||||
wrongOne := sideQuest("Beef Tacos", "Like your Beef Chili, in a shell.", "Chicken Fajitas")
|
||||
var f llm.Fake
|
||||
f.QueueGenerate(reply(batch(long, unnamed, ragu, wrongOne)))
|
||||
out, _, err := suggest(t, &f, household, 1)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
assertSideQuests(t, out, SideQuests{
|
||||
Suggestions: []SideQuest{wantRagu},
|
||||
Dropped: []DroppedSideQuest{
|
||||
{Name: "Beef Stroganoff", Why: "reason has 196 characters, over 180"},
|
||||
{Name: "Chicken Satay", Why: "reason names none of its cited Regulars"},
|
||||
{Name: "Beef Tacos", Why: "reason names none of its cited Regulars"},
|
||||
},
|
||||
})
|
||||
}
|
||||
|
||||
func TestSuggestSideQuestsAllDroppedIsNotRepaired(t *testing.T) {
|
||||
var f llm.Fake
|
||||
f.QueueGenerate(reply(batch(sideQuest("Pad Thai", "Like your Beef Chili.", "Beef Chili"))))
|
||||
out, _, err := suggest(t, &f, household, 1)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
assertSideQuests(t, out, SideQuests{
|
||||
Suggestions: []SideQuest{},
|
||||
Dropped: []DroppedSideQuest{{Name: "Pad Thai", Why: "repeats a Regular, Quest log dinner, pending or passed Side quest"}},
|
||||
})
|
||||
}
|
||||
|
||||
func TestSuggestSideQuestsMalformedRepairedOnce(t *testing.T) {
|
||||
for name, tc := range map[string]struct{ first, want string }{
|
||||
"truncated JSON": {
|
||||
`{"suggestions":[` + shawarma + `,{"name":"Beef`,
|
||||
"reply does not decode: unexpected EOF",
|
||||
},
|
||||
"omitted field": {
|
||||
batch(strings.Replace(shawarma, `"cuisine":"Fusion",`, ``, 1)),
|
||||
`suggestion has no "cuisine" field; give every field the schema requires`,
|
||||
},
|
||||
"unknown field": {
|
||||
batch(strings.Replace(shawarma, `"easy",`, `"easy","spice":"mild",`, 1)),
|
||||
`suggestion has unknown field "spice"`,
|
||||
},
|
||||
"bad method": {
|
||||
batch(strings.Replace(shawarma, `"stovetop"`, `"air fryer"`, 1)),
|
||||
`suggestion 1 method "air fryer" is not one of stovetop, oven, crockpot, grill, other`,
|
||||
},
|
||||
} {
|
||||
t.Run(name, func(t *testing.T) {
|
||||
var f llm.Fake
|
||||
f.QueueGenerate(reply(tc.first), reply(batch(shawarma, ragu)))
|
||||
out, reqs, err := suggest(t, &f, household, 2)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
assertSideQuests(t, out, SideQuests{Suggestions: []SideQuest{wantShawarma, wantRagu}})
|
||||
repair := reqs[1].Messages
|
||||
if len(repair) != 3 || repair[1].Content != tc.first {
|
||||
t.Fatalf("repair turn does not quote the first reply: %+v", repair)
|
||||
}
|
||||
if !strings.Contains(repair[2].Content, tc.want) {
|
||||
t.Fatalf("repair message = %q, want it to contain %q", repair[2].Content, tc.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestSuggestSideQuestsMalformedTwiceFails(t *testing.T) {
|
||||
var f llm.Fake
|
||||
f.QueueGenerate(reply(`{"suggestions":`), reply(`{"suggestions":[]}`))
|
||||
out, _, err := suggest(t, &f, household, 2)
|
||||
var bad *harness.BadOutputError
|
||||
if !errors.As(err, &bad) || !errors.Is(err, llm.ErrBadOutput) {
|
||||
t.Fatalf("err = %v, want a BadOutputError", err)
|
||||
}
|
||||
if !slices.Equal(bad.Repair, []string{"suggestions is empty; give 4 Side quests"}) {
|
||||
t.Fatalf("repair problems = %q", bad.Repair)
|
||||
}
|
||||
if out.Suggestions != nil {
|
||||
t.Fatalf("output = %+v, want none", out)
|
||||
}
|
||||
}
|
||||
|
||||
func TestSideQuestsValidate(t *testing.T) {
|
||||
s := suggestSideQuests{in: household}
|
||||
if p := s.Validate(SideQuests{Suggestions: []SideQuest{wantShawarma}}); len(p) != 0 {
|
||||
t.Fatalf("valid output has problems: %q", p)
|
||||
}
|
||||
if p := s.Validate(SideQuests{Suggestions: []SideQuest{}, Dropped: []DroppedSideQuest{{Name: "x", Why: "y"}}}); len(p) != 0 {
|
||||
t.Fatalf("output whose Side quests were all dropped has problems: %q", p)
|
||||
}
|
||||
for name, tc := range map[string]struct {
|
||||
out SideQuests
|
||||
want []string
|
||||
}{
|
||||
"empty": {SideQuests{}, []string{"suggestions is empty; give 4 Side quests"}},
|
||||
"too many": {
|
||||
SideQuests{Suggestions: []SideQuest{wantShawarma, wantRagu, wantVerde, wantBulgogi, wantShawarma}},
|
||||
[]string{"suggestions has 5 entries; give at most 4 Side quests"},
|
||||
},
|
||||
"malformed": {
|
||||
SideQuests{Suggestions: []SideQuest{{Name: " ", Method: "air fryer", EstMinutes: 0, Difficulty: "trivial", CitedRegulars: []string{}}}},
|
||||
[]string{
|
||||
"suggestion 1 has an empty name; give one",
|
||||
"suggestion 1 has an empty cuisine; give one",
|
||||
"suggestion 1 has an empty reason; give one",
|
||||
"suggestion 1 has an empty search_query; give one",
|
||||
`suggestion 1 method "air fryer" is not one of stovetop, oven, crockpot, grill, other`,
|
||||
"suggestion 1 est_minutes 0 is not positive; give the minutes from prep to table",
|
||||
`suggestion 1 difficulty "trivial" is not one of easy, medium, hard`,
|
||||
},
|
||||
},
|
||||
} {
|
||||
t.Run(name, func(t *testing.T) {
|
||||
if got := s.Validate(tc.out); !slices.Equal(got, tc.want) {
|
||||
t.Fatalf("problems =\n%s\nwant\n%s", strings.Join(got, "\n"), strings.Join(tc.want, "\n"))
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestSideQuestsPromptInputRejects(t *testing.T) {
|
||||
noRegulars := household
|
||||
noRegulars.Context.Regulars = nil
|
||||
none := household
|
||||
none.Needed = 0
|
||||
for name, tc := range map[string]struct {
|
||||
in SideQuestsRequest
|
||||
want string
|
||||
}{
|
||||
"no Regulars": {noRegulars, "household has no Regulars for Side quests to be based on"},
|
||||
"none needed": {none, "0 Side quests needed; the number must be positive"},
|
||||
} {
|
||||
t.Run(name, func(t *testing.T) {
|
||||
_, err := suggestSideQuests{in: tc.in}.PromptInput(tc.in)
|
||||
if err == nil || err.Error() != tc.want {
|
||||
t.Fatalf("err = %v, want %q", err, tc.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,110 @@
|
||||
## Instructions
|
||||
You suggest Side quests: new dinners for the household to try, based on the dinners it already makes.
|
||||
|
||||
Every Side quest must be a dinner the household has not had: never one of its Regulars, its Quest log, its pending Side quests or the dinners it has passed, and never one of those under another name. Base every Side quest on one or more of the household's Regulars, such as a familiar protein in a new cuisine or familiar flavours cooked a new way. Prefer the favorite Regulars, and vary the cuisines across the batch.
|
||||
|
||||
Give for each Side quest:
|
||||
- name: the dinner's usual name, as a recipe site would title it.
|
||||
- cuisine: its cuisine, e.g. "Thai" or "Tex-Mex".
|
||||
- method: how it is mainly cooked: "stovetop", "oven", "crockpot" (slow cooker or crock pot), "grill" or "other".
|
||||
- est_minutes: your estimate of the whole minutes from starting prep to the table.
|
||||
- difficulty: "easy", "medium" or "hard" for a home cook.
|
||||
- reason: one sentence of at most 180 characters saying why the household would like it. It must name at least one of the Regulars in cited_regulars.
|
||||
- cited_regulars: the Regulars the reason relies on, at least one, each written exactly as its name appears in the Regulars list of the household context, without the details in parentheses. Cite only Regulars from that list.
|
||||
- search_query: a web search query that finds a recipe for this dinner.
|
||||
|
||||
One valid Side quest, for a household with a Regular named "Chicken Fajitas" (an example only, not about this household):
|
||||
{"name": "Chicken Shawarma Bowls", "cuisine": "Middle Eastern", "method": "oven", "est_minutes": 40, "difficulty": "easy", "reason": "Spiced roast chicken and crisp toppings, like your Chicken Fajitas with a Middle Eastern twist.", "cited_regulars": ["Chicken Fajitas"], "search_query": "chicken shawarma bowls recipe"}
|
||||
|
||||
## Household context
|
||||
### Today
|
||||
- Tuesday 2026-10-06 (America/New_York)
|
||||
|
||||
### Regulars
|
||||
- Beef Chili (Tex-Mex, crockpot, 240 min)
|
||||
- Chicken Fajitas (Tex-Mex, stovetop, 30 min, favorite)
|
||||
- Spaghetti Bolognese (Italian, stovetop, 45 min)
|
||||
|
||||
### Quest log
|
||||
- Pad Thai
|
||||
|
||||
### Passed, oldest first
|
||||
- Chicken Tikka Masala
|
||||
|
||||
### Recent plans
|
||||
- 2026-10-05 Mon: Chicken Fajitas
|
||||
- 2026-10-06 Tue: Beef Chili
|
||||
|
||||
### Pending Side quests
|
||||
- Shrimp Scampi
|
||||
|
||||
## Output contract
|
||||
This contract overrides any earlier instruction about the form of your reply. Reply with one JSON value matching the JSON Schema "suggest_side_quests" below, and nothing else: no prose and no Markdown code fences.
|
||||
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"suggestions": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"name": {
|
||||
"type": "string"
|
||||
},
|
||||
"cuisine": {
|
||||
"type": "string"
|
||||
},
|
||||
"method": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"stovetop",
|
||||
"oven",
|
||||
"crockpot",
|
||||
"grill",
|
||||
"other"
|
||||
]
|
||||
},
|
||||
"est_minutes": {
|
||||
"type": "integer"
|
||||
},
|
||||
"difficulty": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"easy",
|
||||
"medium",
|
||||
"hard"
|
||||
]
|
||||
},
|
||||
"reason": {
|
||||
"type": "string"
|
||||
},
|
||||
"cited_regulars": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "string"
|
||||
}
|
||||
},
|
||||
"search_query": {
|
||||
"type": "string"
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"name",
|
||||
"cuisine",
|
||||
"method",
|
||||
"est_minutes",
|
||||
"difficulty",
|
||||
"reason",
|
||||
"cited_regulars",
|
||||
"search_query"
|
||||
],
|
||||
"additionalProperties": false
|
||||
}
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"suggestions"
|
||||
],
|
||||
"additionalProperties": false
|
||||
}
|
||||
@@ -0,0 +1 @@
|
||||
Suggest 4 Side quests.
|
||||
Reference in New Issue
Block a user