Build the run's provider from the key it redacts; record render failures
Review findings on b0cf6fb:
1. Config took a ready-made Provider and a separate, optional APIKey used
only for redaction, so a provider holding a decrypted key could run
with APIKey empty and a reply quoting that key unlabelled was stored.
Config now holds the household's llm.Settings, the decrypted APIKey and
a NewProvider constructor (presets.FromSettings in production). Run
builds the provider from exactly that key and redacts the same key, so
the two cannot differ. ProviderName, Model and Temperature come from
Settings. A new test runs presets.FromSettings against a server that
echoes the bearer key unlabelled in the reply text, reply JSON and
model name, across a repair turn, and asserts the stored row never
holds it; with redaction disabled the test fails.
2. A render failure wrote no ai_runs row. Every run with a valid Config
now writes one: a render failure is bad_output and a provider that
cannot be built is provider_error, each with the error text and empty
request and response lists. TestRunRenderError now expects exactly one
such row. A Config error (no store, provider name or model) still
writes none, since the row cannot be written without them.
Redaction rules, the four statuses for provider-reaching runs (one row,
both turns) and Purge are unchanged.
Maestro task 53e382d8-1db3-4555-a31d-e44cf55cc98d.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -150,9 +150,13 @@ automatic** (D-021).
|
||||
Anthropic and Ollama endpoints, called over plain HTTP — no vendor SDKs.
|
||||
- **Skill harness** (`ai/harness`): prompt layering, validation of the
|
||||
structured output, and **one repair retry**; if the repair also fails the
|
||||
run fails — no silent fallback. Every run that reaches the provider writes
|
||||
one `ai_runs` row when it ends, covering the repair turn too, with status
|
||||
`ok`, `repaired`, `bad_output` or `provider_error`. Request and response
|
||||
run fails — no silent fallback. Every run writes one `ai_runs` row when it
|
||||
ends, covering the repair turn too, with status `ok`, `repaired`,
|
||||
`bad_output` or `provider_error`; a prompt that cannot be rendered is
|
||||
`bad_output` and a provider that cannot be built is `provider_error`, both
|
||||
with no turns. The harness builds the provider itself from the settings
|
||||
and the decrypted API key (`presets.FromSettings`), so the key it redacts
|
||||
is always the key the provider sends. Request and response
|
||||
bodies are redacted before they are stored: the decrypted API key,
|
||||
Authorization and API-key header values, Google Chat webhook URLs and URLs
|
||||
with key-like query parameters are removed, inside JSON strings too.
|
||||
|
||||
Reference in New Issue
Block a user