ai_runs logging, redaction and retention #60

Merged

approver_agent merged 2 commits from task/ai-runs-logging_53e382d8-1db3-4555-a31d-e44cf55cc98d into feature/d4a8ed83-9cee-459c-aca3-efd1ccf17ecd-ai-harness

2026-10-05 09:55:06 -04:00

Author SHA1 Message Date

Author	SHA1	Message	Date
dev_agentandClaude Opus 5.5	6a7c040657	Build the run's provider from the key it redacts; record render failures CI / ci (pull_request) Successful in 1m33s Details CI / ci (push) Successful in 1m33s Details Review findings on `b0cf6fb`: 1. Config took a ready-made Provider and a separate, optional APIKey used only for redaction, so a provider holding a decrypted key could run with APIKey empty and a reply quoting that key unlabelled was stored. Config now holds the household's llm.Settings, the decrypted APIKey and a NewProvider constructor (presets.FromSettings in production). Run builds the provider from exactly that key and redacts the same key, so the two cannot differ. ProviderName, Model and Temperature come from Settings. A new test runs presets.FromSettings against a server that echoes the bearer key unlabelled in the reply text, reply JSON and model name, across a repair turn, and asserts the stored row never holds it; with redaction disabled the test fails. 2. A render failure wrote no ai_runs row. Every run with a valid Config now writes one: a render failure is bad_output and a provider that cannot be built is provider_error, each with the error text and empty request and response lists. TestRunRenderError now expects exactly one such row. A Config error (no store, provider name or model) still writes none, since the row cannot be written without them. Redaction rules, the four statuses for provider-reaching runs (one row, both turns) and Purge are unchanged. Maestro task 53e382d8-1db3-4555-a31d-e44cf55cc98d. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-05 09:43:24 -04:00
dev_agentandClaude Opus 5.5	b0cf6fb951	Record every harness run in ai_runs with redaction and purge CI / ci (push) Successful in 1m35s Details CI / ci (pull_request) Successful in 1m32s Details harness.Run now writes one ai_runs row per run that reaches the provider, covering the repair turn: skill, version, provider, model, temperature, status (ok, repaired, bad_output, provider_error), latency, summed token counts when reported, error text and the per-turn request and response bodies. Every stored string is redacted first: the decrypted API key, Authorization and API-key header values, Google Chat webhook URLs and URLs with key-like query or fragment parameters or user info. JSON is decoded and its strings redacted, so escapes cannot hide a credential. A row that cannot be written fails the run. Purge deletes rows created before a cutoff (RunRetention is 30 days), clearing suggestion_batches and week_plans references in the same transaction. Migration 0006 rebuilds ai_runs with the new status CHECK and a created_at index; a pre-harness running or error row stops it. Maestro task 53e382d8-1db3-4555-a31d-e44cf55cc98d. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-05 09:32:20 -04:00

dev_agentandClaude Opus 5.5

6a7c040657

Build the run's provider from the key it redacts; record render failures

CI / ci (pull_request) Successful in 1m33s

Details

CI / ci (push) Successful in 1m33s

Details

Review findings on b0cf6fb:

1. Config took a ready-made Provider and a separate, optional APIKey used
   only for redaction, so a provider holding a decrypted key could run
   with APIKey empty and a reply quoting that key unlabelled was stored.
   Config now holds the household's llm.Settings, the decrypted APIKey and
   a NewProvider constructor (presets.FromSettings in production). Run
   builds the provider from exactly that key and redacts the same key, so
   the two cannot differ. ProviderName, Model and Temperature come from
   Settings. A new test runs presets.FromSettings against a server that
   echoes the bearer key unlabelled in the reply text, reply JSON and
   model name, across a repair turn, and asserts the stored row never
   holds it; with redaction disabled the test fails.

2. A render failure wrote no ai_runs row. Every run with a valid Config
   now writes one: a render failure is bad_output and a provider that
   cannot be built is provider_error, each with the error text and empty
   request and response lists. TestRunRenderError now expects exactly one
   such row. A Config error (no store, provider name or model) still
   writes none, since the row cannot be written without them.

Redaction rules, the four statuses for provider-reaching runs (one row,
both turns) and Purge are unchanged.

Maestro task 53e382d8-1db3-4555-a31d-e44cf55cc98d.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

2026-10-05 09:43:24 -04:00

dev_agentandClaude Opus 5.5

b0cf6fb951

Record every harness run in ai_runs with redaction and purge

CI / ci (push) Successful in 1m35s

Details

CI / ci (pull_request) Successful in 1m32s

Details

harness.Run now writes one ai_runs row per run that reaches the provider,
covering the repair turn: skill, version, provider, model, temperature,
status (ok, repaired, bad_output, provider_error), latency, summed token
counts when reported, error text and the per-turn request and response
bodies. Every stored string is redacted first: the decrypted API key,
Authorization and API-key header values, Google Chat webhook URLs and
URLs with key-like query or fragment parameters or user info. JSON is
decoded and its strings redacted, so escapes cannot hide a credential.
A row that cannot be written fails the run.

Purge deletes rows created before a cutoff (RunRetention is 30 days),
clearing suggestion_batches and week_plans references in the same
transaction.

Migration 0006 rebuilds ai_runs with the new status CHECK and a
created_at index; a pre-harness running or error row stops it.

Maestro task 53e382d8-1db3-4555-a31d-e44cf55cc98d.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

2026-10-05 09:32:20 -04:00