Maniacs_Mansion/NestQuest
11
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix: single coherent input snapshot and skip-invalidated-tuple (NQ-75-001, NQ-75-002)

Browse Source
This commit is contained in:
overseer
2026-09-17 13:43:42 -04:00
parent b2ee502946
commit cc418a997c
4 changed files with 401 additions and 70 deletions
Download Patch File Download Diff File Expand all files Collapse all files
custom_components/nestquest/materialize.py
+57 -32
View File
@@ -8,13 +8,14 @@ generation are separate later tasks and live elsewhere.
The walk is deterministic and idempotent:
- Every active definition is read with its decoded
:class:`~.recurrence.ScheduleRule`, assignees and windows through the
Feature 06 snapshot helpers, so each bundle is a coherent read.
- All inputs — the active definitions with their rules, assignees and
windows, AND the assignee children's presence schedules and scoped
overrides — are read inside ONE locked transaction (see
:func:`~.dao_rules.load_materialization_input`), so the walk sees a
single coherent snapshot rather than a definition state from one
instant and a presence state from another.
- Presence is snapshotted into an immutable
:class:`~.presence.PresenceEngine` from the schedules and overrides of
every assignee child, read in one locked transaction (see
:func:`~.dao_presence.load_snapshot`).
:class:`~.presence.PresenceEngine` from that same coherent read.
- For each date in the range, each definition whose decoded rule
:func:`~.recurrence.occurs_on` on that date, each assignee who is
``is_present`` on that date and each declared window, one instance is
@@ -22,6 +23,13 @@ The walk is deterministic and idempotent:
the window's ``due_time`` snapshotted and a ``generated_at`` stamp
shared by the whole batch (one coherent time per run).
A config change that lands AFTER the snapshot is read but BEFORE a
tuple's upsert (an assignment removed, a definition or child
deactivated, a window removed) makes that tuple SKIP rather than abort
the batch: each tuple is re-validated against the current state right
before its upsert, and only the narrow "no longer valid" conditions are
treated as skippable.
The upsert is idempotent on (definition_id, child_id, due_date, window),
so re-running the materialization over the same range never duplicates a
row; skipped-completed handling is deliberately NOT done here.
@@ -31,10 +39,10 @@ from __future__ import annotations
import datetime
from .dao_instances import QuestInstancesDao
from .dao_presence import load_snapshot
from .dao_rules import (
QuestDefinitionsDao,
_validate_date,
load_materialization_input,
schedule_rule_from_storage,
schedule_rule_storage_from_record,
)
@@ -88,6 +96,22 @@ def _build_engine(
return PresenceEngine(schedules, overrides)
def _is_deleted_tuple(error: ValueError) -> bool:
"""True for the narrow 'tuple no longer valid' upsert rejections.
A change landing between the per-tuple validity re-check and the
upsert can still surface as the upsert's "definition does not exist"
or "not assigned to child" error; those are skippable. Every other
rejection (past due date, immutable completed instance, bad window)
is a real error and propagates.
"""
message = str(error)
return (
"does not exist" in message
or "is not assigned to child" in message
)
async def materialize(
database: NestQuestDatabase,
start_date: str,
@@ -98,13 +122,13 @@ async def materialize(
``start_date`` and ``end_date`` must be strict ISO calendar dates
(YYYY-MM-DD) with ``end_date`` on or after ``start_date``; anything
else raises ValueError naming the offending field. Returns the number
of instances upserted (idempotent — a re-run for the same batch
refreshes the same rows, never duplicates them).
of instances upserted for the range (idempotent — a re-run refreshes
the same rows, never duplicates them).
The active definitions and the presence inputs are each read as a
coherent snapshot, and the presence schedule/overrides are read under
one locked transaction so the walk sees a single stable view of who
is where on each date.
All inputs are read in ONE locked transaction, and each tuple is
re-validated against the current state just before its upsert so a
config change that arrives mid-batch skips the affected tuples rather
than aborting the batch or leaving a partial result.
"""
_validate_date(start_date, "start_date")
_validate_date(end_date, "end_date")
@@ -114,17 +138,8 @@ async def materialize(
f"{end_date!r} < {start_date!r}"
)
snapshots = await QuestDefinitionsDao(database).list_snapshots_active()
assignee_child_ids = sorted(
{
child.id
for snapshot in snapshots
for child in snapshot.assignees
}
)
schedules_records, overrides_records = await load_snapshot(
database, assignee_child_ids, start_date, end_date
snapshots, schedules_records, overrides_records = await (
load_materialization_input(database, start_date, end_date)
)
engine = _build_engine(schedules_records, overrides_records)
@@ -136,6 +151,7 @@ async def materialize(
generated_at = _now_stamp()
instances = QuestInstancesDao(database)
definitions_dao = QuestDefinitionsDao(database)
start = datetime.datetime.strptime(start_date, "%Y-%m-%d").date()
end = datetime.datetime.strptime(end_date, "%Y-%m-%d").date()
@@ -151,14 +167,23 @@ async def materialize(
if not engine.is_present(child.id, cursor):
continue
for window in windows:
await instances.upsert(
definition_id,
child.id,
iso,
generated_at,
window=window.window,
due_time=window.due_time,
)
if not await definitions_dao.still_materializable(
definition_id, child.id, window.window
):
continue
try:
await instances.upsert(
definition_id,
child.id,
iso,
generated_at,
window=window.window,
due_time=window.due_time,
)
except ValueError as error:
if _is_deleted_tuple(error):
continue
raise
count += 1
cursor += datetime.timedelta(days=1)
return count