Run the missed sweep in the API and publish quest-missed on SSE
WHY: The nightly missed-quest sweep only existed on the Home Assistant plane (the rollover listener fired hass.bus), so the API service - now the writer of record with its own settings store and SSE stream - had no way to announce missed quests, and an admin could not trigger a catch-up run after downtime. - Extract the sweep policy (watermark read, list_missed_unswept query, documented payload build, watermark write, per-database lock) into the HA-free core (core/sweep.py), building each event through the shared events payload builder so the shape cannot drift; the integration's sweep.py becomes a thin shim that fires the returned events on hass.bus, keeping its signature. - Admin trigger POST /api/v1/admin/missed-sweep: ONE clock read, ONE core call, each returned event published on the SSE stream through the existing publish_quest_missed helper; returns the fired count. Watermark makes a same-day rerun silent and a post-downtime run sweep the accumulated window once. - Daily scheduler (api/scheduler.py) wired in the lifespan: reads day_rollover_time fresh each cycle, runs the sweep at the rollover, publishes on the app publisher, backs off and logs on failure, cancels cleanly on shutdown; injectable clock/sleep for tests. - Tests: route trigger over a live uvicorn server with a live SSE stream (one transition per instance with documented fields, same-day rerun silent, post-downtime catch-up once, non-admin 403 / absent 401) plus the scheduler rollover/cancellation test.
This commit is contained in:
@@ -0,0 +1,143 @@
|
||||
"""The nightly missed-quest sweep, Home-Assistant-free (Feature 11).
|
||||
|
||||
This is the HA-free heart of the missed sweep, shared by the two
|
||||
planes: the Home Assistant integration's :mod:`~..sweep` shim (which
|
||||
fires the returned events on ``hass.bus``) and the API service (which
|
||||
publishes them on the SSE stream and can also trigger the run on
|
||||
demand). It owns the ONE sweep policy:
|
||||
|
||||
At the configured day-rollover time (or on an explicit trigger) this
|
||||
job finds every instance whose ``due_date`` is before the
|
||||
household-local date, that has NO completion event, and that falls
|
||||
after the persisted sweep watermark — and RETURNS one
|
||||
``nestquest_quest_missed`` event per instance with the documented
|
||||
payload. Like every core module, it never touches the Home Assistant
|
||||
event bus and never mutates an instance: ``missed`` is derived state,
|
||||
never an event_type (Feature 08's contract), so the sweep is read-only
|
||||
over the domain tables.
|
||||
|
||||
Idempotency is the watermark: after a run, ``nestquest_meta_state``
|
||||
records the household-local date the sweep ran, and the next run only
|
||||
sweeps instances due in ``[watermark, today)``. A second run the same
|
||||
night therefore fires nothing new, and a run after days of downtime
|
||||
sweeps exactly the accumulated window once — no per-instance journal,
|
||||
no re-announcing old misses. An instance swept as missed that is
|
||||
later completed simply gains its completion event; history keeps both
|
||||
rows (append-only), and the sweep's later runs skip it via the
|
||||
no-completion-event rule.
|
||||
|
||||
The whole watermark read → query → build → write span runs inside the
|
||||
per-database sweep lock (one lock per connection wrapper and running
|
||||
loop, the settings store's pattern): an overlapping startup sweep and
|
||||
a rollover sweep queue instead of interleaving, and the loser re-reads
|
||||
the watermark INSIDE the lock — sees the winner's date — and announces
|
||||
nothing twice.
|
||||
|
||||
The caller supplies ``today`` as a plain :class:`datetime.date`: the
|
||||
household-local calendar date, read from ``hass.config.time_zone`` by
|
||||
the integration or from the API host's local clock by the API. This
|
||||
module never reads a timezone database or config of its own.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import datetime
|
||||
import logging
|
||||
|
||||
from .const import EVENT_QUEST_MISSED
|
||||
from .dao_meta import MetaStateDao
|
||||
from .dao_instances import QuestInstancesDao
|
||||
from .db import NestQuestDatabase
|
||||
# Intentional intra-package reuse of the shared payload builder: the
|
||||
# missed event's payload is the documented §3 instance payload, and
|
||||
# building it through the ONE builder keeps this sweep's payloads from
|
||||
# drifting from the completion/uncompleted builders' shape.
|
||||
from .events import _instance_payload
|
||||
|
||||
LOGGER = logging.getLogger(__name__)
|
||||
|
||||
#: meta_state key holding the household-local date the sweep last ran.
|
||||
SWEEP_WATERMARK_KEY = "missed_sweep_last_run_date"
|
||||
|
||||
#: One asyncio.Lock per (connection wrapper, running loop), mirroring
|
||||
#: the settings store's pattern: the whole watermark read → query →
|
||||
#: build → write span is one critical section, so an overlapping
|
||||
#: startup sweep and rollover sweep queue instead of double-announcing.
|
||||
#: The loser re-reads the watermark INSIDE the lock, sees the winner's
|
||||
#: date, and fires nothing.
|
||||
_SWEEP_LOCKS: dict[tuple[int, int], asyncio.Lock] = {}
|
||||
|
||||
|
||||
def _sweep_lock(database: NestQuestDatabase) -> asyncio.Lock:
|
||||
"""Return the sweep lock bound to ``database``'s running loop."""
|
||||
loop = asyncio.get_running_loop()
|
||||
key = (id(database), id(loop))
|
||||
lock = _SWEEP_LOCKS.get(key)
|
||||
if lock is None:
|
||||
lock = asyncio.Lock()
|
||||
_SWEEP_LOCKS[key] = lock
|
||||
return lock
|
||||
|
||||
|
||||
#: Strict UTC ISO-8601 timestamp (the DAO layers' policy): one shape,
|
||||
#: explicit offset, second precision.
|
||||
_UTC_TIMESTAMP_FORMAT = "%Y-%m-%dT%H:%M:%S+00:00"
|
||||
|
||||
|
||||
async def run_missed_sweep(
|
||||
database: NestQuestDatabase,
|
||||
*,
|
||||
today: datetime.date,
|
||||
) -> list[tuple[str, dict]]:
|
||||
"""Build one missed event per unswept past-due open instance.
|
||||
|
||||
``today`` is the household-local calendar date the sweep runs for
|
||||
(the caller owns the clock: the integration reads it from
|
||||
``hass.config.time_zone``, the API from its host's local time).
|
||||
Returns the ``(event_type, payload)`` tuples to announce, EMPTY
|
||||
when the watermark already covers ``today`` (a same-night rerun) —
|
||||
the caller decides how to publish them and announces nothing on
|
||||
its own.
|
||||
|
||||
The whole watermark read → query → build → write span runs inside
|
||||
the per-database sweep lock: an overlapping startup sweep and a
|
||||
rollover sweep queue instead of interleaving, and the loser
|
||||
re-reads the watermark inside the lock — sees the winner's date —
|
||||
and announces nothing twice. Every event in one run shares one
|
||||
``occurred_at`` stamp (one run, one announcement moment), and the
|
||||
payload comes from the shared
|
||||
:func:`~.events._instance_payload` builder so the shape can never
|
||||
drift from the documented §3 contract.
|
||||
"""
|
||||
today_iso = today.isoformat()
|
||||
dao = MetaStateDao(database)
|
||||
async with _sweep_lock(database):
|
||||
watermark = await dao.get(SWEEP_WATERMARK_KEY)
|
||||
if watermark == today_iso:
|
||||
LOGGER.debug(
|
||||
"NestQuest missed sweep already ran for %s", today_iso
|
||||
)
|
||||
return []
|
||||
|
||||
instances_dao = QuestInstancesDao(database)
|
||||
rows = await instances_dao.list_missed_unswept(today_iso, watermark)
|
||||
now_stamp = datetime.datetime.now(datetime.timezone.utc).strftime(
|
||||
_UTC_TIMESTAMP_FORMAT
|
||||
)
|
||||
events: list[tuple[str, dict]] = []
|
||||
for instance in rows:
|
||||
events.append(
|
||||
(
|
||||
EVENT_QUEST_MISSED,
|
||||
await _instance_payload(
|
||||
database, instance.id, now_stamp
|
||||
),
|
||||
)
|
||||
)
|
||||
await dao.set(SWEEP_WATERMARK_KEY, today_iso)
|
||||
LOGGER.info(
|
||||
"NestQuest missed sweep for %s built %d event(s)",
|
||||
today_iso,
|
||||
len(events),
|
||||
)
|
||||
return events
|
||||
@@ -1,70 +1,40 @@
|
||||
"""The nightly missed-quest sweep (Feature 11).
|
||||
"""The nightly missed-quest sweep (Feature 11) — the Home Assistant shim.
|
||||
|
||||
At the configured day-rollover time (the same HA-local listener that
|
||||
re-materializes the rolling horizon) this job finds every instance
|
||||
whose ``due_date`` is before the HA-local date, that has NO completion
|
||||
event, and that falls after the persisted sweep watermark — and fires
|
||||
ONE ``nestquest_quest_missed`` event per instance with the documented
|
||||
payload. The sweep is read-only over the domain tables: it never
|
||||
mutates an instance and never writes a completion event (``missed``
|
||||
is derived state, never an event_type — Feature 08's contract).
|
||||
The sweep's policy is the HA-free core's :mod:`~.core.sweep`: the
|
||||
watermark read → ``list_missed_unswept`` query → payload build →
|
||||
watermark span, all inside the per-database sweep lock, shared with
|
||||
the API plane. This module is the integration's thin adapter over it:
|
||||
at the configured day-rollover time (the same HA-local listener that
|
||||
re-materializes the rolling horizon) and once at setup it calls the
|
||||
core's :func:`~.core.sweep.run_missed_sweep` and fires each returned
|
||||
event on ``hass.bus`` — ONE ``nestquest_quest_missed`` per past-due
|
||||
open instance with the documented payload. The core never touches
|
||||
Home Assistant; the firing lives here.
|
||||
|
||||
Idempotency is the watermark: after a run, ``nestquest_meta_state``
|
||||
records the HA-local date the sweep ran, and the next run only
|
||||
sweeps instances due in ``[watermark, today)``. A second run the
|
||||
same night therefore fires nothing new, and a run after days of
|
||||
downtime sweeps exactly the accumulated window once — no
|
||||
per-instance journal, no re-announcing old misses. An instance swept
|
||||
as missed that is later completed simply gains its completion event;
|
||||
history keeps both rows (append-only), and the sweep's later runs
|
||||
skip it via the no-completion-event rule.
|
||||
Idempotency is the core's watermark: after a run, ``nestquest_meta_state``
|
||||
records the HA-local date the sweep ran, and the next run only sweeps
|
||||
instances due in ``[watermark, today)`` — a second run the same night
|
||||
fires nothing new, and a run after days of downtime sweeps exactly the
|
||||
accumulated window once. An instance swept as missed that is later
|
||||
completed simply gains its completion event; the sweep is read-only
|
||||
over the domain tables (``missed`` is derived state, never an
|
||||
event_type — Feature 08's contract).
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import datetime
|
||||
import logging
|
||||
from zoneinfo import ZoneInfo
|
||||
|
||||
from .const import EVENT_QUEST_MISSED
|
||||
from .dao_children import ChildrenDao
|
||||
from .dao_instances import QuestInstancesDao
|
||||
from .dao_meta import MetaStateDao
|
||||
from .dao_rules import QuestDefinitionsDao
|
||||
from .core.sweep import (
|
||||
SWEEP_WATERMARK_KEY,
|
||||
run_missed_sweep as _run_core_missed_sweep,
|
||||
)
|
||||
from .db import NestQuestDatabase
|
||||
|
||||
LOGGER = logging.getLogger(__name__)
|
||||
|
||||
#: meta_state key holding the HA-local date the sweep last ran.
|
||||
SWEEP_WATERMARK_KEY = "missed_sweep_last_run_date"
|
||||
|
||||
#: One asyncio.Lock per (connection wrapper, running loop), mirroring
|
||||
#: the migration runner's pattern: the whole watermark read → query →
|
||||
#: fire → write span is one critical section, so an overlapping startup
|
||||
#: sweep and rollover sweep queue instead of double-announcing. The
|
||||
#: loser re-reads the watermark INSIDE the lock, sees the winner's
|
||||
#: date, and fires nothing.
|
||||
_SWEEP_LOCKS: dict[tuple[int, int], asyncio.Lock] = {}
|
||||
|
||||
|
||||
def _sweep_lock(database: NestQuestDatabase) -> asyncio.Lock:
|
||||
"""Return the sweep lock bound to ``database``'s running loop."""
|
||||
loop = asyncio.get_running_loop()
|
||||
key = (id(database), id(loop))
|
||||
lock = _SWEEP_LOCKS.get(key)
|
||||
if lock is None:
|
||||
lock = asyncio.Lock()
|
||||
_SWEEP_LOCKS[key] = lock
|
||||
return lock
|
||||
|
||||
#: UTC ISO-8601 timestamp policy shared with the DAO layers.
|
||||
_UTC_TIMESTAMP_FORMAT = "%Y-%m-%dT%H:%M:%S+00:00"
|
||||
|
||||
|
||||
def _now_stamp() -> str:
|
||||
return datetime.datetime.now(datetime.timezone.utc).strftime(
|
||||
_UTC_TIMESTAMP_FORMAT
|
||||
)
|
||||
__all__ = ["SWEEP_WATERMARK_KEY", "run_missed_sweep"]
|
||||
|
||||
|
||||
async def run_missed_sweep(
|
||||
@@ -79,54 +49,24 @@ async def run_missed_sweep(
|
||||
materialization and completion layers); when omitted it is read
|
||||
from ``hass.config.time_zone``. Returns the number of events
|
||||
fired. Fires nothing and returns 0 when the watermark already
|
||||
covers ``today`` (a same-night rerun).
|
||||
|
||||
The whole watermark read → query → fire → write span runs inside
|
||||
the per-database sweep lock: an overlapping startup sweep and a
|
||||
rollover sweep queue instead of interleaving, and the loser
|
||||
re-reads the watermark inside the lock — sees the winner's date —
|
||||
and announces nothing twice.
|
||||
covers ``today`` (a same-night rerun) — the watermark read, the
|
||||
query, the payload build and the watermark write all live in the
|
||||
core's locked
|
||||
:func:`~.core.sweep.run_missed_sweep`; this shim only fires each
|
||||
returned event on the bus and counts them.
|
||||
"""
|
||||
today_date = (
|
||||
today
|
||||
if today is not None
|
||||
else datetime.datetime.now(ZoneInfo(hass.config.time_zone)).date()
|
||||
)
|
||||
today_iso = today_date.isoformat()
|
||||
dao = MetaStateDao(database)
|
||||
async with _sweep_lock(database):
|
||||
watermark = await dao.get(SWEEP_WATERMARK_KEY)
|
||||
if watermark == today_iso:
|
||||
LOGGER.debug(
|
||||
"NestQuest missed sweep already ran for %s", today_iso
|
||||
)
|
||||
return 0
|
||||
|
||||
instances_dao = QuestInstancesDao(database)
|
||||
rows = await instances_dao.list_missed_unswept(today_iso, watermark)
|
||||
children_dao = ChildrenDao(database)
|
||||
definitions_dao = QuestDefinitionsDao(database)
|
||||
now_stamp = _now_stamp()
|
||||
fired = 0
|
||||
for instance in rows:
|
||||
child = await children_dao.get(instance.child_id)
|
||||
definition = await definitions_dao.get(instance.definition_id)
|
||||
hass.bus.async_fire(
|
||||
EVENT_QUEST_MISSED,
|
||||
{
|
||||
"child_id": instance.child_id,
|
||||
"child_name": child.display_name if child else None,
|
||||
"instance_id": instance.id,
|
||||
"quest_title": definition.title if definition else None,
|
||||
"window": instance.window,
|
||||
"due_date": instance.due_date,
|
||||
"due_time": instance.due_time,
|
||||
"occurred_at": now_stamp,
|
||||
},
|
||||
)
|
||||
fired += 1
|
||||
await dao.set(SWEEP_WATERMARK_KEY, today_iso)
|
||||
events = await _run_core_missed_sweep(database, today=today_date)
|
||||
for event_type, payload in events:
|
||||
hass.bus.async_fire(event_type, payload)
|
||||
fired = len(events)
|
||||
LOGGER.info(
|
||||
"NestQuest missed sweep for %s fired %d event(s)", today_iso, fired
|
||||
"NestQuest missed sweep for %s fired %d event(s)",
|
||||
today_date.isoformat(),
|
||||
fired,
|
||||
)
|
||||
return fired
|
||||
Reference in New Issue
Block a user