Bound the suggestion batch's end write like a job's final write
runBatch recorded the batch's end with an uncancellable context and no deadline, before its handler returned, so a writer connection that stayed busy could hold the worker, and shutdown, open indefinitely. jobs.FinalWriteContext now gives every final write one bounded attempt context (FinalWriteTimeout, 5 seconds, not cancelled with ctx); the runner's record and finishBatch both use it. On timeout at shutdown the handler returns, the job and batch stay running, the next Start fails the job as interrupted and the next StartBatch fails the orphaned batch. TestBlockedBatchEndDoesNotHoldShutdown holds the writer while a batch reaches finishBatch and asserts the worker stops within the bound and the batch is reconciled after restart. TestBlockedFinalWriteDoesNotHoldShutdown no longer sets a runner field, so it compiles against earlier heads. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -287,7 +287,10 @@ automatic** (D-021).
|
||||
recording the batch's end fails, its job fails; the job runner retries
|
||||
a job's final status write with backoff until it lands, each attempt
|
||||
bounded at 5 seconds; at shutdown an attempt that fails or times out is
|
||||
not retried, and the next start fails the job as interrupted. Each time a
|
||||
not retried, and the next start fails the job as interrupted. Recording
|
||||
the batch's end is bounded the same way (one attempt, 5 seconds, made
|
||||
even during shutdown), so a busy writer cannot hold a stopping worker;
|
||||
a batch whose end timed out is left `running` with its job. Each time a
|
||||
batch is started, before the active batch is selected, every batch
|
||||
still `pending` or `running` with no job queued or running (the process
|
||||
stopped, or recording its end failed) is set `failed`, so it does not
|
||||
|
||||
Reference in New Issue
Block a user