feat(automation): once-per-tick-window dispatch claims for scheduled flows - #16585
Conversation
A `schedule` (cron) flow now takes a persisted `(flow, tick-window)` claim in
the same `sys_flow_dispatch` ledger a `time_relative` flow claims per
`(flow, window, record)`, and settles that claim with the run's outcome. That
closes the two live duplicate-delivery doors: a restart inside a tick window,
and an operator `IJobService.replay()` of a window that was already delivered.
`DbJobAdapter.replay` refuses a delivered window with the ADR-0112
`RESOURCE_CONFLICT` / 409 envelope the contract declares, and `{ force: true }`
is the door past it. The error-isolation catch in the schedule trigger keeps
protecting the ticker exactly as before; it stopped being silent, recording the
throw on the claim as a failed outcome instead of leaving the run
indistinguishable from a delivered one.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012zTkyNHJ7TkuN2oXtP5x37
…handler Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012zTkyNHJ7TkuN2oXtP5x37
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012zTkyNHJ7TkuN2oXtP5x37
…hedule-dispatch-claim-ledger
…it the pins by subject
The ledger's settle() wrote update(object, id, data, options); no engine
dispatches on that four-argument form. It is now the by-id payload shape
(update(object, { id, ...fields }, options)) the ObjectQL engine takes, and the
test doubles route through assertEngineUpdateDispatch so a fake can never again
be looser than the engine it stands in for.
The DbJobAdapter.replay pins move to service-job's own suite: reaching them from
trigger-schedule needed an entry in the shrink-only test-source-alias registry,
and that package's rootDir forecloses the paths route.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012zTkyNHJ7TkuN2oXtP5x37
A pass a job service asked for and then abandoned no longer outlives its window: at most one outstanding pass per bound flow, dropped with the binding. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012zTkyNHJ7TkuN2oXtP5x37
…ngine predicate Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012zTkyNHJ7TkuN2oXtP5x37
📓 Docs Drift CheckThis PR changes 3 package(s): 6 hand-written doc(s) NAME something this change touched and may need an implementation-accuracy re-verification:
⛔ 4 release-owned page(s) also name something this change touched. These are read-only:
What this run could not see
Coarse fallback — 6 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): Which tree this was computed onThis run read A worktree cut from an older # while this PR is open — GitHub drops the merge commit once it closes
git fetch origin 8c84f4f808743f4a0d6fe0e899d69d378f6bfaba && git checkout 8c84f4f808743f4a0d6fe0e899d69d378f6bfaba
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin a7c14cdf371bdb0f22be28e47a40fab9f7be88d0 8e08371e2509d340075a9b9d3a706ad7356d313d && git checkout -B drift-repro a7c14cdf371bdb0f22be28e47a40fab9f7be88d0 && git merge --no-ff 8e08371e2509d340075a9b9d3a706ad7356d313d
node scripts/docs-audit/affected-docs.mjs --json a7c14cdf371bdb0f22be28e47a40fab9f7be88d0
|
Patch round 1 on the contract review. settle() refuses succeeded -> failed in both stores and in the engine's in-process fallback: a forced replay that throws must not rewrite a delivered window, because that silently reopens the unforced re-delivery door this card exists to close. failed -> succeeded stays required. The two headers that recorded the immutability retirement stated a rule the code did not hold; both now state the rule as implemented. Barrel surface narrowed to what has consumers, and ReplayGuard is nameable from service-job. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012zTkyNHJ7TkuN2oXtP5x37
…hedule-dispatch-claim-ledger
…icate It restates update() rather than passing the base double's through, so it is a double in its own right and carries the predicate itself. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012zTkyNHJ7TkuN2oXtP5x37
Text only; no behaviour change and no new pins. - settle()'s absorbing rule is enforced read-then-write against the persisted store, so it is not atomic. The window is named rather than closed: the engine's by-id update cannot express a conditional write. - the replay-pass residue comment claimed a later fire clears an abandoned pass. It does not -- the handler deletes only on a key match -- so the pass outlives every fire in every other window, inert throughout. - the readDispatch degradation warning described the both-absent ledger while reading as if it covered a store with settle() but no read(), which is weaker still. - the once-schedule changeset line asserted an unmeasured restart behaviour instead of the measured consequence: a replay before the due instant claims the single window, so the real fire then no-ops. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012zTkyNHJ7TkuN2oXtP5x37
…hedule-dispatch-claim-ledger
Fixes #14501
Behaviour half of the maintainer's A + a2 ruling (decision batch #13,
issuecomment-5518058223, 「同意」). The contract half landed inpackages/specvia #14766 / PR #15377; this PR only consumes it, and mints no new error code.🔴
Clause-②isyes. An earlier revision of this body derivednofrom zeropackages/specpaths in the diff — a path test standing in for a surface test. Clause-② asks whether the change widens the accept set or the public surface, and this one does: new exported symbols on published barrels, plus two new published columns onsys_flow_dispatchand a deliberately retired row-immutability invariant. Corrected here for the audit record. The isolated contract review atCONTRACT_REVIEW_TIERhas run twice; this revision is its second, text-only patch round.What the ruling asked for, and where each piece landed
A
schedule(cron) flow now takes a persisted(flow, tick-window)claim in the samesys_flow_dispatchledger atime_relativeflow claims per(flow, window, record)(#10220), and settles that claim with what the run turned into.(flow, tick-window)claim before launch, no-op on a hitpackages/triggers/trigger-schedule/src/schedule-trigger.tsservice-automation'sFlowDispatchStore+sys_flow_dispatchreplay()refuses a delivered window,forceis the doorpackages/services/service-job/src/db-job-adapter.tsfailedThe specification read for this was
IJobService.replay's TSDoc decision table onorigin/main, not the issue thread.The measurement the dispatch asked for:
claim()returns a boolean, and that was not enoughFlowDispatchStore.claim(key)answersboolean, and both the store's doc-block andsys_flow_dispatch's own header state the rows are immutable claims — the engine slice deliberately declared onlyfindandinsert, so that a double could not be looser than the writer. The ruling's "the claim records its outcome" cannot be expressed in that shape: a row's existence says only that someone took the key, never whether the dispatch landed.So this is a real, declared schema and interface change, stated rather than smuggled:
sys_flow_dispatchgains two optional columns,outcome(succeeded/failed) andsettled_at. The row is now two-phase:claim()still writes it before the launch and still wins the race on the primary key, andsettle()is the only writer that ever updates one. The object's header records that this is the one place the immutability stopped, and states the write rule exactly —succeededis absorbing:nullsucceeded/failedfailedsucceededsucceededfailedThe refusal is not tidiness about monotonicity. The ruling guarantees that a window whose claim succeeded is refused, and "latest attempt wins" erodes exactly that: a forced replay that throws would rewrite a delivered window to
failedand silently reopen the unforced re-delivery door — the duplicate-delivery harm this card exists to close. An operator whose forced replay failed forces again, which is louder and safer than a door that reopens itself. The predicate isisSettleAllowed, and refusing is a no-op rather than a throw: it is the invariant working, not an error.settle()docblock now says so. Asucceededand afailedfor one window finishing at the same instant — a replay and a tick, or two replicas — can interleave and leave the rowfailed. The reach is narrow and the engine's by-idupdateshape cannot express a conditional write, so the window is named rather than closed; the in-memory store, whose check and write share a turn, does not have it. Closing it needs a conditional update the engine does not have yet.settle()writes "only fromnullto a terminal value". That was false about the very transition this PR adds — the trigger settles after every run, including a replay pass, sofailedtosucceededwas always reachable. Both headers and this body now state the rule as implemented, and the transition is pinned in both directions.FlowDispatchStoregains optionalsettle()andread(). A host supplying its own store keeps working: dedup is unchanged, only the replay refusal is lost, and the engine says so once.FlowDispatchStoreEnginenow requiresupdate— which is why every double inflow-dispatch.test.tsroutes throughassertEngineUpdateDispatch.update(object, id, data, options). No engine here dispatches on that four-argument form; the canonical shape isupdate(object, { id, ...fields }, options).check:engine-double-contractrefused the loose double, and the producer was wrong, not the gate.A row left at
outcome: null— a process that died mid-launch, a row written before this PR, or atime-relative:claim, which is never settled at all — reads as not delivered, which is thereplay()contract's "failed" row: it re-runs rather than being refused on a claim nobody settled.The window key
There is exactly one notion of "window", derived from the schedule descriptor itself, so the tick that takes a claim and the replay that reads it back cannot disagree.
croner, the library the job adapter already schedules with. A DST or leap boundary therefore moves the fire and the window key together by construction; there is no second calendar to disagree with, which is what the dispatch asked to be checked.cronerresolves to whole seconds and answers strictly-before, so the reference is advanced to the start of the next second to make that at-or-before.schedule.timezonefalls back to'UTC'here, the same defaultCronJobAdapterapplies to the fire. A host that constructed its adapter with a different default timezone would fire on its calendar while this keys on UTC.floor(now / intervalMs). Anchored to the epoch, not to registration time: a restart re-registers the timer at a new offset, and a window that moved with it would forget every claim across exactly the restart this ledger exists to survive.oncejob re-fires at all after a restart is not measured here and is not asserted.CronJobAdapterfiresrunScheduled(name)with the job name only, andJobHandler's context is{ jobId, data }— no deadline reaches the handler. The window has to be computed from the descriptor, which is whycronermoves from a devDependency to a dependency of@objectstack/trigger-schedule(it is already a runtime dependency of@objectstack/service-job, same version range).Verification
Every pin was written with its prediction recorded first, then proven able to fail by ablation — mutate, prove the mutation reached disk, run, restore, prove the restore by blob hash and a whole-tree
git status --porcelain.DbJobAdapter.replayisSettleAllowedalways true (own suite)service-automation's builtdistThe rethrow prediction was written against an earlier harness. After the pins were split by subject (below), the throwing-run tests no longer route through an adapter, so a rethrow propagates in four places rather than two. Direction as predicted, count higher; recorded here rather than rewritten.
distleg answeredexit 1withNo test files found— the runner had used one package for both the rebuild and the suite, so the trigger's suite was looked for insideservice-automation. That is NOT MEASURED, not a red: it was discarded, the runner was fixed to take the test package separately, and the leg was re-run against the right suite with the marker verified live indistfirst (ablation-dist-preflight.mjs, marker present in 2 built files; absent again after restore).kernel-rebuild-rebind.test.ts'sexpect(logger.warns).toHaveLength(0)— a pin whose whole point is that a bind failure must not degrade back to a WARN. The pin was not weakened. The warning is now emitted only when a ledger is actually attached, because with no ledger nothing is ever recorded as delivered, so there is no refusal to lose and the line was a false alarm.Public surface, narrowed to what has consumers.
computeTickWindow,scheduleDispatchKeyandTickWindoware not on@objectstack/trigger-schedule's barrel: measured 0 consumers outside the package, they appear in no exported signature, and the in-package users import them from./schedule-trigger.js. They stay exported from that module. Symmetrically,ReplayGuardandReplayGuardDecisionare now exported from@objectstack/service-job's barrel —DbJobAdapter.setReplayGuard(name, guard)is a public method on a barrel-exported class, so leaving its parameter type unnameable from that package is the asymmetry the repo's own nameability precedent forbids.Pins split by subject, at a gate's own instruction. Reaching
DbJobAdapterfromtrigger-schedule's tests needed a new entry inscripts/check-test-source-alias.mjs's registry, which is shrink-only, and thepathsroute is foreclosed by that package'srootDirof./src.check:type-source-resolutionnames exactly that case and says to reach the subject through in-package source instead. So the adapter's refusal, itsforcedoor and its guard protocol are pinned inpackages/services/service-job/src/db-job-adapter.replay-guard.test.ts, and the trigger's side pins the guard it actually registers.Gate family, derived mechanically on the final head⚠️ Three gates that read built output were also once measured while a workspace build was still in flight; those results were discarded rather than reported, and the whole family was then re-run against the finished tree, which is the sweep quoted here.
8e08371e2withnode scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack: 103 derived, 103 run, 0 NOT-MEASURED, 0 UNRUN, 0 red, reconciled with--ran. Each exit code was captured immediately after its own single redirection, never through a pipe. The family grew 67 → 103 once the census regeneration putcontent/docsin the change set; the docs families were run too. Every gate that answeredexit 3 PREREQUISITE NOT MET, and the spec docs gates that needed generated artefacts, was re-run for real after a full workspace build (72/72 tasks, exit 0) rather than being read as a pass.Suites:
service-automation1476 ✓ ·service-job117 ✓ ·trigger-schedule82 ✓;typecheckgreen for all three.验收备注
Out of scope, noted and not filed:
IntervalJobAdapter.executeJobswallows a handler rejection too and records it as afailedexecution. So a scheduled flow's throw passes through two swallows, not the one the card names — the trigger's, and the adapter's. It changes nothing for this ruling (the claim now records the failure either way) but it is why the ticker pin is asserted on the handler the trigger registers rather than on a fire routed through the adapter: through the adapter, the pin would be green against a trigger that rethrows.stop()— a later fire does not clear it, because the handler deletes the entry only when the pass matches the window it just computed. It stays inert regardless: a pass naming a window that has passed can never match again. An earlier revision of that comment said a later fire clears it, which was simply wrong.content/docs/permissions/tenant-audit-census.mdxand its counts file move 222 → 223 write call sites, with the hand-written prose figures updated alongside. That is the ledger'ssettle()write being counted, regenerated withscripts/tenant-audit-census.mjs --write.Generated by Claude Code