Roadmap: durable turns, monitors, and adaptive partial-plan execution (ADR 002/005/006)

[HCHogan]

Goal

Land ADR 002, ADR 005, and ADR 006 as a sequence of independently useful, reversible vertical slices:

durability substrate
  → durable turns and digest continuity
  → unified monitor scheduler and typed triggers
  → replay continuity
  → read-only partial-plan execution
  → effectful plans, actors, standing network authority, refinement

This issue tracks sequencing and release gates. Concrete implementation should be split into child issues; do not turn this into one long-lived implementation branch.

Design authorities:

• ADR 002 — Partial Plans and Adaptive Elaboration
• ADR 005 — Turn Continuity
• ADR 006 — Monitors
• Architecture after ADR 006

Sequencing principles

• Do not implement all of ADR 002 before ADR 005/006. ADR 002 contains several projects: the shared journal, Plan IR, executor, concurrency/actors, and self-refinement.
• Persist facts before exposing read-side continuity; stabilize turn continuity before asking Plans and monitors to consume it.
• Every phase must preserve the current horizon-1 tool loop as a feature-flagged fallback.
• Expand the executable effect frontier one class at a time.
• Exactly-once applies to effects, not model reasoning.
• A handle is syntax, never authority; every resolution rechecks conversation, resource, and information-flow scope.

E0 — Shared durability substrate

Dependency: #14.

• Complete v1.0 pillar B: complete the durability roadmap #14 L3 send linkage, L4 two-phase tool calls, durable sandbox state, and L2 checkpoint resume.
• Persist agent_turns.turn_ordinal with UNIQUE (conversation_id, turn_ordinal).
• Persist journal execution_ordinal with UNIQUE (turn_id, execution_ordinal).
• Capture per-turn trace archives in the dispatch epilogue, even before replay reads them.
• Provide scoped result envelopes/artifact spill metadata without exposing internal blob refs.
• Reclaim in-flight turns and unknown effects safely after restart.
• Add crash-window tests around send commit, tool start/finish, archive capture, and resume.

Exit gate: no user-visible behavior change; restart cannot duplicate a committed send or silently retry an outcome-unknown effect.

E1 — ADR 005 digest continuity

• Render recent worked turns as persisted t#<turn_ordinal> handles.
• Add scoped context_expand(t#) over normalized journal traces.
• Produce deterministic turn digests from journal facts.
• Route replies to in-flight turns as steer.
• Route replies to finished turns as fresh fork-from turns.
• Persist turn_edges and inject a deterministic ambient delta.
• Enforce clear boundaries, archive TTL/LRU, and cross-conversation denial tests.

Exit gate: continuation works correctly at the digest tier with no provider-wire replay.

E2 — ADR 006 scheduler unification

Behavior-preserving first slice.

• Migrate reminders into monitors as TimeCron + canned.
• Add persisted, conversation-scoped m#<monitor_ordinal>.
• Generalize the reminder worker into one scheduler with boot reconciliation.
• Persist monitor_fire admission state, idempotency key, and reclaimable lease.
• Preserve existing reminder attempt/parked behavior.
• Add a crash matrix covering observation, pending fire, claim, dispatch, and delivery.

Exit gate: all existing reminder behavior passes unchanged through the monitor scheduler.

E3 — ADR 006 elaborated continuations and LedgerMatch

• Let TimeCron open a fresh ordinary horizon-1 turn after fire-time revalidation.
• Add host-authenticated IngestClass = LiveDelivery | Backfill to the adapter envelope, platform-event reservation, and canonical row.
• Migrate old rows as Backfill; unknown ingest classes fail closed for monitor evaluation.
• Implement LedgerMatch inside canonical ingest with unique (monitor_id, canonical_message_id) fire records.
• Make cooldown advancement an atomic guard in the ingest transaction.
• Exclude outbound/internal rows from LedgerMatch.
• Enforce TTL, max-fire count, per-group caps, role-gated arming, and per-group elaboration budgets.
• Record trigger evidence in monitor_fire; keep fork-from as turn provenance only.
• Support elaborated silence: a fire does not imply speech.

Exit gate: live inbound rows fire at most once; backfill never fires; a committed fire survives restart without admitting two turns.

E4 — ADR 005 replay tier

• Implement provider-specific replay filtering using the same path as in-dispatch round trips.
• Rebuild requests from the current system prompt, replayed turn segments, ambient delta, and current trigger.
• Implement validity checks for model family, archive retention, provider validity, prompt/tool major versions, and chain budget.
• Fall back to the digest tier on every validity failure.
• Deduplicate replayed ledger messages from the ordinary conversation window.
• Implement fork-chain compression as a digest prefix plus contiguous verbatim suffix.
• Share narrator/turn-digest projection code where specified by ADR 005.

Exit gate: replay produces no more duplicate or outcome-unknown effects than digest continuity and does not reduce answer quality.

E5 — ADR 002 Plan core, offline first

• Define versioned Goal, Plan, Expr, Predicate, VerifierRef, and ValueRef codecs.
• Keep Expr/Predicate total, pure, bounded, and host-call-free.
• Implement the restricted pseudo-code DSL as whole-input parse-or-reject.
• Implement schema/binding/effect/authority/information-flow/cardinality/budget validation.
• Implement preview and symbolic interpreters.
• Implement acceptance verification; verifier failure re-holes with bounded scoped/tainted evidence.
• Implement bounded push/pull elaboration sessions and observed read-set dependencies.
• Build recorded-conversation replay evaluation before production execution.

Exit gate: offline fixtures quantify parse/validation rate, quality, context occupancy, total tree tokens/cost, and expected deoptimizations.

E6 — Read-only partial-plan runtime

• Execute short, read-only validated Plan segments behind a feature flag.
• Resolve t#<turn_ordinal>:r<execution_ordinal> through persisted alternate keys and current scope.
• Revalidate residual Plans after feedback, policy/catalog/prompt changes, stale dependencies, and budget changes.
• Preserve already observed effects across deoptimization.
• Tier down to the current one-step tool loop on any unsupported frontier.
• Emit normalized Plan events through the existing journal/trace surfaces.

Exit gate: no increase in duplicate/outcome-unknown effects; answer quality is no worse than the current loop; rollback is a flag flip.

E7 — Expand the effect frontier

Open one class per evaluated release:

• Scoped sandbox writes.
• Idempotent external writes.
• Visible sends and approval/frontier handling.
• Explicit bounded Fork/Join.
• Resource-scope lock sets and deterministic collection/cancellation.

Each checkbox requires its own replay gate and crash/deoptimization tests.

E8 — Durable child actors and typed futures

• Spawn child turns with narrowed conversation scope, catalog, information clearance, and effect ceiling.
• Keep spawn, depend subscription, completion fulfillment, and steer/annotate delivery distinct.
• Suspend consumers on durable futures rather than blocking workers.
• Require explicit parent-minted recipient capabilities for sibling communication.
• Enforce cycle, cross-family, cross-conversation, taint, rate, and effect checks.

Exit gate: child work cannot widen the initiating human principal's authority or leak hidden child context.

E9 — ADR 006 ExternalPoll

• Add a host-managed HTTP probe boundary with scoped targets, redirects, credential policy, interval floor, and bounded output.
• Journal request target, response digest, outcome, and observation artifact.
• Use non-overlapping durable poll claims.
• In the admission transaction, materialize an immutable turn-owned trigger_input result row linked to monitor_fire.
• Expose the observation only as a scoped t#:r handle.
• Decide and document the defense-in-depth release dependency on sandbox egress governance.

Exit gate: no ambient credentials or target widening; poll/fire/dispatch crash windows are idempotent.

E10 — Journaled self-refinement

• Define conversation/session-scoped HarnessScope writes under ordinary effect and role policy.
• Keep the base system prompt, validator, authority policy, and effect kernel immutable.
• Store harness objects append-only with a CAS current pointer.
• Journal motivating trace, candidate, validation, activation, and later outcome.
• Make rollback an explicit CAS activation of a previous immutable version.
• Activate only at a turn/hole boundary and re-hole/revalidate affected residual Plans.
• Require stronger promotion for executable code/tool widening and explicit admin/offline-eval approval for global scope.

Exit gate: refinement is reversible, attributable, scoped by default, and cannot edit the kernel that validates it.

Cross-phase release gates

Every user-visible phase must report and compare:

• main-model context occupancy;
• total tree tokens and cost;
• critical-path latency;
• artifact pulls and bytes;
• verifier pass/failure rates;
• deoptimizations;
• duplicate effects;
• outcome-unknown effects;
• answer quality on recorded conversation replay;
• scope/taint denial coverage;
• crash-window and restart behavior.

No phase advances solely because it reduces LLM calls.

[HCHogan]

E0 is complete and the E0 checklist above has been updated.

Landed: durable turn/execution ordinals, normalized two-phase journal and scoped spill envelopes, L3 canonical send provenance, trace archives, same-turn restart re-hole/resume, and durable sandbox volume reconciliation with observed manifests.

The exit gate is covered by crash-window tests for committed sends, tool start/finish, archive checkpointing, and resume. E1 and later phases remain intentionally untouched.

Implementation committed locally as 292f27a (feat: land E0 durable turn substrate). This update did not push the branch.

Validation:

• cabal build all -j2
• 811 unit/integration examples, 0 failures
• 149 fresh-database migration/DB examples, 0 failures
• cabal run max-prompt-flow and cabal run max-prompt-flow -- --check
• isolated Docker exec/manifest smoke test; temporary container and volume removed

[HCHogan]

E1 implementation is complete and its seven checklist items are now checked.

Landed locally as b5c7986 (feat: land E1 digest continuity): persisted scoped turn_edges; recent worked-turn t# handles; scoped/paginated context_expand(t#); deterministic journal/output/sandbox/ambient digests; exact reply steering for live turns; fresh fork-from continuation for terminal turns; prompt/catalog drift; !clear, cross-conversation denial, and continuous 14-day/50-turn archive retention. The E1 path deliberately performs no provider-wire replay; E2 and later phases are untouched.

Validation:

• cabal build all -j2
• 817 non-DB examples, 0 failures
• 153 PostgreSQL migration/DB examples, 0 failures, including concurrent 50-turn LRU enforcement
• cabal run max-prompt-flow
• cabal run max-prompt-flow -- --check
• git diff --check

The commit is local and has not been pushed yet. The roadmap issue remains open for E2 and later phases; rollout metrics under the cross-phase release gates remain to be observed before advancing phases.

[HCHogan]

E2 and E3 implementation are complete, and their 15 checklist items above are now checked.

Landed locally as:

• 00b0271 (feat: land ADR 006 E2 monitor scheduler): migrated canned reminders onto the durable monitor scheduler with scoped m# ordinals, persisted fire admission/leases, boot reconciliation, bounded retry/parking, and crash-window coverage.
• 384d8b5 (feat: complete ADR 006 E3 elaborated monitors): added elaborated TimeCron turns, host-authenticated live/backfill ingest provenance, typed LedgerMatch, atomic edge dedupe/cooldown, TTL/max-fire/cap/budget enforcement, trigger evidence, role-gated monitor tools, frozen arm-time effect ceilings, durable PostgreSQL wakeups, ordinary-turn recovery, and elaborated silence.

The E2 exit gate preserves canned reminder behavior through the unified scheduler. The E3 exit gate is covered for live-only matching, backfill exclusion, one-fire-per-ledger-edge admission, and same-turn restart recovery.

Validation:

• cabal build all
• 823 non-DB examples, 0 failures
• 160 PostgreSQL DB examples, 0 failures; migration 074 also verified from a fresh database
• cabal run max-prompt-flow
• cabal run max-prompt-flow -- --check
• git diff --check

Both commits are local and have not been pushed. The roadmap remains open for E4 and later phases; production rollout metrics under the cross-phase release gates remain to be observed.