fix: reconcile stopped Codex runtime cleanup (#101)

[Niko4417]

Scope

• Accepted issue: Defect: Make Codex cleanup coverage proof deterministic #101
• Accepted planning-contract version: v7
• Automated readiness record: Defect: Make Codex cleanup coverage proof deterministic #101 (comment)
• Actual source branch: codex/101-cleanup-fixture-v6
• Accepted target branch: epic/98-codex-tracer
• Parent epic and Quality Envelope rows: Epic: Governed Codex lifecycle tracer — one workspace, one no-effect turn #98 v5; runtime cleanup, cancellation/crash recovery, zero descendants, deterministic native coverage
• Change classification: defect
• Product decision, finding, or incident: authoritative run 31297856180 retained K101:C0A0M0G1 on macOS 14 and 26, proving the absent process group was not reconciled into a successful reap and ownership retirement
• In scope: owning macOS runtime cleanup/reconciliation and adjacent tests in native/crates/keiko-host-macos/src/runtime.rs
• Out of scope: cleanup deadlines or signal order, UI/protocol/docs/dependencies/workflows/gates, Integrated packaged macOS acceptance for the Codex tracer #104 files, Epic: Contract-as-Code migration and lifecycle activation cutover (ADR-0003/ADR-0004) #49, Knowledge Capsules/RAG
• Native targets, contracts, or trust boundaries affected: keiko-host-macos; authenticated process identity, direct-child reap proof, and ownership retirement remain fail closed

Product and architecture alignment

• The implemented contract version and fingerprint match the automated readiness record; no
  semantic planning change was absorbed during implementation.
• The change follows the Decision Addendum, CONTEXT.md, accepted ADRs, and the issue Quality
  Plan.
• Existing Keiko material was used only after a recorded Reuse Assessment, or the issue records
  why Existing Keiko evidence is not applicable.
• This greenfield change creates no mandatory build-time or runtime dependency on Existing
  Keiko.
• Product authority, policy, evidence, and privileged effects remain in their owning Native
  layer.
• Any durable architecture change is recorded in an ADR.

Acceptance criteria and evidence

Evidence must identify the exact file, test, command, artifact, or platform result. Do not mark a
row complete from intention or a processing badge.

Acceptance criterion	Evidence	Exact head or artifact	Result
AC1	Readiness reserves the post-KILL budget for direct-child reap, zero authenticated descendants, absent group, and exact-identity retirement; turn cleanup retains its prior parent-reap grace	24d87e94dfcd6dbafda7a908d3443c1e9139fde3	Complete
AC2	Authoritative K101R:F001 proved group KILL occurred after the deadline; the phase-policy regression failed before the correction and passes after it	24d87e94dfcd6dbafda7a908d3443c1e9139fde3	Complete
AC3	Descendant-first KILL ordering, parent wait/reap semantics, identity mismatch/reuse, poisoned ownership, and already-reaped recovery remain independently covered	24d87e94dfcd6dbafda7a908d3443c1e9139fde3	Complete
AC4	Focused cleanup tests, host 161/161 x3, complete quality/audit receipts, packaged macOS acceptance, and the authoritative macOS 14/26 matrix are green; all temporary diagnostics are absent	24d87e94dfcd6dbafda7a908d3443c1e9139fde3	Complete

Acceptance journey evidence

Complete for user-facing work. Otherwise state Not applicable with the issue rationale and remove
the placeholder data row. Evidence records what ran; the issue remains the source for expected
behavior.

• Applicability: Not applicable — non-user-facing runtime cleanup defect; parent J1 behavior is unchanged and covered by runtime and packaged gates.

• Automated checks exercise user-visible outcomes rather than incidental implementation
  details.

• Required failure, recovery, accessibility, visual, and platform observations are settled.

Quality Plan settlement

• Applicable positive, negative, boundary, failure, cancellation, and recovery behavior is
  covered.
• The actually wired production composition was tested where this change crosses layers.
• Applicable security, accessibility, performance, resource, visual, and platform evidence is
  attached or linked.
• Excluded quality areas retain the rationale accepted in the issue.
• Secrets, credentials, raw customer content, private endpoints, and PII are absent from source,
  tests, logs, evidence, artifacts, issues, and this pull request.

Verification

• npm ci --ignore-scripts
• npm run quality
• npm audit --audit-level=high
• Every declared native target-specific gate passed on its authoritative platform.
• I reviewed the complete diff against requirements, contracts, trust boundaries, and failure
  modes.

Additional affected checks and concise results:

Focused reconciliation/cleanup suite: 6/6 green
Complete keiko-host-macos suite: 161/161 x3 (9.06 s, 8.97 s, 8.69 s)
verify receipt: npm run quality + npm audit green at 24d87e94
audit receipt: findings=0, non-user-facing, bound to 24d87e94
npm run acceptance:macos: exit 0 at 24d87e94
GitHub CI run 31317318841 attempt 2: complete matrix green at 24d87e94; macOS 14/26 coverage, native:test, and packaged acceptance passed
GitHub CI run 31317318841 attempt 1: macOS 14 native:test failed with an anonymized last-eight-line tail after coverage passed; exact-head rerun passed and macOS 26 passed the same path
npm run acceptance:codex-tracer:macos: deferred until #104 rebases; #104 owns the corrected canonical metric schema
git diff --check: green

Independent audit and findings

• Audit scope and dimensions: Defect: Make Codex cleanup coverage proof deterministic #101 v7 acceptance and Quality Plan; reconciliation ordering; exact identity; reaping/retirement; poisoned, stale, reused, and mismatched states; fixed 300 ms; resources; signatures; strict Integrated packaged macOS acceptance for the Codex tracer #104/Epic: Contract-as-Code migration and lifecycle activation cutover (ADR-0003/ADR-0004) #49 exclusion
• Audited commit: 24d87e94dfcd6dbafda7a908d3443c1e9139fde3

Confirmed finding	Evidence	Disposition	Settlement evidence or follow-up
Initial helper identity gap and later readiness final-phase starvation	A/B regression at 92bd7d7; authoritative F001 at diagnostic head 98ee304	Settled	Exact-identity retirement plus readiness-only budget preservation at 24d87e94dfcd6dbafda7a908d3443c1e9139fde3; independent re-audit findings=0

• Findings are evidence-cited; speculative observations are advisory rather than blockers.
• Every confirmed finding is resolved, explicitly accepted by an authorized human, or linked to
  a scoped follow-up that does not invalidate current acceptance.
• Verification and audit were repeated after the latest implementation or audit fix.

Integrated epic acceptance

Complete when this PR delivers or changes an epic's integrated acceptance surface; otherwise state
Not applicable with rationale.

• Applicability: Not applicable — #101 proves owning runtime cleanup; rebased #104 owns final canonical tracer requalification on the integrated epic head.
• Production-composition result: exact-head local quality and packaged macOS acceptance green
• Machine-enforced acceptance result: local exact-head gates and GitHub CI run 31317318841 attempt 2 green
• macOS evidence: local exact-head acceptance plus macOS 14/26 coverage, native test, and packaged acceptance green
• Windows evidence: not applicable under ADR-0006 and Defect: Make Codex cleanup coverage proof deterministic #101 v7
• Manual usability, accessibility, visual, signing, or packaging evidence: no UI change; signing and packaging gates green

Delivery

• Target path: child issue -> epic branch
• The target branch matches the delivery path accepted in the issue; no direct push, force
  push, gate bypass, finding dismissal, or authority widening occurred.
• Commits are signed and every required check is bound to the exact current head and expected
  producer.
• Advisory tools are not treated as required merge authority under the current quality-gate
  policy.
• Documentation, ADRs, contracts, known limitations, and follow-ups are current.
• A draft pull request was not promoted to Ready for Human Review before every required
  Acceptance Journey result and exact-head gate was complete.

For a child-issue pull request targeting its designated epic branch:

• The accepted issue authorizes this epic-branch target.
• Acceptance and audit evidence is complete, every applicable exact-head gate is green, and no
  blocking finding or review conversation remains.

This child pull request does not target dev; the epic delivery remains human-only.

Residual risks and follow-ups

• Codex review is pending; no local or platform code finding remains. The first macOS 14 attempt's anonymized native-test failure did not reproduce on the exact-head rerun; its preceding coverage suite and the macOS 26 matrix were green.
• After Defect: Make Codex cleanup coverage proof deterministic #101 integrates, Integrated packaged macOS acceptance for the Codex tracer #104 v6 must be rebased and its packaged/canonical acceptance rerun.

[Niko4417]

#101 v6 implementation and audit evidence

Exact head: 5b437fd82329c6cdbd0bc6532afd623ffec72771

• Failure-first evidence is retained in issue Defect: Make Codex cleanup coverage proof deterministic #101: macOS 26 run 31281731351 failed only the named cleanup test in 3/3 attempts; macOS 14 included the same failure in its first attempt and passed 156/156 on retry. The workflow log retains the test identity and 155/156 result but truncates the panic line; control-flow inspection identifies the failed cleanup outcome assertion.
• Diagnosis measured the old TERM-resistant fixture descendant at 44.0% runnable CPU versus 0.5% for the blocking replacement. The old unbounded busy loop uniquely competed with the parent inside the immutable 300 ms cleanup proof window.
• Exact implementation delta: one #[cfg(test)] line changes while :; do :; done to while :; do /bin/sleep 1; done.
• Both parent and descendant still ignore TERM; the test still observes descendant SIGKILL, parent wait/reap marker, cleared runtime ownership, and absent process group. No production code, deadline, constant, assertion, gate, workflow, dependency, Integrated packaged macOS acceptance for the Codex tracer #104, or Epic: Contract-as-Code migration and lifecycle activation cutover (ADR-0003/ADR-0004) #49 file changed.
• Focused nightly LLVM test: 1/1 passed in 0.24-0.25 s, with Killed: 9 observed.
• Full host LLVM coverage: 156/156 passed three consecutive times in 8.61 s, 8.95 s, and 9.42 s.
• npm run native:coverage: three explicit direct repetitions green.
• Exact-head full npm run quality, npm audit --audit-level=high, and npm run acceptance:macos: green.
• Independent exact-head audit: findings 0; signed commit and private repository email verified; worktree and diff check clean.

The literal canonical tracer command is intentionally sequenced after integration through rebased #104 v6. This target predates #104's accepted separation of native-picker dismissal from the 100 ms Keiko projection metric; on the current pre-#104 target it fails only the superseded mixed timing evidence. No #101-scope change can or should alter that harness. After this child integrates, #104 v6 will be rebased and its exact packaged/canonical acceptance rerun before the epic is handed off.

[Niko4417]

Post-failure exact-head evidence — 663ca90d900a6a80503d5ef569ed0a76892c0492

The first candidate (5b437fd) supplied authoritative failure-first evidence on PR run 31290165062: macOS 14 job 93185860519 and macOS 26 job 93185860524 each failed 155/156 with only readiness_cleanup_reaps_after_escalating_a_term_resistant_process failing.

The remaining fixture race was an unproven readiness handshake: the outer shell could emit ready before the descendant installed trap '' TERM. An injected 200 ms ordering probe demonstrated ready with the descendant trap marker absent under the old ordering, while descendant-owned readiness returned only after the marker existed.

The additive signed commit moves printf 'ready\n' into the descendant immediately after its TERM trap. The outer shell still waits and writes the reap marker; SIGKILL escalation, ownership retirement, zero-descendant/process-group assertions, and the 300 ms reserve are unchanged. Net issue diff remains one #[cfg(test)] fixture line (1+/1−) with no production, gate, #104, or #49 change.

Exact-head local evidence:

• focused nightly LLVM test: 1/1, 0.24 s, observed Killed: 9
• complete instrumented host suite: 156/156 three consecutive times
• npm run native:coverage: three exact-head repetitions green
• full npm run quality and npm audit --audit-level=high: green receipt at 663ca90
• npm run acceptance:macos: green at 663ca90
• independent re-audit: findings 0; non-user-facing; audit receipt at 663ca90
• Good ED25519 signature and private repository email; clean worktree and diff check

Fresh exact-head macOS 14/26 CI is now running. Codex review will be requested only after those authoritative jobs and the metadata contract are green.

[Niko4417]

Final fixture candidate — 73cc14f97cb7ab4c23781593cc8f8d36322fb473

The second candidate (663ca90) supplied additional authoritative failure-first evidence: PR run 31291795915 failed 155/156 on both macOS 26 job 93190146990 and macOS 14 job 93190147001, again only on the named cleanup proof.

That isolated transient descendant churn in the fixture. The final net issue diff remains one named #[cfg(test)] fixture (2+/1−):

• readiness comes from the descendant after it installs trap '' TERM;
• the descendant blocks in shell-builtin read instead of spawning /bin/sleep children;
• Rust keeps a stdin pipe open and the background shell explicitly inherits it with <&0.

Independent audit caught and rejected the first unredirected-read attempt because noninteractive async /bin/sh otherwise uses /dev/null and spins. The corrected exact command was independently re-probed: outer and inner were both sleeping, inner CPU fell to 0.0%, TERM left both alive, SIGKILL of the inner produced the parent's reaped marker and exit 0, and no group residue remained.

Final exact-head evidence:

• focused nightly LLVM: 1/1, 0.24–0.26 s, inner SIGKILL observed;
• focused contention: 40/40 under eight bounded CPU loads;
• complete host LLVM suite: 156/156 ×3;
• exact npm run native:coverage: three repetitions green;
• complete npm run quality plus npm audit --audit-level=high: green receipt at 73cc14f;
• npm run acceptance:macos: green at 73cc14f;
• independent final audit: findings 0, non-user-facing, receipt at 73cc14f;
• net scope excludes production code, constants, the 300 ms reserve, assertions, gates/workflows, Integrated packaged macOS acceptance for the Codex tracer #104, and Epic: Contract-as-Code migration and lifecycle activation cutover (ADR-0003/ADR-0004) #49;
• all additive commits have Good ED25519 signatures using the private repository email; worktree and diff check are clean.

Fresh exact-head macOS 14/26 CI is running. Codex review remains intentionally deferred until the authoritative matrix and PR metadata contract are green.

[Niko4417]

Temporary authoritative-run diagnostic at 27c6d3dba2e649600424f4ad45ae23358631b158: after repeated runner-only failures hid the exact predicate behind the native gate tail, this test-local head emits only K101:C#A#M#G# (cleanup result, active ownership retired, exact parent-reap marker, process group absent) immediately before the unchanged assertions. The compact diagnostic passed focused testing and the exact local native coverage gate; independent audit found zero diagnostic-purpose findings. It will be removed additively after the remote flag is captured and is not final delivery evidence.

[Niko4417]

Authoritative diagnosis captured at temporary head 55253c93bfc168f883dc1d78d534c441bfc67810 on both macOS 14 and macOS 26: K101:C0A0M0G1. The process group was absent, while cleanup_after had not reported success, ownership remained retained, and the shell parent had not published its post-wait marker before the immutable 300 ms deadline. This identifies fixture-only post-descendant scheduling/I/O on the critical path; it does not justify a production deadline change. All temporary diagnostics are now being removed, and the final correction is limited to replacing the parent post-wait filesystem marker with the already-open fixture pipe while retaining exact TERM→KILL, parent wait, ownership-retirement, and group-absence assertions.

[Niko4417]

Final #101 v6 candidate — df9ac2c7b7241cb07217b59087c882e49560b53a

The authoritative temporary diagnostic run 31297856180 produced K101:C0A0M0G1 on both macOS 26 job 93205756656 and macOS 14 job 93205756657. The process group was already absent, while cleanup status, ownership retirement, and the filesystem post-wait marker had not completed inside the immutable 300 ms window. This isolated fixture-only filesystem scheduling/I/O on the critical path; it did not justify a production deadline change.

The final additive commit removes every diagnostic and keeps the correction inside the named #[cfg(test)] fixture:

• readiness is emitted by the TERM-resistant descendant only after its trap is installed;
• the descendant blocks without CPU spin or child churn on an explicitly inherited Rust-owned pipe;
• the outer parent still executes wait, but publishes the strict post-wait reaped proof through the already-open stdout pipe instead of opening/writing a temporary file during cleanup;
• Rust asserts cleaned and retired ownership before reading that buffered marker, so a failing cleanup path cannot block;
• the original timed-out state, cleanup, ownership, exact reap-marker, and process-group absence assertions remain strict.

Final evidence at the exact head:

• net epic delta: one named test fixture, 7+/11−; no product/runtime constants, 300 ms reserve, gate, workflow, dependency, Integrated packaged macOS acceptance for the Codex tracer #104, or Epic: Contract-as-Code migration and lifecycle activation cutover (ADR-0003/ADR-0004) #49 change;
• focused nightly LLVM test: 1/1 in 0.23–0.24 s with descendant SIGKILL observed;
• complete host LLVM suite: 156/156 three consecutive times;
• exact repository npm run native:coverage: three consecutive repetitions green;
• npm ci --ignore-scripts, complete npm run quality, and npm audit --audit-level=high: green;
• npm run acceptance:macos: green;
• exact-head verify receipt and independent zero-findings audit receipt: written at df9ac2c;
• independent audit: findings 0; all commits Good ED25519 signatures with the repository private email; clean worktree and diff.

Fresh exact-head macOS 14/26 CI is running. The canonical tracer acceptance remains intentionally sequenced after integration through rebased #104 v6, whose accepted harness owns the corrected native-picker/Keiko timing separation.

[Niko4417]

v6 stop condition reached; #101 replanned as accepted v7

Final v6 head df9ac2c7b7241cb07217b59087c882e49560b53a is not mergeable. Exact run 31301131252 disproved the remaining fixture-only approach: macOS 26 failed only the named cleanup test during native coverage, while macOS 14 passed coverage and later failed the native test gate under full load. Two independent analyses and 1,000 additional stressed local passes found no deterministic fixture-only correction that preserves the strict proof inside the immutable phase split.

Issue #101 has therefore been semantically replanned as contract v7 with accepted fingerprint 6729b594bca2063fb6c4f8bc3238d51ba3fe3d4b753e0001a71fc23f9f07d37d. v7 keeps the parent outcome and total 300 ms readiness reserve unchanged, but authorizes the owning cleanup/final-reconciliation layer and separates:

1. the 300 ms product proof: direct child reaped, zero authenticated descendants, process group absent, ownership retired only after proof; and
2. the independent deterministic semantic proof: descendant-first SIGKILL permits the TERM-resistant parent to execute wait, publish its reap marker, and exit.

The PR remains draft and its prior v6 receipts are intentionally stale. Implementation resumes only under the accepted v7 authority; no #104 or #49 scope is included.

[Niko4417]

Final exact-head evidence — 24d87e94dfcd6dbafda7a908d3443c1e9139fde3

The accepted #101 v7 defect is complete at the exact signed/private-email head.

• Failure-first: temporary bounded diagnostics on authoritative runners captured K101R:F001, proving readiness cleanup reached descendant and group SIGKILL only after exhausting the fixed 300 ms deadline, leaving no final reconciliation window.
• Owning correction: readiness alone preserves the final reconciliation budget after descendant-first SIGKILL; turn cleanup and the independent parent-wait/reap proof retain their prior bounded grace. The 300 ms reserve, signal order, direct-child reap requirement, exact process identity, zero-descendant/group proof, and fail-closed ownership retirement remain unchanged.
• Deterministic regression: the extracted old policy returned descendant_started + 25 ms for a 50 ms remainder and failed before the behavior change; readiness now returns descendant_started, while the existing parent-reap policy remains capped at 100 ms and otherwise half the remainder.
• Local exact-head verification: focused cleanup/policy suite 6/6; complete host suite 161/161 three consecutive times (9.06 s, 8.97 s, 8.69 s); full npm run quality; npm audit --audit-level=high; npm run acceptance:macos; verify receipt and independent audit receipt all green.
• Independent final audit: findings 0. Temporary K101/K101R diagnostics are fully absent; exact scope is native/crates/keiko-host-macos/src/runtime.rs; no Integrated packaged macOS acceptance for the Codex tracer #104 or Epic: Contract-as-Code migration and lifecycle activation cutover (ADR-0003/ADR-0004) #49 delta.
• Authoritative CI: run 31317318841 attempt 2 is fully green at this exact head, including macOS 14 and macOS 26 coverage, native:test, package/signing/security, and packaged acceptance.
• Transparency: attempt 1's macOS 14 native:test failed after that same job's full coverage suite passed. The native gate retained only an anonymized last-eight-line stderr tail, so it did not identify a test. The exact-head macOS 14 rerun passed the complete path, and macOS 26 passed it on attempt 1; no code change or gate change occurred between attempts.

The canonical tracer acceptance is intentionally sequenced after #101 integrates and #104 is rebased, because #104 owns the accepted native-picker/Keiko timing evidence schema. This PR remains a strict #101 child delivery to epic/98-codex-tracer.

[Niko4417]

@codex review

[Niko4417]

@codex review

Review exact head 24d87e94dfcd6dbafda7a908d3443c1e9139fde3. Focus on the fixed 300 ms readiness cleanup path, descendant-first KILL ordering, final reconciliation budget, exact process identity, direct-child reap proof, zero-descendant/group retirement, and preservation of turn-cleanup behavior. This PR is ready for review and all exact-head gates are green.

[Niko4417]

@codex review

[Niko4417]

Dedicated Codex review — exact head 24d87e94dfcd6dbafda7a908d3443c1e9139fde3

The GitHub @codex review integration did not acknowledge three valid triggers despite repository code review being enabled, so I ran Codex's official dedicated local review against origin/epic/98-codex-tracer as a transparent fallback.

Result: no findings. Codex concluded that the cleanup changes preserve fail-closed ownership semantics while adding exact-identity reconciliation and bounded readiness phase allocation. It independently ran the affected host suite (161/161) and additional focused stress evidence, including 100 serial and 3,200 direct concurrent focused executions without a failure.

This comment does not claim that the GitHub webhook review ran. The visible trigger comments remain as integration evidence. No code or repository state changed during the dedicated review.