Update .gitignore with 29 changed files (#4477)

.gitignore

@@ -38,3 +38,4 @@ scripts/dashboard-audit/out/
 pr*-worktree/
 *-worktree/
 .simard/
+gym_history.db

Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "simard"
-version = "0.35.0"
+version = "0.36.0"
 edition = "2024"
 default-run = "simard"
 

docs/concepts/reconcile-and-self-deploy.md

@@ -215,7 +215,7 @@ of the following hold:
 | **Memory intact** | cognitive-memory fact count ≥ the pre-deploy count (within tolerance), via the `CognitiveMemoryOps` count API |
 | **Goal board intact** | the goal board loads and the active-goal count is preserved |
 | **Brains LLM-backed** | zero `BrainJudgmentRecord.fallback == true` records over a probe cycle (see [parse-failure record](../reference/ooda-brain-parse-failure-record.md)) |
-| **No quarantine** | no *fresh* corrupt-store quarantine appeared in the live cognitive-store directory since the deploy window opened (retained historical forensic snapshots are ignored) |
+| **No quarantine** | no *fresh*, *unacknowledged* corrupt-store quarantine appeared in the live cognitive-store directory since the deploy window opened — retained historical forensic snapshots and acknowledged artifacts are ignored (see below) |
 
 Any single failing probe fails the health check and triggers rollback. The probe
 output is the same structured JSON whether it is run by the orchestrator or by an
@@ -238,6 +238,29 @@ The probe also reports the in-window (`fresh_quarantines`) and retained
 only retained forensic snapshots (see
 [`NoQuarantineProbe`](../reference/self-deploy-api.md#noquarantineprobe)).
 
+### Clearing a stuck quarantine
+
+The window filter alone does not cover one pathological case (#4469): the largest
+corrupt store is a *recovery asset* that `simard cleanup` deliberately never
+deletes (#2550). If such an asset lands *inside* the deploy window — or a fresh
+corruption keeps re-landing on the one artifact protected from deletion — the
+probe stays red on exactly the artifact that can never be swept, and self-deploy
+freezes commits behind merged `main`.
+
+The probe is therefore also **acknowledgement-aware**. An operator can
+acknowledge a genuinely-stuck quarantine (`simard self-health
+--acknowledge-quarantine`) — or, for the protected recovery asset past the
+30-day forensic window, the daemon auto-acknowledges it — by writing a durable
+`.ack` sidecar next to the artifact. Acknowledgement silences the probe
+**without deleting the recovery asset**: an acknowledged quarantine counts as
+neither *fresh* nor *retained*, so `all_healthy()` can converge. A *new*
+corruption event writes a fresh, unacknowledged artifact and correctly reddens
+the probe again, because the marker is keyed to the exact filename. Full
+mechanics:
+[self-deploy quarantine-acknowledge](../reference/self-deploy-quarantine-acknowledge.md)
+and the runbook [Clear a stuck memory
+quarantine](../howto/clear-a-stuck-memory-quarantine.md).
+
 ## Why build-from-source, not release-download
 
 A merged-but-unreleased commit *cannot* be fetched as a published binary — that

docs/howto/clear-a-stuck-memory-quarantine.md

@@ -0,0 +1,138 @@
+---
+title: How to clear a stuck memory quarantine
+description: Operator runbook for the self-health `no_quarantine` deadlock (#4469) — how to recognize a genuinely-stuck cognitive-memory quarantine that freezes self-deploy, acknowledge it with `simard self-health --acknowledge-quarantine` so the probe clears WITHOUT deleting the #2550 recovery asset, confirm convergence, and reverse the acknowledgement if needed.
+last_updated: 2026-07-22
+review_schedule: as-needed
+owner: simard
+doc_type: howto
+status: implemented
+related:
+  - ../reference/self-deploy-quarantine-acknowledge.md
+  - ../reference/self-deploy-api.md
+  - ../concepts/reconcile-and-self-deploy.md
+  - ../howto/verify-and-roll-back-a-self-deploy.md
+---
+
+# How to clear a stuck memory quarantine
+
+> **Status: implemented.** `simard self-health --acknowledge-quarantine` writes
+> a durable `.ack` sidecar next to each cognitive-memory quarantine artifact so
+> the `no_quarantine` probe can clear **without** deleting the artifact. The
+> underlying convention is documented in the
+> [quarantine-acknowledge reference](../reference/self-deploy-quarantine-acknowledge.md).
+
+## When to use this
+
+Use this runbook when self-deploy is frozen **only** because of a quarantine
+that can never clear on its own — the deadlock from issue #4469:
+
+- `simard self-health` reports `[FAIL] no_quarantine   quarantined=true`, **and**
+- the overseer keeps emitting "DeployDrift — running binary is N commit(s) behind
+  merged main", **and**
+- the offending artifact is the long-lived **recovery asset** (the largest
+  `cognitive*.corrupt-<ts>` file, which `simard cleanup` deliberately never
+  deletes — see issue #2550).
+
+If the quarantine is **fresh** (recent corruption you have not yet
+investigated), do **not** acknowledge it — investigate the corruption first. The
+autonomous auto-ack only ever touches the protected recovery asset once it is
+older than the 30-day forensic window; everything else stays red by design.
+
+## Step 1 — Confirm the deadlock
+
+```console
+$ simard self-health
+simard self-health: UNHEALTHY
+  [ok  ] version_advanced   running=<commit> target=<commit>
+  [ok  ] memory_intact      live_facts=1206 baseline=n/a
+  [ok  ] goal_board_intact  active_goals=5
+  [ok  ] brains_llm_backed  fallback_records=0
+  [FAIL] no_quarantine      quarantined=true
+  [ok  ] entrypoint_parity  path=/home/you/.local/bin/simard version=simard 0.35.0 mismatch=false foreign=false
+```
+
+Only `no_quarantine` is red, and the artifact is the retained recovery asset.
+Inspect what is present under **both** locations the probe scans — the
+top-level state root and the live-store subdir `<state_root>/state/` (where the
+de-forked backend drops corrupt snapshots):
+
+```console
+$ ls -1 ~/.simard/ ~/.simard/state/ 2>/dev/null | grep '\.corrupt-'
+cognitive.corrupt-20260601T090412Z        # large recovery asset — retained by #2550
+```
+
+(If `SIMARD_STATE_ROOT` is set, look under that root and its `state/` subdir
+instead — the probe, the acknowledge path, and `simard cleanup` all resolve the
+same directory set.)
+
+## Step 2 — Acknowledge the quarantine
+
+```console
+$ simard self-health --acknowledge-quarantine
+simard self-health: HEALTHY
+  [ok  ] version_advanced   running=<commit> target=<commit>
+  [ok  ] memory_intact      live_facts=1206 baseline=n/a
+  [ok  ] goal_board_intact  active_goals=5
+  [ok  ] brains_llm_backed  fallback_records=0
+  [ok  ] no_quarantine      quarantined=false
+  [ok  ] entrypoint_parity  path=/home/you/.local/bin/simard version=simard 0.35.0 mismatch=false foreign=false
+```
+
+This writes an `.ack` sidecar next to each present quarantine artifact — in
+both the top-level state root and `<state_root>/state/` — and re-runs the probe.
+The artifact is **not** deleted:
+
+```console
+$ ls -1 ~/.simard/ | grep '\.corrupt-'
+cognitive.corrupt-20260601T090412Z          # still here — recovery asset retained
+cognitive.corrupt-20260601T090412Z.ack      # acknowledgement sidecar
+```
+
+The command is idempotent — running it again is safe and reports the artifacts
+as already acknowledged. Exit code is `0` once every probe is healthy.
+
+## Step 3 — Confirm self-deploy converges
+
+With `no_quarantine` green, `all_healthy()` reaches `true`, the post-deploy
+health check passes, and the next deploy is accepted instead of rolled back:
+
+```console
+$ simard self-deploy
+# … canary + gates pass, swap accepted, health check HEALTHY …
+
+$ simard self-health
+simard self-health: HEALTHY
+```
+
+The recurring "DeployDrift — N commit(s) behind merged main" signal stops once
+the running binary advances to merged `main`.
+
+## Reversing an acknowledgement
+
+Acknowledgement is reversible. Delete the sidecar to make the probe count the
+artifact again:
+
+```console
+$ rm ~/.simard/cognitive.corrupt-20260601T090412Z.ack
+$ simard self-health          # no_quarantine reddens again
+```
+
+Deleting the sidecar never affects the quarantine artifact itself.
+
+## What this does *not* do
+
+- It does **not** delete the quarantine artifact — the #2550 recovery asset is
+  retained so you can still salvage records from it.
+- It does **not** silence *future* corruption. A new corruption event writes a
+  new `cognitive.corrupt-<ts>` artifact with no sidecar, so `no_quarantine`
+  reddens again immediately and self-deploy blocks — exactly as intended.
+- It does **not** change any other probe or the `self-health` exit-code
+  convention.
+
+## See also
+
+- [Self-deploy quarantine-acknowledge reference](../reference/self-deploy-quarantine-acknowledge.md)
+  — the `.ack` convention, the `quarantine_ack` API, and the guarded auto-ack.
+- [Self-deploy API reference](../reference/self-deploy-api.md#simard-self-health)
+  — the `simard self-health` subcommand and the six probes.
+- [Verify and roll back a self-deploy](../howto/verify-and-roll-back-a-self-deploy.md).

docs/reference/overseer-deploy-canary-diagnostics.md

@@ -173,6 +173,48 @@ downstream sink at once. `refusal_reason` re-applies the same idempotent bound
 defensively for `CanaryResult`s built by other paths. Truncation never splits a
 multi-byte character.
 
+### `unit-test` gate `first_failure=` detail (#4470)
+
+The `failing_detail` surfaced above is only as useful as the underlying
+`GateResult.detail`. For the `unit-test` gate — the gate that reddened the
+self-deploy canary in the #4470 incident — the raw `cargo test` stderr tail
+often does **not** contain the failing test's name near the end, so the bounded
+512-byte tail could name no test at all. `run_unit_test_gate`
+([`src/self_relaunch/gates.rs`](https://github.com/rysweet/Simard/blob/main/src/self_relaunch/gates.rs))
+therefore **extracts the first failing test path** from the full `cargo test`
+output and prepends it to the gate detail as a stable `first_failure=` prefix:
+
+```text
+tests failed (exit 101): first_failure=<crate>::<module>::<test_name>; <bounded stderr tail>
+```
+
+| Field | Meaning |
+| --- | --- |
+| `first_failure=<test::path>` | The first test path parsed from a `test <path> ... FAILED` line (or the `failures:` block) in the `cargo test` output. Omitted only when no test name can be parsed (e.g. a link/compile abort with no test lines) — the bounded stderr tail is still included. |
+| `<bounded stderr tail>` | The existing truncated stderr, unchanged. |
+
+Extraction rules:
+
+- **Parsed from the runner output**, not guessed — it reads the `... FAILED`
+  lines / `failures:` section that `cargo test` emits. The first failing test
+  wins (deterministic).
+- **Bounded** to ≤ 512 bytes total, at a UTF-8 char boundary, consistent with the
+  `failing_detail` cap above.
+- **Sanitized**: CR, LF, and other control characters are stripped from the
+  parsed test name before it is embedded, so the detail is a single clean line
+  and cannot forge additional log fields or JSON. The parsed name is treated as
+  **data, not a format string**.
+- **Schema-stable**: `GateResult` keeps its `{ gate, passed, detail }` shape;
+  only the *content* of `detail` is enriched. `exit 101` (a Rust test-binary
+  panic/abort) still surfaces as before, now accompanied by the specific test.
+
+Because the failing gate's `detail` is what `TargetCanaryReport.failing_detail`
+copies from, the `first_failure=` prefix rides all the way up to the operator
+`deploy_refused` reason, the `overseer::deploy` WARN, and the `failing_detail`
+OTel attribute — so a red `unit-test` canary now names the exact test to fix in
+one glance. Acting on it is covered in
+[STEP 2: acting on the surfaced detail](#step-2-acting-on-the-surfaced-detail).
+
 ### `CanaryResult::refusal_reason`
 
 A new inherent method composes the enriched, human-readable refusal string
@@ -386,3 +428,6 @@ weakened or disabled to mask a real regression.
   WARN event and the per-problem detail rows.
 - [Overseer tick self-healing](./overseer-tick-self-healing.md) — the
   `is_transient` fail-closed classifier and the SR-1 latch invariant.
+- [Self-deploy quarantine-acknowledge](./self-deploy-quarantine-acknowledge.md)
+  — the paired `no_quarantine` deadlock fix (#4469): the *other* self-deploy
+  blocker that had to clear alongside the red canary for self-deploy to converge.

docs/reference/self-deploy-api.md

@@ -10,6 +10,7 @@ related:
   - ../concepts/reconcile-and-self-deploy.md
   - ../concepts/operational-autonomy-model.md
   - ./self-deploy-source-prep.md
+  - ./self-deploy-quarantine-acknowledge.md
   - ./overseer-operator-notifications.md
   - ./overseer-tick-details.md
   - ../safe-self-update.md
@@ -334,16 +335,31 @@ internally. There are **six** probes: `version_advanced`, `memory_intact`,
 [`entrypoint_parity`](#entrypointparityprobe).
 
 ```text
-simard self-health [--json] [--pre-deploy-facts=N]
+simard self-health [--json] [--pre-deploy-facts=N] [--acknowledge-quarantine]
 
   --json               Emit the SelfHealthReport as JSON (default: human table).
   --pre-deploy-facts   Baseline fact count to compare against (the orchestrator
                        passes the count captured before the swap). When omitted,
                        the "memory intact" probe reports the live count only.
+  --acknowledge-quarantine
+                       Acknowledge every currently-present cognitive-memory
+                       quarantine artifact under the state root (writing an
+                       `.ack` sidecar next to each) before probing, so a
+                       genuinely-stuck quarantine clears the `no_quarantine`
+                       probe WITHOUT deleting the #2550 recovery asset.
+                       Idempotent. See the quarantine-acknowledge reference.
 
 Exit code: 0 when every probe is healthy; non-zero when any probe fails.
 ```
 
+The additive `--acknowledge-quarantine` flag resolves the `no_quarantine`
+deadlock (#4469) in which the retained #2550 recovery asset keeps the probe red
+forever. Acknowledgement silences the probe for a specific, named artifact but
+never deletes it, and a *new* corruption event reddens the probe again. The full
+`.ack` convention, the `quarantine_ack` module API, the ack-aware
+`count_quarantine_files`, and the guarded autonomous auto-ack are specified in
+[self-deploy quarantine-acknowledge](./self-deploy-quarantine-acknowledge.md).
+
 ### `self-health` output
 
 ```json
@@ -420,6 +436,16 @@ both lets operators tell "clean store" (`0` / `0`) apart from "clean since deplo
 N forensic snapshots retained" (`0` / `N`) directly from the health JSON, without
 inspecting the store directory by hand.
 
+> **Acknowledgement-aware counting (#4469).** The probe's JSON schema is unchanged,
+> but both `count_quarantine_files` and the fresh/retained tally count only
+> **unacknowledged** `cognitive*.corrupt-<ts>` artifacts: an artifact with a
+> sibling `.ack` sidecar — and the `.ack` files themselves — are skipped, counting
+> as neither fresh nor retained. This lets a genuinely-stuck quarantine (the
+> retained #2550 recovery asset) clear so `all_healthy()` can converge without
+> deleting it, while a *new* (unacknowledged) corruption event still reddens the
+> probe. See
+> [self-deploy quarantine-acknowledge](./self-deploy-quarantine-acknowledge.md).
+
 > **Known limitation — mtime freshness.** Freshness is keyed on filesystem mtime.
 > Any operation that rewrites the mtime of a *retained* historical snapshot —
 > a rename, a `.bak` copy that preserves the original name, or a manual `touch` —