feat(cognitive-memory): durable fact_snapshot_dedup_ratio graph-memory hygiene self-metric

docs/reference/cognitive-memory-provenance.md

@@ -450,9 +450,61 @@ self-metrics they sit alongside.
 
 ---
 
+## Observability: snapshot-dedup-hygiene self-metric
+
+Grounding coverage watches whether facts enter the graph *connected*; a sibling
+self-metric watches whether the **goal-board snapshot layer stays lean**.
+Goal-board snapshots are revisioned: each new revision `SUPERSEDES` the prior
+one, and
+`prune_superseded` (controlled forgetting) reclaims the archived revisions over
+time. `graph_stats()` already reports two raw counts for this layer:
+
+| Field | Meaning |
+|---|---|
+| `snapshot_facts_total` | every goal-board snapshot revision still held (live + not-yet-pruned superseded) |
+| `distinct_snapshot_caller_keys` | distinct logical goal-board snapshot streams behind them |
+
+The durable `goal_board_snapshot_dedup_ratio` self-metric is the hygiene *health*
+signal derived from them. After each successful OODA cycle, when `graph_stats()`
+succeeds, the daemon emits one sample to the `metrics.jsonl` series from the
+**same** snapshot it already collects for the OTel edge gauges and the
+grounding-coverage metric:
+
+```
+goal_board_snapshot_dedup_ratio = distinct_snapshot_caller_keys / snapshot_facts_total
+```
+
+- **What it measures.** The average *liveness* of goal-board snapshot streams,
+  in `[0.0, 1.0]`. `1.0` means every stream holds a single live revision; the
+  value falls toward `0` as superseded revisions pile up. When every snapshot
+  fact has a valid caller key, its inverse — total / distinct — is the mean
+  revisions retained per stream.
+- **Why it matters.** That accumulation is exactly the monotonic-growth failure
+  controlled forgetting exists to prevent: if `prune_superseded` stops keeping
+  pace, archived revisions bloat semantic memory. Previously that was visible
+  only as the raw `graph_stats()` counts. The store-size-independent ratio adds
+  a durable, comparable history for operator analysis and future automated
+  regression detection; it does not currently create a Gym history signal.
+- **Undefined on an empty goal-board snapshot layer.** When the store holds zero
+  goal-board snapshot facts, the ratio is *undefined* and **no** sample is
+  emitted (skip rather than
+  drag the series to a misleading `0.0`), mirroring the `fact_provenance_coverage`
+  convention. `distinct_snapshot_caller_keys` is clamped to `snapshot_facts_total`
+  defensively so a miscount can never yield a ratio above `1.0`. The emitter is
+  best-effort — a metrics-write failure is logged, never propagated — and pure
+  observation: it never changes memory state.
+
+The scoring is a pure function
+(`cognitive_memory::metrics::goal_board_snapshot_dedup_ratio`) with the
+per-cycle emitter (`record_goal_board_snapshot_dedup_ratio_metric`) beside
+`record_provenance_coverage_metric`, so both graph-memory hygiene self-metrics
+sit together.
+
+---
+
 ## Testing
 
-The feature is covered by a TDD round-trip test (in
+Provenance is covered by a TDD round-trip test (in
 `src/cognitive_memory/tests_provenance.rs`) that:
 
 1. opens an in-memory backend via `LibraryCognitiveMemory::in_memory()`
@@ -468,11 +520,29 @@ returns an empty list) and passes once the adapter records and traverses
 the `DERIVES_FROM` edge — proving the link is recallable end-to-end
 through Simard's own API.
 
+Goal-board snapshot hygiene is covered by:
+
+1. ratio boundary tests in `src/cognitive_memory/metrics.rs`,
+2. a hermetic injected-writer metric-entry construction test that asserts the
+   metric name, value, and serialized context (it does not exercise the real
+   JSONL storage path), and
+3. a daemon wiring test that passes asymmetric `GraphStats` counts through
+   `record_graph_memory_self_metrics` and the goal-board emitter's injected
+   writer, then asserts the resulting metric name, `0.25` value, and serialized
+   `snapshot_facts` / `distinct_caller_keys` context. The asymmetric counts make
+   the test fail if the numerator and denominator are reversed.
+
+The process-boundary `memory stats --json` test in
+`tests/bin_simard_memory_cli.rs` separately proves the same goal-board counts
+are exposed to operators.
+
 Run the relevant suites with:
 
 ```bash
 cargo test cognitive_memory
 cargo test memory_consolidation
+cargo test graph_memory_metric_sweep_uses_goal_board_graph_stats_fields
+cargo test --test bin_simard_memory_cli stats_shows_edges_and_dedup_section_via_direct_open
 ```
 
 ---
@@ -488,10 +558,13 @@ cargo test memory_consolidation
   `facts_with_provenance` / `facts_total` snapshot the coverage metric reads.
 - `src/cognitive_memory/metrics.rs` — `provenance_coverage()` (pure ratio) and
   `record_provenance_coverage_metric()` (per-cycle `fact_provenance_coverage`
-  emitter), plus the `GraphStats` snapshot type in `src/memory_cognitive.rs`.
+  emitter), plus the sibling `goal_board_snapshot_dedup_ratio()` /
+  `record_goal_board_snapshot_dedup_ratio_metric()` (per-cycle
+  `goal_board_snapshot_dedup_ratio` emitter), plus the `GraphStats` snapshot
+  type in `src/memory_cognitive.rs`.
 - `src/operator_commands_ooda/daemon/mod.rs` — the per-cycle sweep that reads
-  `graph_stats()` for the OTel edge gauges and emits the coverage self-metric
-  from the same snapshot.
+  `graph_stats()` for the OTel edge gauges and emits both graph-memory
+  self-metrics from the same snapshot.
 - `src/memory_consolidation/distillation.rs` — distillation writer that
   threads `source_episode_id` as `DERIVES_FROM` provenance.
 - `src/memory_consolidation/mod.rs` — `reflection_memory_operations` that

docs/reference/telemetry-metrics.md

@@ -187,6 +187,17 @@ gauges the section renders `absent`, never a fabricated zero.
 > comparable and regressable, not just a raw count. See
 > [Cognitive-memory provenance § Observability](./cognitive-memory-provenance.md#observability-grounding-coverage-self-metric).
 
+> **Goal-board snapshot hygiene.** A sibling durable
+> **`goal_board_snapshot_dedup_ratio`**
+> self-metric emits, from the same per-cycle `graph_stats()` snapshot, the
+> average *liveness* of goal-board snapshot revisions
+> (`distinct_snapshot_caller_keys / snapshot_facts_total` ∈ `[0, 1]`, higher is
+> healthier). It falls when superseded snapshot revisions accumulate faster than
+> controlled forgetting (`prune_superseded`) reclaims them, turning a pruning
+> regression into a durable time series for operator and future automated
+> analysis rather than only a raw count. See
+> [Cognitive-memory provenance § Snapshot dedup hygiene](./cognitive-memory-provenance.md#observability-snapshot-dedup-hygiene-self-metric).
+
 ### LLM usage — `simard.llm.*`
 
 Mirrored from `cost_tracking` (the ledger format is unchanged; these are

src/cognitive_memory/metrics.rs

@@ -272,6 +272,105 @@ pub fn record_provenance_coverage_metric(facts_with_provenance: u64, facts_total
     }
 }
 
+// ─────────────────────── graph-memory snapshot dedup hygiene ────────────────
+//
+// A graph-memory *hygiene* signal complementary to grounding coverage above.
+// Goal-board snapshot facts are revisioned: each new revision SUPERSEDES the
+// prior one, and `prune_superseded` (controlled forgetting) reclaims archived
+// revisions over time. `snapshot_facts_total` counts every goal-board snapshot
+// revision the store still holds (live + not-yet-pruned superseded);
+// `distinct_snapshot_caller_keys` counts the distinct logical streams behind
+// them. Their ratio is the average *liveness* of the goal-board snapshot layer:
+// 1.0 when
+// every stream holds exactly one revision, falling toward 0 as superseded
+// revisions accumulate faster than pruning reclaims them. That accumulation is
+// exactly the monotonic-growth failure controlled forgetting exists to prevent,
+// and — like grounding coverage — it was previously visible only as raw
+// `graph_stats()` counts, never as a durable, comparable `metrics.jsonl`
+// series. Emitting the ratio gives operators a store-size-independent history
+// for manual and future automated regression analysis.
+
+/// Durable self-metric name for goal-board snapshot dedup hygiene, emitted to
+/// `metrics.jsonl` after each successful OODA cycle when `graph_stats()` succeeds
+/// by
+/// [`record_goal_board_snapshot_dedup_ratio_metric`].
+pub const GOAL_BOARD_SNAPSHOT_DEDUP_RATIO_METRIC: &str = "goal_board_snapshot_dedup_ratio";
+
+/// Average liveness of goal-board snapshot facts
+/// (`distinct_snapshot_caller_keys / snapshot_facts_total`), in `[0.0, 1.0]`.
+/// Higher is healthier: `1.0` means every goal-board stream holds a single live
+/// revision; a value approaching `0` means superseded revisions have piled up
+/// When every snapshot fact has a valid caller key, the inverse — total /
+/// distinct — is the mean revisions retained per stream.
+///
+/// Returns `None` (undefined, **not** `0.0`) when the store holds no goal-board
+/// snapshot facts, so an empty goal-board snapshot layer contributes no
+/// misleading `0.0` sample — the same "skip rather than drag the series to
+/// zero" convention [`provenance_coverage`] and [`precision_at_k`] use.
+/// `distinct_snapshot_caller_keys` is clamped to `snapshot_facts_total`
+/// defensively (a stream always has ≥1 revision, so distinct ≤ total holds), so
+/// a backend that miscounts can never yield a ratio above `1.0`.
+pub fn goal_board_snapshot_dedup_ratio(
+    distinct_snapshot_caller_keys: u64,
+    snapshot_facts_total: u64,
+) -> Option<f64> {
+    if snapshot_facts_total == 0 {
+        return None;
+    }
+    let distinct = distinct_snapshot_caller_keys.min(snapshot_facts_total);
+    Some(distinct as f64 / snapshot_facts_total as f64)
+}
+
+/// Emit one durable [`GOAL_BOARD_SNAPSHOT_DEDUP_RATIO_METRIC`] sample (the
+/// goal-board snapshot liveness ratio over the current `graph_stats()` snapshot)
+/// to `metrics.jsonl`.
+///
+/// Called after each successful OODA cycle when the daemon's metric sweep can
+/// read `graph_stats()`, from the same block that records OpenTelemetry edge
+/// gauges and [`record_provenance_coverage_metric`], so it adds no extra store
+/// read. A snapshot-shaped metric (store state, not a per-cycle accumulator).
+///
+/// No-op when the store holds no goal-board snapshot facts (undefined ratio —
+/// see [`goal_board_snapshot_dedup_ratio`]), so the series carries signal only.
+/// Best-effort: a metrics-write failure is logged, never propagated.
+pub fn record_goal_board_snapshot_dedup_ratio_metric(
+    distinct_snapshot_caller_keys: u64,
+    snapshot_facts_total: u64,
+) {
+    if cfg!(test) {
+        return;
+    }
+    if let Err(e) = record_goal_board_snapshot_dedup_ratio_metric_with(
+        distinct_snapshot_caller_keys,
+        snapshot_facts_total,
+        crate::self_metrics::record_metric,
+    ) {
+        tracing::warn!(
+            target: "simard::memory",
+            error = %e,
+            "failed to record goal_board_snapshot_dedup_ratio metric (memory unaffected)",
+        );
+    }
+}
+
+pub(crate) fn record_goal_board_snapshot_dedup_ratio_metric_with<E>(
+    distinct_snapshot_caller_keys: u64,
+    snapshot_facts_total: u64,
+    writer: impl FnOnce(&str, f64, &str) -> Result<(), E>,
+) -> Result<(), E> {
+    let Some(ratio) =
+        goal_board_snapshot_dedup_ratio(distinct_snapshot_caller_keys, snapshot_facts_total)
+    else {
+        return Ok(());
+    };
+    let context = serde_json::json!({
+        "snapshot_facts": snapshot_facts_total,
+        "distinct_caller_keys": distinct_snapshot_caller_keys,
+    })
+    .to_string();
+    writer(GOAL_BOARD_SNAPSHOT_DEDUP_RATIO_METRIC, ratio, &context)
+}
+
 #[cfg(test)]
 mod tests {
     use super::*;
@@ -415,4 +514,61 @@ mod tests {
         record_provenance_coverage_metric(3, 4);
         record_provenance_coverage_metric(0, 0);
     }
+
+    // ── graph-memory snapshot dedup hygiene: pure math ──────────────────────
+
+    #[test]
+    fn snapshot_dedup_ratio_is_distinct_streams_over_total_revisions() {
+        // Two streams, four revisions retained → each stream averages two
+        // revisions → liveness 0.5. One-revision-per-stream is a healthy 1.0.
+        assert_eq!(goal_board_snapshot_dedup_ratio(2, 4), Some(0.5));
+        assert_eq!(goal_board_snapshot_dedup_ratio(4, 4), Some(1.0));
+        assert_eq!(goal_board_snapshot_dedup_ratio(1, 8), Some(0.125));
+        // Snapshot facts present but none carry a grouping key → distinct 0 over
+        // a nonzero total is a real, maximally-unhealthy 0.0 (emit it), NOT the
+        // undefined None reserved for an empty goal-board snapshot layer.
+        assert_eq!(goal_board_snapshot_dedup_ratio(0, 4), Some(0.0));
+    }
+
+    #[test]
+    fn snapshot_dedup_ratio_is_none_for_an_empty_snapshot_layer() {
+        // No snapshot facts → undefined ratio (skip, do NOT emit a misleading
+        // 0.0), matching the provenance_coverage / precision@k convention. A
+        // nonzero distinct count with a zero denominator is still None.
+        assert_eq!(goal_board_snapshot_dedup_ratio(0, 0), None);
+        assert_eq!(goal_board_snapshot_dedup_ratio(3, 0), None);
+    }
+
+    #[test]
+    fn snapshot_dedup_ratio_clamps_overcount_to_one() {
+        // distinct ≤ total always holds (a stream has ≥1 revision); a backend
+        // that miscounts must never yield a ratio above 1.0.
+        assert_eq!(goal_board_snapshot_dedup_ratio(9, 4), Some(1.0));
+    }
+
+    #[test]
+    fn record_goal_board_snapshot_dedup_ratio_metric_builds_expected_entry() {
+        let mut recorded = None;
+        record_goal_board_snapshot_dedup_ratio_metric_with(2, 4, |name, value, context| {
+            recorded = Some((name.to_string(), value, context.to_string()));
+            Ok::<(), ()>(())
+        })
+        .expect("record metric through injected writer");
+
+        let (name, value, context) = recorded.expect("one metric entry");
+        assert_eq!(name, GOAL_BOARD_SNAPSHOT_DEDUP_RATIO_METRIC);
+        assert_eq!(value, 0.5);
+        let context: serde_json::Value =
+            serde_json::from_str(&context).expect("metric context JSON");
+        assert_eq!(context["snapshot_facts"], 4);
+        assert_eq!(context["distinct_caller_keys"], 2);
+
+        let mut called = false;
+        record_goal_board_snapshot_dedup_ratio_metric_with(0, 0, |_, _, _| {
+            called = true;
+            Ok::<(), ()>(())
+        })
+        .expect("empty snapshot layer is a no-op");
+        assert!(!called);
+    }
 }