feat: freshness engine — keep code_verified claims honest as code changes

[divo12]

Summary

Keeps code_verified memory claims honest as the code changes. Today a claim is checked against the code exactly once — at write time — and then drifts silently: a refactor renames a function, but the claim keeps its code_verified badge while pointing at code that no longer says what it claims. Worse than no fact — it hands the next agent a confident lie.

This PR builds a freshness engine that closes the loop end to end:

detect → surface → demote → re-verify

A per-anchor content fingerprint is the authority. The foreground (every graph context query) labels claims fresh/stale/unknown and quarantines stale ones — read-only. The background (hook worker) heals the durable graph: it demotes genuinely-drifted claims and hands them to an agent to rewrite. The compiler (tree-sitter) + a content hash are the judges — deterministic, no LLM in the detection path, zero false positives.

Why

Two ways a code_verified claim rots after it's written:

Kind	What happens	Detected before this PR?
Structural drift	the anchored symbol is renamed / moved / deleted → the anchor no longer resolves	detect-only (graph audit anchors), never healed
Content drift	the symbol still exists at the same file#symbol, but its body changed → the fact is now wrong	❌ invisible — anchor resolves fine, so audit/query report it healthy

A per-anchor content hash of the resolved span unifies both: fresh iff the anchor resolves and the span hash matches the stored one; stale iff it won't resolve (structural) or the hash changed (content). Because the hash is taken over the working-tree file, it also catches uncommitted edits.

How it works

flowchart TD
    subgraph Write["① Write time — applyProposal"]
      A["agent saves a code_verified claim"] -->|hash resolved span| FP[("anchor_fingerprints<br/>baseline hash per anchor")]
    end

    subgraph FG["② Foreground — every graph context query (READ-ONLY)"]
      Q["graph context query"] --> RC["resolve anchors<br/>+ stat-prefiltered re-hash"]
      RC --> CF{"classifyFreshness"}
      CF -->|fresh / unknown| BEST["## Best Claims<br/>(unknown gets an 'unverifiable' caveat)"]
      CF -->|stale| NRV["## Needs re-verification<br/>(distrust signal to the agent)"]
    end

    subgraph BG["③ Background — hook worker on session end"]
      H["healDriftedAnchors<br/>changed files since checkpoint (git diff ∪ status)"] --> RIDX["reverse index: file → claims"]
      RIDX --> CF2{"classifyFreshness"}
      CF2 -->|fresh / unknown| SKIP["skip (early cutoff)"]
      CF2 -->|stale| DEM["demote: supersede → truth: unknown<br/>+ invalidation_events (one txn)"]
      DEM --> WL["re-verify worklist"]
      WL --> AG["④ agent re-reads current code,<br/>writes a corrected code_verified claim"]
    end

    FP -. baseline .-> RC
    FP -. baseline .-> CF2
    AG -. supersedes the unknown claim .-> FP

Loading

The single rule (code-anchors/freshness.ts, shared by both planes so the label the agent sees can never disagree with the demotion the daemon writes):

classifyFreshness(checks) →
  stale · structural   if every anchor stopped resolving
  stale · content      if a still-resolving anchor's span hash changed
  unknown              if a span can't be read right now (never a false "fresh")
  fresh                otherwise

Cheap by construction: a stat prefilter (mtime+size) skips the re-hash when a file is untouched; a reverse file → claims index means the background only re-checks claims in files that actually changed (never a whole-graph scan); early cutoff means no write when nothing drifted; a per-repo checkpoint (last-checked SHA) scopes each background pass.

Architecture & folder structure

New code is small and slots into the existing layer-organized knowledge-graph lib (pure domain → util/IO → repository → service → surface). Nothing reaches "upward" into apps/.

libs/
├── knowledge-graph/
│   ├── code-anchors/
│   │   ├── freshness.ts          ● DOMAIN  the one fresh/stale/unknown rule (classifyFreshness)
│   │   ├── span-hash.ts          ○ UTIL    sha256 of an anchored span (+ stat); realpath-contained to the repo
│   │   └── drift.ts              ● DOMAIN  structural-only drift scan (Option A: demote iff *all* anchors broken)
│   ├── anchor-fingerprints.ts    ● DOMAIN  index stored fingerprints; build stat-prefiltered AnchorChecks
│   ├── anchor-invalidation.ts    ● DOMAIN  pure: demoted claims → supersession writes (rebuilt claim + edges + event)
│   ├── invalidation.ts           ● DOMAIN  InvalidationEvent types + reasons (anchor_drift | content_drift)
│   ├── graph-context/
│   │   ├── claim-freshness.ts    ◐ SURFACE attachFreshness — one batched read, verdict per retrieved claim (read-only)
│   │   ├── context-builder.ts    ◐ SURFACE wires attachFreshness into the query path
│   │   ├── render.ts             ◐ SURFACE renders "## Needs re-verification" + truth on each claim
│   │   └── types.ts              ◐ SURFACE + freshness field on ClaimContextResult
│   └── service.ts                ■ SERVICE orchestration: applyProposal→writeFingerprints, healDriftedAnchors,
│                                            reverifyWorklist, invalidateDriftedAnchors (scan→build→apply→embed)
├── storage/sqlite/
│   ├── schema.ts                 ▲ DATA    + anchor_fingerprints, freshness_checkpoints, invalidation_events
│   └── repository.ts             ▲ DATA    fingerprint R/W, reverse index, checkpoint, worklist, demotion txn
├── hooks/worker.ts               ◆ BG      change-scoped heal pass + agent re-verify handoff (best-effort, per-repo isolated)
├── config/greplica-config.ts     ◆ BG      session.autoHealDrift flag (default on)
└── utils/git.ts                  ○ UTIL    git helpers: gitHeadSha + changedFilesSince (diff ∪ status, uncommitted-aware)
apps/cli/main.ts                            graph audit anchors [--invalidate]  (on-demand structural demotion)
scripts/check-*.js                          deterministic tests — no embedding-model download

Legend: ● pure domain · ○ util/IO · ▲ data-access · ■ service · ◐ read surface · ◆ background.

Data model (3 new tables, additive, IF NOT EXISTS)

erDiagram
    anchor_fingerprints {
        TEXT repo_id     "scopes the reverse lookup"
        TEXT claim_id
        TEXT file
        TEXT symbol      "'' sentinel for file-only anchors"
        TEXT content_hash "sha256 of the resolved span"
        INT  file_mtime_ms "stat prefilter"
        INT  file_size     "stat prefilter"
        TEXT resolver_status
        TEXT checked_at
    }
    freshness_checkpoints {
        TEXT repo_id          PK
        TEXT last_checked_sha  "background scopes diff from here"
        TEXT checked_at
    }
    invalidation_events {
        TEXT id                   PK
        TEXT repo_id
        TEXT original_claim_id
        TEXT superseding_claim_id
        TEXT reason               "anchor_drift | content_drift"
        TEXT broken_anchor
        TEXT resolver_status
        TEXT git_commit_sha
    }

Loading

• anchor_fingerprints PK (claim_id, file, symbol); the '' symbol sentinel avoids SQLite's "NULLs are distinct in a composite PK" gotcha so INSERT OR REPLACE stays idempotent. Index (repo_id, file) powers the reverse file → claims lookup, scoped per repo (the DB is shared across repos).
• Insert-only / non-destructive: claims are never mutated or deleted — a demotion supersedes the original with a rebuilt truth: unknown copy (broken anchors kept as evidence, about/evidenced_by edges re-pointed), so full history is preserved. invalidation_events is the queryable drift audit trail.

User-facing surface

# On-demand structural demotion (CLI, unchanged report + new write flag)
greplica graph audit anchors              # report-only — doubles as a dry run; exit 1 in CI if issues
greplica graph audit anchors --invalidate # demote fully-drifted claims (one memory commit + one txn)

• Foreground: every graph context packet now shows each claim's truth, routes stale claims into a ## Needs re-verification section, and tags unknown-freshness claims with an "unverifiable" caveat. Zero graph writes on the read path.
• Background: the hook worker heals drift and hands demoted claims to an agent to rewrite, gated by session.autoHealDrift (default on).

Key design decisions

• One shared verdict rule (classifyFreshness) for both planes — the foreground signal and the background demotion can never disagree.
• Read plane never writes; write plane is the only mutator. A query that hits an uncommitted edit still catches it live (re-hash) without paying to persist.
• unknown never demotes. An unreadable span is flagged unverifiable, never auto-demoted — no false stales.
• Option A (conservative structural policy). A multi-anchor claim demotes structurally only when every anchor is broken.
• Content hash over working tree, not HEAD — catches uncommitted edits (the SHA-gate's blind spot); mtime is only a prefilter, the hash is the authority.
• No re-verify queue table. The worklist is derived from existing state (truth: unknown + an invalidation_events row); a claim drops out for free once an agent supersedes it.

Test plan

• npx tsc --noEmit clean; full npm test green.
• Deterministic suites (no embedding-model download — stub context builder + fake agent runner):
  • check-anchor-drift.js — structural demote/supersede/event/history/edges + Option A + idempotency.
  • check-span-hash.js — span hashing, CRLF normalization, repo-containment (missing / absolute / .. / symlink → safe).
  • check-freshness.js — the classifyFreshness rule (fresh / structural / content / unknown) + fingerprint store + repo-scoped reverse index.
  • check-freshness-foreground.js — per-claim freshness in the builder + ## Needs re-verification rendering (read-only proof).
  • check-freshness-background.js — change-scoped heal (reverse-index scoping, early cutoff, uncommitted-aware, unknown-never-demotes, checkpoint), the re-verify worklist, and the agent-handoff prompt.
• Verified with a real agent, live: drove greplica graph context through all four verdicts (content drift incl. uncommitted, structural (missing symbol), unknown via chmod 000, back to fresh on restore) and confirmed the durable graph was untouched by the read path.

Non-goals / limitations

• Unsaved in-editor buffers (edited, not yet written to disk) are invisible to stat/git/hash — caught on the next query after save.
• Granularity: the fingerprint hashes the whole resolved span, so a whitespace-only edit counts as a change (errs toward re-check).
• Non-code (source_verified) staleness is a future detector behind the same classifyFreshness seam.
• Background heal is coupled to the memory-update worker cycle for v1; decoupling its trigger is a follow-up.

[divo12]

@kushalpatil07 please review the PR , e2e testing with a coding agent will put results soon ; unit & e2e testing with CLI working

[divo12]

Hi @kushalpatil07 ,
Please review the solution and PR. Happy to discuss over it

[kushalpatil07]

Bro this is too extensive. Please discuss with me before implementing. Very unlikely this PR gets merged.
The 3 steps that you have defined, each of those need a lot of thought to be implemented.
Few questions I can think of right now.

1. What span are you using for each symbol and why?
2. Are you doing this for component code anchors as well?
3. Why are you storing invalidation events?
4. Is the checking of fingerprint expensive or not>
5. Should the same hook worker that updates memory solve drift or a new one should be there?

[kushalpatil07]

Closing this broad implementation as superseded by the incremental #111/#128 freshness path. Remaining detection, quarantine, and invalidation behavior stays tracked in #90/#91.