diff --git a/CHANGELOG.md b/CHANGELOG.md index 7547c58..f25e224 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,70 @@ the `mimir-mem` crate, and the on-disk schema move together. ## [Unreleased] ### Added +- **`--confidence certain|likely|unsure` separates how sure the author was + from how often the memory gets used.** `strength` conflated the two: a + guess recalled enough times outranked things that had been checked, with + nothing left to show it was ever uncertain. Confidence is author-declared + at capture, on the same argument `expires_at` and `resolves_when` are + built on — the person writing it is the only one who knows, they know it + at write time, and nobody backfills. A level inferred later by a model + would be exactly the unfalsifiable label this replaces: computed from the + text, so incapable of contradicting it. Absent is a real state and is + **not** a synonym for `likely`; an unparseable level is rejected before + anything is written. Shown in full on `get`, and on the compact recall + line only when `unsure` — that line is what an agent acts from, so the + case worth a token is the one where acting without checking is a mistake. + Does not gate and does not score: the drift-eval baseline is byte-identical. +- **`mimir grounding` — which memories are attached to something Mimir can + re-check, and which of those attachments have broken.** A memory linked + to an indexed artifact (code symbol, source chunk, doc chunk, file) makes + a claim that code can test: the indexer and `graph build` soft-delete + what stops existing, so "this note is about `retry_with_backoff`" becomes + *stale* the moment the symbol goes. Unlike strength, marks or type + priors — all opinions computed from how people treated a claim — this is + the one signal on a memory that Mimir can prove wrong about itself. + Surfaced in `doctor`, in `mimir grounding --stale`, and inline on `get` + (so an agent reading the memory sees it too). Stale does **not** mean + wrong: a note about a renamed function is usually still good. It means + nothing has revisited it since the ground moved, which is why it is + reported and deliberately **not** scored — making grounding a ranking + input is a separate decision that would need the drift-eval baseline + re-cut, not smuggled in behind a display field. +- **`mimir refusals` — an audit trail for the secret guard that is not + itself a pile of secrets.** Refusals now record a blake3 fingerprint of + what was offered, the detector's label, the surface, and first/last seen + with a count — never the value. Repeat offers increment one row rather + than inserting, because one secret offered forty times is an agent in a + loop and forty rows would hide that. `doctor` says so when offers exceed + distinct values. Guarded by a test that sweeps every column of every + table for the plaintext: a record of a leak must never become a second + copy of it. + +### Fixed +- **`mimir remember --link ` now resolves symbol names.** Both the + CLI and MCP advertise "a code symbol or node", but the CLI only ever + called `resolve_ref`, which resolves ids — so linking a memory to + `retry_with_backoff` failed with "no node matching" and the only links + anyone could make by hand were between things they already had ids for. + MCP had the fallback already; the CLI is now at parity. Found while + building grounding, which this is the main path into: shipping it broken + would have repeated the anchors-at-zero-adoption failure exactly. +- **The retrieval eval is now a gate, not just a report.** The hermetic + corpus was committed and deterministic but nothing failed when the + numbers moved: the only assertions were well-formedness checks with an + `MRR > 0.2` floor, and every actual baseline was `#[ignore]`d. Two tests + now run on plain `cargo test` — a committed per-set baseline that must + reproduce **exactly** (in both directions, so an improvement also has to + be written down), and an ablation asserting each scoring knob still beats + its own absence. Verified by zeroing `type_prior_alpha`: previously + green, now two failures naming the metric and the delta. + - Fell out of writing it: **`scoring.code_damp` changes nothing on the + corpus** — same numbers in every category, including the + `code-vs-memory` scenario that exists to guard it, whose single + question ranks the memory first either way. Pinned by a test rather + than papered over, so authoring a fixture that finally exercises the + knob will fail loudly and move it into the ablation where it belongs. + - **`mimir anchor --pattern ...`** sets guard anchors on an *existing* memory. `remember --anchor` only covered capture time, which is why anchor adoption sits at zero in practice: by the time you know @@ -12,6 +76,19 @@ the `mimir-mem` crate, and the on-disk schema move together. the existing set. ### Fixed +- **A forgotten memory can no longer come back on its own.** `forget` set a + tombstone, but duplicate detection only ever looked at live nodes — so + offering the same text again created a fresh node with no warning, and + the deletion left nothing behind that could catch it. An importer re-run, + an extractor re-reading an unchanged source, or an agent that saw the + fact twice would each quietly undo the delete. `remember` now refuses + with the date it was forgotten on, and takes `--force` to override; the + MCP tool has no `force` parameter at all, so an agent cannot resurrect + something a human deleted. Matching is by exact normalized hash *and* by + token overlap, because a reword is the common shape of re-extraction. + Decay-archived nodes (`meta.archived`) are deliberately excluded — nobody + decided those, and refusing them would train everyone to pass `--force` + by reflex. - **The CLI no longer overflows the stack on Windows.** Windows gives the main thread a 1 MiB stack where Linux and macOS give 8, and clap's derived builder for Mimir's ~50 subcommands outgrew it in debug builds: diff --git a/crates/mimir-cli/src/commands.rs b/crates/mimir-cli/src/commands.rs index 83039c2..d8b0fa6 100644 --- a/crates/mimir-cli/src/commands.rs +++ b/crates/mimir-cli/src/commands.rs @@ -2,7 +2,6 @@ use std::collections::HashMap; use anyhow::{anyhow, bail, Context, Result}; use mimir_core::config::{Config, Paths}; -use mimir_core::format::agent_line; use mimir_core::memory::{self, Remember, RememberOutcome}; use mimir_core::model::{now_unix, short_uid, Kind, MemoryType, Node, Rel, Scope}; use mimir_core::search::SearchQuery; @@ -884,6 +883,50 @@ pub fn doctor(check_only: bool) -> Result<()> { }), &mut failures, ); + // Informational, and never a failure: a refusal is the guard + // working. What's worth seeing is a *retry* — one value offered + // repeatedly means something upstream keeps trying to store it + // and doesn't know it's being turned away. + let month_ago = mimir_core::model::now_unix() - 30 * 86_400; + if let Ok((distinct, offers)) = mimir_core::secrets::refusal_counts(&conn, month_ago) { + if distinct > 0 { + check( + "secret guard", + true, + format!( + "{distinct} value(s) refused in the last 30d over {offers} attempt(s)\ + {} (fingerprints only — the values were never stored; \ + `mimir refusals` for detail)", + if offers > distinct { + " — something is retrying" + } else { + "" + } + ), + &mut failures, + ); + } + } + // Also informational. Stale grounding is the one signal here + // that Mimir derived by checking rather than by policy: the + // memory said it was about a symbol, and the symbol is gone. + if let Ok((grounded, stale, ungrounded)) = mimir_core::grounding::tally(&conn) { + if grounded + stale > 0 { + check( + "grounding", + true, + format!( + "{grounded} grounded, {stale} stale, {ungrounded} ungrounded{}", + if stale > 0 { + " — `mimir grounding --stale` to review" + } else { + "" + } + ), + &mut failures, + ); + } + } } Err(e) => check("db", false, e.to_string(), &mut failures), } @@ -968,6 +1011,7 @@ pub fn remember( anchors: Vec, expires_in: Option, resolves_when: Option, + confidence: Option, ) -> Result<()> { let mut mimir = Mimir::open()?; let mtype: MemoryType = mtype.parse()?; @@ -982,12 +1026,31 @@ pub fn remember( })?), None => None, }; + // Same rule: reject before writing. A memory stored without the + // certainty its author asked to record is worse than an error, because + // nothing afterwards reveals the omission. + let confidence = confidence + .as_deref() + .map(|c| { + c.parse::() + .map_err(|_| { + anyhow!("invalid --confidence {c:?}: expected certain, likely or unsure") + }) + }) + .transpose()?; let project = if global { None } else { mimir.project_for_cwd(&std::env::current_dir()?)? }; if let Some(kind) = mimir_core::secrets::scan_capture(&text, &tags, &fires_when) { + mimir_core::secrets::record_refusal( + &mimir.conn, + &mimir_core::secrets::capture_hash(&text, &tags, &fires_when), + kind, + mimir_core::secrets::Surface::CliRemember, + mimir_core::model::now_unix(), + ); bail!(mimir_core::error::Error::Secret(kind)); } let outcome = memory::remember( @@ -1010,7 +1073,7 @@ pub fn remember( println!("{}", line(&node, &projects, snippet)); } if let Some(r) = link_ref { - let target = store::resolve_ref(&mimir.conn, &r)?; + let target = resolve_link_target(&mimir.conn, &r, project.as_ref().map(|p| p.id))?; store::link(&mimir.conn, node.id, target.id, Rel::Relates, 1.0)?; println!("linked → {}", line(&target, &projects, 0)); } @@ -1038,6 +1101,10 @@ pub fn remember( println!("resolves when: {c}"); } } + if let Some(c) = confidence { + store::set_confidence(&mimir.conn, node.id, c)?; + println!("confidence: {c}"); + } // Keep semantic recall fresh; harmless no-op without a model. if let Err(err) = mimir.embed_pending() { tracing::warn!(%err, "embedding new memory failed"); @@ -1048,6 +1115,11 @@ pub fn remember( "refused: near-duplicate of\n {}\nuse --force to store anyway", line(&existing, &projects, snippet) ), + RememberOutcome::Forgotten(gone) => bail!( + "refused: this was forgotten on {}\n {}\nuse --force to bring it back deliberately", + mimir_core::format::full_date(gone.deleted_at.unwrap_or(gone.updated_at)), + line(&gone, &projects, snippet) + ), } } @@ -1120,7 +1192,12 @@ pub fn recall( } else { println!( "{}", - line(&hit.node, &projects, mimir.config.output.snippet_chars) + line_q( + &hit.node, + &projects, + mimir.config.output.snippet_chars, + Some(&query.text) + ) ); } if linked && !json { @@ -1305,6 +1382,96 @@ pub fn mark(reference: &str, useful: bool) -> Result<()> { Ok(()) } +/// Resolve a `--link` / `link:` target by node id *or* by symbol name. +/// +/// Both surfaces advertise "a code symbol or node", but only ever called +/// `store::resolve_ref`, which resolves ids and nothing else — so linking a +/// memory to `retry_with_backoff` failed with "no node matching", and the +/// only links anyone could make by hand were between things they already +/// had ids for. Since a link to an indexed symbol is exactly what makes a +/// memory *grounded* (see `mimir_core::grounding`), leaving this broken +/// would have shipped a grounding feature that nearly nothing could reach — +/// the same shape as the anchors that sat at zero adoption because the only +/// way to set one was at capture time. +fn resolve_link_target( + conn: &rusqlite::Connection, + reference: &str, + project_id: Option, +) -> Result { + match store::resolve_ref(conn, reference) { + Ok(node) => Ok(node), + Err(err) => { + let Some(pid) = project_id else { + return Err(err.into()); + }; + // Symbol lookup is project-scoped; outside a project there is + // nothing to search, so report the original id-shaped error. + mimir_graph::resolve_symbol(conn, pid, reference).map_err(|_| err.into()) + } + } +} + +pub fn grounding(stale_only: bool, limit: usize) -> Result<()> { + let mimir = Mimir::open()?; + let (grounded, stale, ungrounded) = mimir_core::grounding::tally(&mimir.conn)?; + println!("{grounded} grounded, {stale} stale, {ungrounded} ungrounded"); + if !stale_only { + println!( + "\nGrounded means the memory links to something Mimir indexes and can \ + re-check.\nUngrounded is normal — most notes aren't about a specific \ + symbol or file." + ); + return Ok(()); + } + let rows = mimir_core::grounding::stale_memories(&mimir.conn, limit)?; + if rows.is_empty() { + println!("\nnothing stale"); + return Ok(()); + } + let projects = store::project_titles(&mimir.conn)?; + println!(); + for (node, target) in &rows { + println!( + "{}", + line(node, &projects, mimir.config.output.snippet_chars) + ); + println!(" was about: {target} (no longer indexed)"); + } + println!( + "\nStale means the thing it pointed at is gone, not that the memory is \ + wrong.\nRe-link with `mimir link`, retire with `mimir supersede`, or leave it." + ); + Ok(()) +} + +pub fn refusals(limit: usize) -> Result<()> { + let mimir = Mimir::open()?; + let rows = mimir_core::secrets::refusals(&mimir.conn, limit)?; + if rows.is_empty() { + println!("nothing refused"); + return Ok(()); + } + println!( + "{:<24} {:<14} {:>6} {:<12} {:<12}", + "kind", "surface", "offers", "first", "last" + ); + for r in &rows { + println!( + "{:<24} {:<14} {:>6} {:<12} {:<12}", + r.kind, + r.surface, + r.count, + mimir_core::format::full_date(r.first_seen), + mimir_core::format::full_date(r.last_seen) + ); + } + println!( + "\nfingerprints only — the refused values were never written to disk, \ + so there is nothing here to recover them from." + ); + Ok(()) +} + pub fn consolidate(dry_run: bool) -> Result<()> { let mimir = Mimir::open()?; let report = @@ -1371,6 +1538,13 @@ pub fn edit( if let Some(kind) = mimir_core::secrets::scan_capture(&text, tags.as_deref().unwrap_or(&[]), &[]) { + mimir_core::secrets::record_refusal( + &mimir.conn, + &mimir_core::secrets::capture_hash(&text, tags.as_deref().unwrap_or(&[]), &[]), + kind, + mimir_core::secrets::Surface::CliEdit, + mimir_core::model::now_unix(), + ); bail!(mimir_core::error::Error::Secret(kind)); } let edit = memory::Edit { @@ -1812,6 +1986,13 @@ fn finish_import(mimir: &mut Mimir, stats: mimir_core::import::ImportStats) -> R "imported {} memorie(s), skipped {} duplicate(s)", stats.imported, stats.skipped_duplicates ); + if stats.skipped_forgotten > 0 { + println!( + "skipped {} previously forgotten memorie(s) — re-add with \ + `mimir remember --force` if that was wrong", + stats.skipped_forgotten + ); + } let embedded = mimir.embed_pending()?; if embedded > 0 { println!("embedded {embedded} node(s)"); @@ -1899,11 +2080,22 @@ fn parse_since(s: &str) -> Result { } fn line(node: &Node, projects: &HashMap, snippet_chars: usize) -> String { + line_q(node, projects, snippet_chars, None) +} + +/// [`line`] for ranked recall results, where the query is available and the +/// snippet can be centred on what matched. +fn line_q( + node: &Node, + projects: &HashMap, + snippet_chars: usize, + query: Option<&str>, +) -> String { let project = node .project_id .and_then(|id| projects.get(&id)) .map(String::as_str); - agent_line(node, project, snippet_chars) + mimir_core::format::agent_line_for_query(node, project, snippet_chars, query) } fn print_full(node: &Node, mimir: &Mimir, projects: &HashMap) -> Result<()> { @@ -2220,3 +2412,43 @@ mod inject_addr_tests { assert!(inject_addr("http://").is_err()); } } + +#[cfg(test)] +mod link_target_tests { + use super::resolve_link_target; + use mimir_core::model::{Kind, NewNode}; + use mimir_core::store; + + /// `--link` advertises "a code symbol or node" but only ever resolved + /// ids, so linking a memory to a symbol by name failed outright — and a + /// link to an indexed symbol is precisely what makes a memory grounded. + #[test] + fn resolves_a_symbol_by_bare_name_not_just_by_id() { + let conn = mimir_core::db::open_in_memory().unwrap(); + let mut proj = NewNode::new(Kind::Project); + proj.title = Some("probe".into()); + let project = store::insert_node(&conn, proj).unwrap(); + + let mut sym = NewNode::new(Kind::Symbol); + sym.title = Some("retry_with_backoff".into()); + sym.project_id = Some(project.id); + let symbol = store::insert_node(&conn, sym).unwrap(); + + let found = + resolve_link_target(&conn, "retry_with_backoff", Some(project.id)).expect("by name"); + assert_eq!(found.id, symbol.id); + + // The id form must keep working, and must not need a project. + let by_id = resolve_link_target(&conn, &symbol.uid, None).expect("by id"); + assert_eq!(by_id.id, symbol.id); + } + + /// Outside a project there is nothing to search, so the caller should + /// see the id-shaped error rather than a confusing symbol-lookup one. + #[test] + fn unknown_name_still_errors() { + let conn = mimir_core::db::open_in_memory().unwrap(); + assert!(resolve_link_target(&conn, "nope_not_here", None).is_err()); + assert!(resolve_link_target(&conn, "nope_not_here", Some(1)).is_err()); + } +} diff --git a/crates/mimir-cli/src/main.rs b/crates/mimir-cli/src/main.rs index d4ed1d3..56bd0f2 100644 --- a/crates/mimir-cli/src/main.rs +++ b/crates/mimir-cli/src/main.rs @@ -146,6 +146,12 @@ enum Command { /// so the reader can see what would make it wrong. #[arg(long = "resolves-when", value_name = "CONDITION")] resolves_when: Option, + /// How sure you are: certain | likely | unsure. Records the + /// author's certainty, which is a different thing from how often + /// the memory gets used — so a guess stays visibly a guess no + /// matter how much it is recalled. Does not affect ranking. + #[arg(long, value_name = "LEVEL")] + confidence: Option, }, /// Search memories (and later docs/code) with hybrid ranking. Recall { @@ -322,6 +328,23 @@ enum Command { #[arg(long)] dry_run: bool, }, + /// Whether memories are attached to something Mimir can re-check, and + /// which of those attachments have since broken. + Grounding { + /// List the memories whose linked artifact is gone. + #[arg(long)] + stale: bool, + /// How many to list. + #[arg(long, default_value_t = 20)] + limit: usize, + }, + /// What the secret guard turned away — fingerprints and counts, never + /// the refused values. + Refusals { + /// How many rows to show (most recently offered first). + #[arg(long, default_value_t = 20)] + limit: usize, + }, /// Import memories from the tools Mimir replaces. Import { #[command(subcommand)] @@ -712,6 +735,7 @@ fn run(cli: Cli) -> anyhow::Result<()> { anchors, expires_in, resolves_when, + confidence, } => commands::remember( cli.json, text.join(" "), @@ -724,6 +748,7 @@ fn run(cli: Cli) -> anyhow::Result<()> { anchors, expires_in, resolves_when, + confidence, ), Command::Recall { query, @@ -829,6 +854,8 @@ fn run(cli: Cli) -> anyhow::Result<()> { commands::mark(&reference, useful) } Command::Consolidate { dry_run } => commands::consolidate(dry_run), + Command::Refusals { limit } => commands::refusals(limit), + Command::Grounding { stale, limit } => commands::grounding(stale, limit), Command::Import { cmd } => match cmd { ImportCmd::Openbrain { file } => commands::import_openbrain(&file), ImportCmd::ClaudeMemory { dir } => commands::import_claude_memory(&dir), diff --git a/crates/mimir-cli/src/mcp.rs b/crates/mimir-cli/src/mcp.rs index 8f0bd2b..c91bc2b 100644 --- a/crates/mimir-cli/src/mcp.rs +++ b/crates/mimir-cli/src/mcp.rs @@ -8,7 +8,7 @@ use std::sync::{Arc, Mutex}; use anyhow::Result; -use mimir_core::format::agent_line; +use mimir_core::format::{agent_line, agent_line_for_query}; use mimir_core::memory::{self, RememberOutcome}; use mimir_core::model::{short_uid, Kind, MemoryType, Rel, Scope}; use mimir_core::search::SearchQuery; @@ -103,6 +103,14 @@ pub struct RememberArgs { /// condition is an event rather than a date. #[serde(default)] pub resolves_when: Option, + /// How sure you are that this is true: "certain", "likely" or + /// "unsure". Record what you actually knew when you wrote it — this is + /// tracked separately from how often the memory gets used, so a guess + /// stays legible as a guess however much it is later recalled. Leave + /// unset rather than guessing a level; absent means undeclared, not + /// medium. Does not affect ranking. + #[serde(default)] + pub confidence: Option, } #[derive(Deserialize, schemars::JsonSchema)] @@ -274,10 +282,11 @@ impl MimirServer { .project_id .and_then(|id| projects.get(&id)) .map(String::as_str); - out.push(agent_line( + out.push(agent_line_for_query( &hit.node, project, m.config.output.snippet_chars, + Some(&query.text), )); } Ok(out.join("\n")) @@ -300,6 +309,17 @@ impl MimirServer { if let Some(kind) = mimir_core::secrets::scan_capture(&args.text, &args.tags, &args.fires_when) { + mimir_core::secrets::record_refusal( + &m.conn, + &mimir_core::secrets::capture_hash( + &args.text, + &args.tags, + &args.fires_when, + ), + kind, + mimir_core::secrets::Surface::McpRemember, + mimir_core::model::now_unix(), + ); return Err(engine_err(mimir_core::error::Error::Secret(kind))); } let outcome = memory::remember( @@ -365,6 +385,18 @@ impl MimirServer { ) .map_err(engine_err)?; } + // Same contract as expires_in: an unparseable level is + // an error, not a silently undeclared memory. + if let Some(spec) = args.confidence.as_deref() { + let level: mimir_core::model::MemoryConfidence = + spec.parse().map_err(|_| { + engine_err(mimir_core::error::Error::Invalid(format!( + "invalid confidence {spec:?}: expected certain, likely \ + or unsure" + ))) + })?; + store::set_confidence(&m.conn, node.id, level).map_err(engine_err)?; + } let mut msg = format!("stored {}", line(&node)); if let Some(target) = args.link.as_deref() { let resolved = store::resolve_ref(&m.conn, target).ok().or_else(|| { @@ -392,6 +424,12 @@ impl MimirServer { "refused: near-duplicate of\n{}\n(edit that entry instead, or rephrase if genuinely different)", line(&existing) )), + RememberOutcome::Forgotten(gone) => Ok(format!( + "refused: a human forgot this on {}\n{}\n(it was deleted on purpose — do not re-add it \ + unless the user asks; if they do, they can run `mimir remember --force`)", + mimir_core::format::full_date(gone.deleted_at.unwrap_or(gone.updated_at)), + line(&gone) + )), } }) .await @@ -1656,6 +1694,7 @@ mod tests { anchors: Vec::new(), expires_in: None, resolves_when: None, + confidence: None, } } @@ -1759,6 +1798,7 @@ mod tests { anchors: vec!["deploy.sh".into()], expires_in: None, resolves_when: None, + confidence: None, })) .await; assert!(stored.starts_with("stored"), "remember: {stored}"); @@ -1809,6 +1849,7 @@ mod tests { anchors: Vec::new(), expires_in: None, resolves_when: None, + confidence: None, })) .await; assert!( diff --git a/crates/mimir-core/src/db/migrations.rs b/crates/mimir-core/src/db/migrations.rs index d650358..f659c31 100644 --- a/crates/mimir-core/src/db/migrations.rs +++ b/crates/mimir-core/src/db/migrations.rs @@ -197,6 +197,27 @@ CREATE TABLE session_state ( PRIMARY KEY (session_id, key) ); CREATE INDEX session_state_at ON session_state(updated_at); +"#, + // Refusal ledger: what the secret guard turned away, WITHOUT the thing + // it turned away. `hash` is blake3 over the normalized text — enough to + // recognize the same value being offered again, useless for recovering + // it. Storing the value would make the record of a leak a copy of the + // leak, which is the one outcome worse than having no record. + // + // `kind` is the detector's own label ("an AWS access key"), never a + // fragment of the match. Repeat offers update `last_seen`/`count` + // rather than inserting, so a looping agent shows up as one row with a + // high count instead of flooding the table. + r#" +CREATE TABLE refusal ( + hash BLOB PRIMARY KEY, + kind TEXT NOT NULL, + surface TEXT NOT NULL, + first_seen INTEGER NOT NULL, + last_seen INTEGER NOT NULL, + count INTEGER NOT NULL DEFAULT 1 +); +CREATE INDEX refusal_last_seen ON refusal(last_seen); "#, ]; @@ -284,6 +305,54 @@ mod tests { assert_eq!(n, 1, "old rows must be searchable (stemmed) after rebuild"); } + /// An existing store must pick up the newest table without losing what + /// it already had. Written generically against `MIGRATIONS.len() - 1` + /// so it keeps guarding the upgrade path for whatever the latest + /// migration happens to be, rather than rotting the moment one lands + /// after it. + #[test] + fn existing_store_gains_the_latest_table_and_keeps_its_rows() { + let conn = rusqlite::Connection::open_in_memory().unwrap(); + let previous = MIGRATIONS.len() - 1; + let sql: String = MIGRATIONS[..previous].join("\n"); + conn.execute_batch(&format!( + "BEGIN;\n{sql}\nPRAGMA user_version = {previous};\nCOMMIT;" + )) + .unwrap(); + conn.execute( + "INSERT INTO node (uid, kind, body, created_at, updated_at) + VALUES ('Z', 'memory', 'a fact from before the upgrade', 1, 1)", + [], + ) + .unwrap(); + + migrate(&conn).unwrap(); + + let v: i64 = conn + .query_row("PRAGMA user_version", [], |r| r.get(0)) + .unwrap(); + assert_eq!(v, SCHEMA_VERSION); + let kept: i64 = conn + .query_row("SELECT count(*) FROM node WHERE uid = 'Z'", [], |r| { + r.get(0) + }) + .unwrap(); + assert_eq!(kept, 1, "migration must not drop existing rows"); + // The table this migration added is usable, not merely present. + crate::secrets::record_refusal( + &conn, + b"fingerprint", + "a JWT", + crate::secrets::Surface::CliRemember, + 1_700_000_000, + ); + assert_eq!( + crate::secrets::refusal_counts(&conn, 0).unwrap(), + (1, 1), + "upgraded store must be able to write the refusal ledger" + ); + } + #[test] fn refuses_newer_database() { let conn = crate::db::open_in_memory().unwrap(); diff --git a/crates/mimir-core/src/eval/mod.rs b/crates/mimir-core/src/eval/mod.rs index 9828164..716771d 100644 --- a/crates/mimir-core/src/eval/mod.rs +++ b/crates/mimir-core/src/eval/mod.rs @@ -207,12 +207,9 @@ fn evaluate_all( ids: &fixtures::FixtureIds, k: usize, model: &str, + scoring: &crate::config::ScoringConfig, mut query_vec: impl FnMut(&fixtures::Question) -> Result>, ) -> Result> { - // Score with the same weights a real user gets (config defaults), so a - // ranking-config change (recency_alpha, type_prior_alpha, ...) actually - // shows up here instead of the eval scoring silently diverging from it. - let scoring = crate::config::ScoringConfig::default(); let mut cache = None; let mut rows = Vec::new(); for q in fixtures::questions() { @@ -423,11 +420,22 @@ pub fn run_inject_eval_real_model() -> Result> { /// Hermetic baseline: synthetic vectors, no model, no network. Safe for /// normal `cargo test`. pub fn run_hermetic(k: usize) -> Result { + // The same weights a real user gets, so a ranking-config change + // (recency_alpha, type_prior_alpha, ...) actually shows up here instead + // of the eval scoring silently diverging from the product's. + run_hermetic_with(k, &crate::config::ScoringConfig::default()) +} + +/// [`run_hermetic`] with the scoring weights supplied, so an ablation can +/// ask what a knob is actually buying. Nothing in the product calls this +/// with a non-default config — if it did, the defaults would no longer be +/// what the baseline measures. +pub fn run_hermetic_with(k: usize, scoring: &crate::config::ScoringConfig) -> Result { let conn = db::open_in_memory()?; let ids = fixtures::insert_fixture_nodes(&conn)?; fixtures::seed_synthetic_vectors(&conn, &ids)?; - let rows = evaluate_all(&conn, &ids, k, fixtures::SYNTHETIC_MODEL, |q| { + let rows = evaluate_all(&conn, &ids, k, fixtures::SYNTHETIC_MODEL, scoring, |q| { Ok(fixtures::synthetic_query_vector(q.topic, q.query)) })?; Ok(build_result(k, rows)) @@ -474,7 +482,8 @@ pub fn run_real_model(k: usize) -> Result> { } let model_name = embedder.name.clone(); - let rows = evaluate_all(&conn, &ids, k, &model_name, |q| { + let scoring = crate::config::ScoringConfig::default(); + let rows = evaluate_all(&conn, &ids, k, &model_name, &scoring, |q| { Ok(embedder.embed(vec![q.query.to_string()])?.remove(0)) })?; Ok(Some(build_result(k, rows))) @@ -492,6 +501,145 @@ mod tests { /// `fixtures.rs` are deliberately hard so a future ranking change has /// headroom to prove a win. The floor here only catches "the harness /// itself is broken" (empty results, ids not resolving, etc.). + /// Landed hermetic numbers at k=5, by question set. Committed so a + /// ranking change has to show up as a deliberate edit to this table in + /// the diff, rather than as a number nobody reads drifting quietly. + /// + /// The hermetic mode is fully deterministic (synthetic vectors, fixed + /// corpus), so these reproduce exactly — which is why the gate below + /// compares in *both* directions. An improvement failing the build is + /// the point: it means someone made ranking better and should say so + /// here, where the next person can see what the number used to be. + const HERMETIC_BASELINE: &[(&str, f64, f64)] = &[ + // (set, mrr, recall) + ("core", 1.0000, 1.0000), + ("tuning", 0.9000, 0.9000), + ("holdout", 1.0000, 1.0000), + ("overall (k=5)", 0.9565, 0.9565), + ]; + + const BASELINE_TOL: f64 = 5e-4; + + #[test] + fn hermetic_baseline_reproduces_exactly() { + let result = run_hermetic(5).unwrap(); + for (label, want_mrr, want_recall) in HERMETIC_BASELINE { + let got = result + .by_set + .iter() + .find(|r| &r.label == label) + .unwrap_or_else(|| panic!("no '{label}' row in by_set")); + for (metric, want, have) in [ + ("mrr", *want_mrr, got.mrr), + ("recall", *want_recall, got.recall), + ] { + assert!( + (have - want).abs() <= BASELINE_TOL, + "{label} {metric}: baseline {want:.4}, now {have:.4}.\n\ + If this change was intended, update HERMETIC_BASELINE in this file \ + and say why in the commit — a ranking change that edits no baseline \ + is a ranking change nobody reviewed." + ); + } + } + } + + /// The ablation memsem-style: every scoring knob must pay for itself on + /// the corpus, and the stack as a whole must beat scoring with all of + /// them off. Without this, a knob can rot into a no-op — or worse, into + /// a net negative — while its config field and docs still claim a job. + #[test] + fn every_ranking_knob_earns_its_place() { + use crate::config::ScoringConfig; + let d = ScoringConfig::default(); + let mrr = |cfg: &ScoringConfig| { + run_hermetic_with(5, cfg) + .unwrap() + .by_category + .last() + .unwrap() + .mrr + }; + let base = mrr(&d); + + for (knob, cfg) in [ + ( + "strength_alpha", + ScoringConfig { + strength_alpha: 0.0, + ..d.clone() + }, + ), + ( + "recency_alpha", + ScoringConfig { + recency_alpha: 0.0, + ..d.clone() + }, + ), + ( + "type_prior_alpha", + ScoringConfig { + type_prior_alpha: 0.0, + ..d.clone() + }, + ), + ] { + let without = mrr(&cfg); + assert!( + base > without, + "{knob} buys nothing: MRR {base:.4} with it, {without:.4} without. \ + Either it stopped working or the corpus stopped testing it — \ + both need a human, neither should pass silently." + ); + } + + let all_off = mrr(&ScoringConfig { + strength_alpha: 0.0, + recency_alpha: 0.0, + type_prior_alpha: 0.0, + code_damp: 1.0, + ..d.clone() + }); + assert!( + base > all_off, + "the whole scoring stack buys nothing: {base:.4} vs {all_off:.4} unscored" + ); + } + + /// `code_damp` is deliberately absent from the ablation above, because + /// it currently changes *nothing* on this corpus — including the + /// `code-vs-memory` category that exists to guard it. That is a gap in + /// the fixtures, not proof the knob is useless: the one scenario there + /// ranks the memory first with or without the damp, so it never + /// exercises the tie the damp is meant to break. + /// + /// Pinned rather than ignored, so the day someone authors a fixture + /// that does exercise it, this test fails and forces the knob into the + /// ablation above where it belongs. + #[test] + fn code_damp_is_still_inert_on_this_corpus() { + use crate::config::ScoringConfig; + let d = ScoringConfig::default(); + let with = run_hermetic_with(5, &d).unwrap(); + let without = run_hermetic_with( + 5, + &ScoringConfig { + code_damp: 1.0, + ..d.clone() + }, + ) + .unwrap(); + let overall = |r: &EvalResult| r.by_category.last().unwrap().mrr; + assert!( + (overall(&with) - overall(&without)).abs() <= BASELINE_TOL, + "code_damp now moves the corpus ({:.4} vs {:.4}) — good. Move it into \ + every_ranking_knob_earns_its_place and delete this test.", + overall(&with), + overall(&without) + ); + } + #[test] fn hermetic_scores_are_well_formed() { let result = run_hermetic(5).unwrap(); diff --git a/crates/mimir-core/src/format.rs b/crates/mimir-core/src/format.rs index 3eaf594..d008d37 100644 --- a/crates/mimir-core/src/format.rs +++ b/crates/mimir-core/src/format.rs @@ -1,12 +1,15 @@ //! Agent-format output: one token-lean line per hit, shared by the CLI //! and the MCP server. Shape: `m:ABCDEF [gotcha pr:mimir 06-11 ↑7] title — snippet` +use std::cmp::Reverse; use std::collections::HashMap; use rusqlite::Connection; use crate::error::Result; +use crate::memory::tokens; use crate::model::{short_uid, Node}; +use crate::search::fts::is_stopword; use crate::store; /// `MM-DD` of a unix timestamp (UTC). Compact by design: recall output is @@ -49,9 +52,174 @@ pub fn truncate_chars(s: &str, max: usize) -> String { out } +/// Chars of lead-in kept before a matched term when the snippet window is +/// shifted, so the match lands in context rather than flush at the edge. +const MATCH_LEAD_CHARS: usize = 32; + +/// Shortest query token worth centring on. Below this, prefix matching is +/// noise ("id" hits "identity", "video", …). +const MIN_MATCH_TOKEN: usize = 3; + +/// Prefix length used to approximate the FTS5 porter stemmer: "splitting" +/// and "splits" both reduce to "split". Cheap and one-directional — it can +/// over-match slightly, which only costs us a differently-placed window. +const STEM_PREFIX: usize = 5; + +/// How far into a body we hunt for a window. Recall is a hot path and the +/// scan is O(body × stems): measured at 7.4 ms on the largest chunk in a +/// real store (215k chars), enough on its own to blow a single-digit-ms +/// warm recall. 20k covers all but 83 of ~19k chunks whole; past it we +/// simply head-truncate, which is what this code replaced anyway. +const MATCH_SCAN_CHARS: usize = 20_000; + +/// Content-bearing query stems, sharing `tokens`/`is_stopword` with the FTS +/// leg so the snippet can't drift from what actually matched. +fn query_stems(query: &str) -> Vec { + tokens(query) + .into_iter() + .filter(|t| t.chars().any(char::is_alphanumeric)) + .filter(|t| !is_stopword(t)) + .filter(|t| t.chars().count() >= MIN_MATCH_TOKEN) + .map(|t| t.chars().take(STEM_PREFIX).collect()) + .collect() +} + +/// Lowercased char view of `text`'s first `limit` chars, index-aligned 1:1 +/// with the original. Deliberately ASCII-only: `char::to_lowercase` can emit +/// more chars than it consumes (U+0130 'İ' becomes two), which desyncs every +/// index we then slice the original with. Non-ASCII uppercase simply fails to +/// match, which costs a differently-placed window — not a panic, and not a +/// wrong one. +fn lower_chars(text: &str, limit: usize) -> Vec { + text.chars() + .take(limit) + .map(|c| c.to_ascii_lowercase()) + .collect() +} + +/// Every char offset in `hay` where `needle` begins. +/// +/// Deliberately plain substring matching, not word-anchored. Anchoring was +/// tried and measured worse: it removes the ~3% of matches that land inside +/// an unrelated word ("use" in "because"), but also drops legitimate ones +/// inside compound identifiers, costing 2pp of memory previews for no gain +/// on docs. Density scoring already tolerates the occasional stray match. +fn occurrences(hay: &[char], needle: &[char]) -> Vec { + let Some(last) = hay.len().checked_sub(needle.len()) else { + return Vec::new(); + }; + if needle.is_empty() { + return Vec::new(); + } + (0..=last) + .filter(|&i| hay[i..].starts_with(needle)) + .collect() +} + +/// Whether any offset in the sorted list `occ` places a `needle_len`-long +/// match wholly inside `[start, end)`. Binary search, so scoring a candidate +/// costs O(log occurrences) rather than a fresh O(window × needle) rescan of +/// text we already indexed — the difference between 1.6 ms and ~60 µs on a +/// repetitive body where a stem recurs thousands of times. +fn covers(occ: &[usize], needle_len: usize, start: usize, end: usize) -> bool { + let Some(last) = end.checked_sub(needle_len) else { + return false; + }; + let i = occ.partition_point(|&at| at < start); + occ.get(i).is_some_and(|&at| at <= last) +} + +/// The `snippet_chars`-wide window over `text` covering the most *distinct* +/// query stems, ties going to the earliest window. Density beats +/// first-match: a window holding three of the query's terms tells the reader +/// far more than one holding the earliest single term, which is often an +/// incidental mention well before the passage that actually answers. +/// +/// Returns `None` when the leading window already wins, so hits that read +/// fine today are left byte-identical. +fn match_window(text: &str, stems: &[String], snippet_chars: usize) -> Option { + if stems.is_empty() || snippet_chars == 0 { + return None; + } + let lc = lower_chars(text, MATCH_SCAN_CHARS); + // Each stem indexed once as (match length, sorted offsets), then reused + // for both candidate generation and scoring. + let index: Vec<(usize, Vec)> = stems + .iter() + .map(|s| { + let needle: Vec = s.chars().collect(); + (needle.len(), occurrences(&lc, &needle)) + }) + .collect(); + + // Candidates: the head, plus a lead-in ahead of each stem occurrence. + let mut starts = vec![0usize]; + for (_, offsets) in &index { + starts.extend(offsets.iter().map(|at| at.saturating_sub(MATCH_LEAD_CHARS))); + } + // Scoring below is O(candidates × stems), so it is worth paying a sort to + // drop duplicates — nearby stems collapse to the same start. + starts.sort_unstable(); + starts.dedup(); + + let best = starts.into_iter().max_by_key(|&s| { + let end = (s + snippet_chars).min(lc.len()); + let hits = index + .iter() + .filter(|(len, offsets)| covers(offsets, *len, s, end)) + .count(); + (hits, Reverse(s)) + })?; + if best == 0 { + return None; + } + + // `lower_chars` is index-aligned, so `best` indexes the original safely. + // Collect only as far as the window plus its word-boundary lead-in can + // reach — bodies run to hundreds of thousands of chars. + let chars: Vec = text + .chars() + .take(best + MATCH_LEAD_CHARS + snippet_chars) + .collect(); + // Step forward to a word boundary so the window doesn't open mid-word. + // Bounded by the lead-in, so this can never step past the match itself. + let start = chars[best..] + .iter() + .take(MATCH_LEAD_CHARS) + .position(|c| c.is_whitespace()) + .map_or(best, |off| best + off + 1); + // Already bounded to `snippet_chars` by the take, so no truncation here. + let window: String = chars[start..].iter().take(snippet_chars).collect(); + Some(format!("…{window}")) +} + /// One-line agent format for a node. /// `project` is the display name of the node's project, if scoped. pub fn agent_line(node: &Node, project: Option<&str>, snippet_chars: usize) -> String { + agent_line_for_query(node, project, snippet_chars, None) +} + +/// [`agent_line`], but with the search query in hand so the snippet can be +/// centred on what matched instead of always starting at the body's first +/// character. Only worth passing a query where one exists — ranked recall +/// results; a `remember` echo or an anchor trigger has none. +/// +/// Why this matters for docs: a memory's title is a self-contained claim, +/// but a doc chunk's is a breadcrumb ("Doc › Section › Sub"), so the body +/// snippet is the only thing telling the reader why the hit came back. Head +/// truncation frequently spends it on the section's opening sentence. +/// +/// Deliberately left alone: the breadcrumb titles themselves, which spend +/// ~75 chars on location rather than content. Fixing that belongs in +/// `index::chunker`, where they're built — and would shift ranking, since +/// `title` carries the heaviest BM25 weight. Revisit if snippet-only proves +/// insufficient. +pub fn agent_line_for_query( + node: &Node, + project: Option<&str>, + snippet_chars: usize, + query: Option<&str>, +) -> String { let id = short_uid(node.kind, &node.uid); let tag = node.subkind.as_deref().unwrap_or(node.kind.as_str()); let scope = project.map(|p| format!(" pr:{p}")).unwrap_or_default(); @@ -62,9 +230,24 @@ pub fn agent_line(node: &Node, project: Option<&str>, snippet_chars: usize) -> S String::new() }; let title = node.title.as_deref().unwrap_or("(untitled)"); - let mut line = format!("{id} [{tag}{scope} {date}{uses}] {title}"); + // Only `unsure` rides the compact line, and it costs one word. The + // asymmetry is deliberate: this line is the one an agent reads before + // acting, so the case worth spending tokens on is the one where acting + // without checking would be a mistake. Printing "certain" everywhere + // would be reassurance nobody asked for, at a cost per recall. + let doubt = match node.confidence() { + Some(crate::model::MemoryConfidence::Unsure) => " unsure", + _ => "", + }; + let mut line = format!("{id} [{tag}{scope} {date}{uses}{doubt}] {title}"); if let Some(body) = node.body.as_deref() { let flat = collapse_ws(body); + let stems = query.map(query_stems).unwrap_or_default(); + // Centre on the match when there is one; otherwise head-truncate. + let excerpt = |text: &str| { + match_window(text, &stems, snippet_chars) + .unwrap_or_else(|| truncate_chars(text, snippet_chars)) + }; // Skip the part of the body the (possibly truncated) title covers. let covered = title.trim_end_matches('…'); let rest = flat.strip_prefix(covered).map(str::trim_start); @@ -72,11 +255,11 @@ pub fn agent_line(node: &Node, project: Option<&str>, snippet_chars: usize) -> S Some("") => {} // title covers everything Some(rest) => { line.push_str(" — "); - line.push_str(&truncate_chars(rest, snippet_chars)); + line.push_str(&excerpt(rest)); } None if flat.len() > title.len() => { line.push_str(" — "); - line.push_str(&truncate_chars(&flat, snippet_chars)); + line.push_str(&excerpt(&flat)); } None => {} } @@ -137,6 +320,31 @@ pub fn full_record( )); } } + // Only annotate memories, and only when there is something to say. A + // "grounded" badge on a doc chunk is noise (of course it is — it *is* + // the artifact), and stamping "ungrounded" on every ordinary note would + // read as a defect rather than the normal case that it is. + if node.kind == crate::model::Kind::Memory { + // The three axes a reader has to keep apart, and which a single + // `strength` number silently merges: what the author claimed, how + // much the store has used it, and whether the link it rests on + // still holds. Only printed when declared — see MemoryConfidence + // on why absent is not the middle. + if let Some(c) = node.confidence() { + out.push_str(&format!( + "\nconfidence: {c} (author-declared; not a ranking signal)" + )); + } + if let crate::grounding::Grounding::Stale { kind, label } = + crate::grounding::grounding(conn, node.id)? + { + out.push_str(&format!( + "\n! grounding stale: {} {label} is no longer indexed \ + (the note may still be right; nothing has revisited it since)", + kind.as_str() + )); + } + } Ok(out) } @@ -171,6 +379,81 @@ mod tests { ); } + /// Only doubt is worth the tokens on the line an agent acts from. + /// Certainty is the assumption already; spending a word per recall to + /// restate it would be pure cost. + #[test] + fn only_unsure_reaches_the_compact_line() { + let conn = crate::db::open_in_memory().unwrap(); + let make = |level: Option| { + let mut new = NewNode::new(Kind::Memory); + new.subkind = Some("note".into()); + new.title = Some("dns is the cause".into()); + new.body = Some("dns is the cause".into()); + let node = store::insert_node(&conn, new).unwrap(); + if let Some(l) = level { + store::set_confidence(&conn, node.id, l).unwrap(); + } + let node = store::get_node(&conn, node.id).unwrap(); + agent_line(&node, None, 120) + }; + assert!(make(Some(crate::model::MemoryConfidence::Unsure)).contains("unsure")); + assert!(!make(Some(crate::model::MemoryConfidence::Certain)).contains("certain")); + assert!(!make(Some(crate::model::MemoryConfidence::Likely)).contains("likely")); + let bare = make(None); + assert!(!bare.contains("unsure") && !bare.contains("certain")); + } + + /// Confidence must be readable in full, and must be legibly the + /// author's claim rather than something Mimir computed. + #[test] + fn full_record_attributes_confidence_to_the_author() { + let conn = crate::db::open_in_memory().unwrap(); + let mut new = NewNode::new(Kind::Memory); + new.subkind = Some("insight".into()); + new.title = Some("the flake is dns".into()); + new.body = Some("the flake is dns".into()); + let node = store::insert_node(&conn, new).unwrap(); + store::set_confidence(&conn, node.id, crate::model::MemoryConfidence::Unsure).unwrap(); + let node = store::get_node(&conn, node.id).unwrap(); + let out = full_record(&conn, &node, &HashMap::new()).unwrap(); + assert!(out.contains("confidence: unsure"), "{out}"); + assert!(out.contains("author-declared"), "{out}"); + assert!(out.contains("not a ranking signal"), "{out}"); + } + + /// The separation that makes this worth having: recall a guess as much + /// as you like and it is still marked a guess. `strength` moves, + /// `confidence` does not — one records use, the other a claim. + #[test] + fn recall_raises_strength_without_touching_confidence() { + let conn = crate::db::open_in_memory().unwrap(); + let mut new = NewNode::new(Kind::Memory); + new.subkind = Some("insight".into()); + new.title = Some("probably a cache issue".into()); + new.body = Some("probably a cache issue".into()); + let node = store::insert_node(&conn, new).unwrap(); + store::set_confidence(&conn, node.id, crate::model::MemoryConfidence::Unsure).unwrap(); + + let before = store::get_node(&conn, node.id).unwrap(); + for _ in 0..8 { + crate::learn::record_opened(&conn, node.id).unwrap(); + } + let after = store::get_node(&conn, node.id).unwrap(); + + assert!( + after.strength > before.strength, + "being used should move strength ({} -> {})", + before.strength, + after.strength + ); + assert_eq!( + after.confidence(), + Some(crate::model::MemoryConfidence::Unsure), + "being used must NOT promote a guess" + ); + } + #[test] fn agent_line_skips_redundant_snippet() { let conn = crate::db::open_in_memory().unwrap(); @@ -182,4 +465,111 @@ mod tests { let line = agent_line(&node, None, 120); assert!(!line.contains('—'), "no snippet expected: {line}"); } + + /// The defect this exists to prevent: a doc chunk whose breadcrumb title + /// says only where it lives, and whose matching text sits past the head + /// window — so plain truncation shows nothing about why it was returned. + fn breadcrumb_chunk() -> Node { + let conn = crate::db::open_in_memory().unwrap(); + let mut new = NewNode::new(Kind::Chunk); + new.title = Some("proxy › mimir proxy › What it actually does".into()); + new.body = Some(format!( + "proxy › mimir proxy › What it actually does {}the proxy adds ephemeral \ + breakpoints on the system prompt and the last message.", + "You cannot compress text and have the API bill fewer tokens. ".repeat(4) + )); + store::insert_node(&conn, new).unwrap() + } + + #[test] + fn query_aware_snippet_centres_on_the_match() { + let node = breadcrumb_chunk(); + + let blind = agent_line(&node, None, 120); + assert!( + !blind.contains("breakpoints"), + "head truncation should miss the match: {blind}" + ); + + let aware = agent_line_for_query(&node, None, 120, Some("prompt cache breakpoints")); + assert!( + aware.contains("breakpoints"), + "snippet should be centred on the match: {aware}" + ); + assert!(aware.contains("— …"), "shifted window is elided: {aware}"); + } + + #[test] + fn query_aware_snippet_falls_back_when_nothing_matches() { + let node = breadcrumb_chunk(); + assert_eq!( + agent_line_for_query(&node, None, 120, Some("kubernetes ingress")), + agent_line(&node, None, 120), + "a query with no lexical match must not change the line" + ); + } + + #[test] + fn query_aware_snippet_leaves_early_matches_alone() { + let node = breadcrumb_chunk(); + // "compress" is inside the head window, which already shows it. + assert_eq!( + agent_line_for_query(&node, None, 120, Some("compress")), + agent_line(&node, None, 120), + ); + } + + #[test] + fn query_stems_drop_stopwords_and_short_tokens() { + assert_eq!(query_stems("how is it split"), vec!["split"]); + // Stemmed to a shared prefix, so "splitting" finds "splits". + assert_eq!(query_stems("splitting"), vec!["split"]); + } + + #[test] + fn match_window_is_char_safe_on_multibyte_text() { + let text = format!("{}naïve café — chunking rules apply", "ø".repeat(200)); + let stems = query_stems("chunking"); + let win = match_window(&text, &stems, 40).expect("match past the window"); + assert!(win.contains("chunking"), "{win}"); + assert!(win.starts_with('…'), "{win}"); + } + + #[test] + fn match_window_survives_chars_that_grow_when_lowercased() { + // U+0130 lowercases to TWO chars, so a char index computed in the + // lowercased text can run past the original's char count. + let text = format!("{}chunking rules apply", "İ".repeat(200)); + let stems = query_stems("chunking"); + let win = match_window(&text, &stems, 40).expect("match past the window"); + assert!(win.contains("chunking"), "{win}"); + } + + #[test] + fn match_window_prefers_density_over_the_earliest_match() { + // "cache" appears early and alone; the passage that answers the + // query — all three terms together — sits much later. + let text = format!( + "the cache is warm. {} prompt cache breakpoints are set here.", + "filler prose that says nothing useful. ".repeat(12) + ); + let stems = query_stems("prompt cache breakpoints"); + let win = match_window(&text, &stems, 80).expect("a denser window exists"); + assert!(win.contains("breakpoints"), "{win}"); + assert!(win.contains("prompt"), "{win}"); + } + + #[test] + fn match_window_keeps_the_head_when_it_is_already_densest() { + let text = format!( + "prompt cache breakpoints explained up front. {}", + "later filler with no query terms at all. ".repeat(12) + ); + let stems = query_stems("prompt cache breakpoints"); + assert_eq!( + match_window(&text, &stems, 80), + None, + "head already wins — must not churn the line" + ); + } } diff --git a/crates/mimir-core/src/grounding.rs b/crates/mimir-core/src/grounding.rs new file mode 100644 index 0000000..a1414d6 --- /dev/null +++ b/crates/mimir-core/src/grounding.rs @@ -0,0 +1,271 @@ +//! Whether a memory is attached to something Mimir can go and check. +//! +//! Every other signal on a memory is a *policy*: strength rises because it +//! got used, a type prior favours gotchas, a human marks it useful. None of +//! those can ever be proven wrong by the code that assigns them — they are +//! opinions about a claim, computed from how people treated the claim. +//! +//! Grounding is different, and that is the whole point of it. A memory is +//! grounded when it links to an artifact Mimir indexes from disk — a code +//! symbol, a source chunk, a doc chunk, a file. That link is a falsifiable +//! assertion: the indexer and `graph build` soft-delete artifacts that stop +//! existing, so "this memory is about `retry_with_backoff`" becomes checkable +//! by looking, and stops being true the moment the symbol is deleted. +//! +//! [`Grounding::Stale`] is that check failing — the memory still claims to be +//! about something the codebase no longer has. It does **not** mean the +//! memory is wrong; a note about a function that was renamed is usually still +//! valuable. It means nobody has revisited it since the ground moved. +//! +//! Deliberately inert in ranking. Grounding is reported, never scored: it +//! changes what you can see about a memory, not which memories you get back. +//! Making it a ranking input is a real option, but it is a different decision +//! with a different blast radius, and it would need the drift-eval baseline +//! re-cut rather than smuggled in behind a display field. + +use rusqlite::Connection; + +use crate::error::Result; +use crate::model::Kind; + +/// Artifact kinds whose existence Mimir verifies against disk. A link to +/// any of these is checkable; a link to another memory or a tag is not, +/// which is why those don't ground anything. +const GROUNDING_KINDS: [Kind; 4] = [Kind::Symbol, Kind::CodeChunk, Kind::Chunk, Kind::File]; + +/// What a memory is attached to, and whether that thing still exists. +#[derive(Debug, Clone, PartialEq, Eq)] +pub enum Grounding { + /// Linked to an artifact that is still indexed. + Grounded { kind: Kind, label: String }, + /// Linked to an artifact that has since been soft-deleted — the file + /// left the collection, or the symbol went away on the last + /// `graph build`. The claim was checked and it failed. + Stale { kind: Kind, label: String }, + /// No link to anything checkable: a free-standing assertion, trusted + /// entirely because someone wrote it down. + Ungrounded, +} + +impl Grounding { + /// Short display token. Kept blunt on purpose — `stale` should read as + /// something to look at, `ungrounded` as a plain statement of fact and + /// not an accusation, since most memories are legitimately ungrounded. + pub fn label(&self) -> &'static str { + match self { + Grounding::Grounded { .. } => "grounded", + Grounding::Stale { .. } => "stale", + Grounding::Ungrounded => "ungrounded", + } + } + + pub fn is_stale(&self) -> bool { + matches!(self, Grounding::Stale { .. }) + } + + /// What it points at, for display. + pub fn target(&self) -> Option<(Kind, &str)> { + match self { + Grounding::Grounded { kind, label } | Grounding::Stale { kind, label } => { + Some((*kind, label.as_str())) + } + Grounding::Ungrounded => None, + } + } +} + +/// Resolve one memory's grounding. +/// +/// A live artifact wins over a dead one: a memory linked to both a deleted +/// symbol and a live file is grounded, not stale. Only when every artifact +/// it points at is gone does it count as stale — otherwise ordinary +/// re-chunking, which soft-deletes and re-inserts, would flap the status of +/// memories that are perfectly well attached. +pub fn grounding(conn: &Connection, node_id: i64) -> Result { + let kinds = GROUNDING_KINDS + .iter() + .map(|k| format!("'{}'", k.as_str())) + .collect::>() + .join(","); + let mut stmt = conn.prepare(&format!( + "SELECT n.kind, COALESCE(n.title, n.path, ''), n.deleted_at IS NULL AS alive + FROM edge e + JOIN node n ON n.id = CASE WHEN e.src = ?1 THEN e.dst ELSE e.src END + WHERE (e.src = ?1 OR e.dst = ?1) AND n.kind IN ({kinds}) + ORDER BY alive DESC" + ))?; + let mut rows = stmt.query([node_id])?; + let mut stale: Option = None; + while let Some(row) = rows.next()? { + let kind: String = row.get(0)?; + let label: String = row.get(1)?; + let alive: bool = row.get(2)?; + let Ok(kind) = kind.parse::() else { + continue; + }; + if alive { + return Ok(Grounding::Grounded { kind, label }); + } + stale.get_or_insert(Grounding::Stale { kind, label }); + } + Ok(stale.unwrap_or(Grounding::Ungrounded)) +} + +/// (grounded, stale, ungrounded) across every live memory. Used by `doctor` +/// to surface the one number worth acting on: how many memories describe +/// something the codebase no longer contains. +pub fn tally(conn: &Connection) -> Result<(usize, usize, usize)> { + let mut stmt = conn.prepare( + "SELECT id FROM node WHERE kind = 'memory' AND deleted_at IS NULL + AND superseded_by IS NULL", + )?; + let ids: Vec = stmt + .query_map([], |r| r.get(0))? + .collect::>()?; + let (mut grounded, mut stale, mut ungrounded) = (0, 0, 0); + for id in ids { + match grounding(conn, id)? { + Grounding::Grounded { .. } => grounded += 1, + Grounding::Stale { .. } => stale += 1, + Grounding::Ungrounded => ungrounded += 1, + } + } + Ok((grounded, stale, ungrounded)) +} + +/// Every live memory whose grounding has been falsified, newest first. +pub fn stale_memories( + conn: &Connection, + limit: usize, +) -> Result> { + let mut stmt = conn.prepare(&format!( + "SELECT {} FROM node WHERE kind = 'memory' AND deleted_at IS NULL + AND superseded_by IS NULL ORDER BY created_at DESC", + crate::store::NODE_COLS + ))?; + let nodes: Vec = stmt + .query_map([], crate::store::row_to_node)? + .collect::>()?; + let mut out = Vec::new(); + for node in nodes { + if out.len() >= limit { + break; + } + if let Grounding::Stale { kind, label } = grounding(conn, node.id)? { + out.push((node, format!("{} {label}", kind.as_str()))); + } + } + Ok(out) +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::db; + use crate::model::{NewNode, Rel}; + use crate::store; + + fn memory(conn: &Connection, body: &str) -> i64 { + let mut n = NewNode::new(Kind::Memory); + n.subkind = Some("note".into()); + n.title = Some(body.into()); + n.body = Some(body.into()); + store::insert_node(conn, n).unwrap().id + } + + fn artifact(conn: &Connection, kind: Kind, title: &str) -> i64 { + let mut n = NewNode::new(kind); + n.title = Some(title.into()); + store::insert_node(conn, n).unwrap().id + } + + #[test] + fn unlinked_memory_is_ungrounded() { + let conn = db::open_in_memory().unwrap(); + let m = memory(&conn, "the retry backoff doubles each attempt"); + assert_eq!(grounding(&conn, m).unwrap(), Grounding::Ungrounded); + } + + #[test] + fn memory_linked_to_a_live_symbol_is_grounded() { + let conn = db::open_in_memory().unwrap(); + let m = memory(&conn, "the retry backoff doubles each attempt"); + let s = artifact(&conn, Kind::Symbol, "retry_with_backoff"); + store::link(&conn, m, s, Rel::About, 1.0).unwrap(); + assert!(matches!( + grounding(&conn, m).unwrap(), + Grounding::Grounded { kind: Kind::Symbol, label } if label == "retry_with_backoff" + )); + } + + /// The falsification: nothing about the memory changed, the codebase + /// did. This is the one status that a policy-assigned trust label could + /// never produce about itself. + #[test] + fn grounding_goes_stale_when_the_symbol_disappears() { + let conn = db::open_in_memory().unwrap(); + let m = memory(&conn, "the retry backoff doubles each attempt"); + let s = artifact(&conn, Kind::Symbol, "retry_with_backoff"); + store::link(&conn, m, s, Rel::About, 1.0).unwrap(); + assert!(!grounding(&conn, m).unwrap().is_stale()); + + store::soft_delete(&conn, s).unwrap(); + let g = grounding(&conn, m).unwrap(); + assert!(g.is_stale(), "symbol is gone; the claim is falsified"); + assert_eq!(g.target(), Some((Kind::Symbol, "retry_with_backoff"))); + } + + /// Re-chunking soft-deletes and re-inserts constantly. A memory that + /// still has one live artifact must not flap to stale just because + /// another one it references was rewritten. + #[test] + fn one_live_artifact_outweighs_a_dead_one() { + let conn = db::open_in_memory().unwrap(); + let m = memory(&conn, "the indexer walks gitignore files"); + let dead = artifact(&conn, Kind::CodeChunk, "old_chunk"); + let live = artifact(&conn, Kind::File, "src/index/mod.rs"); + store::link(&conn, m, dead, Rel::About, 1.0).unwrap(); + store::link(&conn, m, live, Rel::About, 1.0).unwrap(); + store::soft_delete(&conn, dead).unwrap(); + assert!(matches!( + grounding(&conn, m).unwrap(), + Grounding::Grounded { + kind: Kind::File, + .. + } + )); + } + + /// A link to another memory is not grounding. Two assertions agreeing + /// with each other is not evidence, and letting memory→memory links + /// count would make grounding trivially self-satisfiable. + #[test] + fn linking_to_another_memory_grounds_nothing() { + let conn = db::open_in_memory().unwrap(); + let a = memory(&conn, "the retry backoff doubles each attempt"); + let b = memory(&conn, "backoff was tuned in the 0.9 release"); + store::link(&conn, a, b, Rel::Relates, 1.0).unwrap(); + assert_eq!(grounding(&conn, a).unwrap(), Grounding::Ungrounded); + } + + #[test] + fn tally_and_stale_listing_agree() { + let conn = db::open_in_memory().unwrap(); + let grounded = memory(&conn, "chunker splits on symbol boundaries"); + let s = artifact(&conn, Kind::Symbol, "chunk_source"); + store::link(&conn, grounded, s, Rel::About, 1.0).unwrap(); + + let broken = memory(&conn, "the old pruner ran on every write"); + let gone = artifact(&conn, Kind::Symbol, "prune_on_write"); + store::link(&conn, broken, gone, Rel::About, 1.0).unwrap(); + store::soft_delete(&conn, gone).unwrap(); + + memory(&conn, "prefer ripgrep over find on this box"); + + assert_eq!(tally(&conn).unwrap(), (1, 1, 1)); + let stale = stale_memories(&conn, 10).unwrap(); + assert_eq!(stale.len(), 1); + assert_eq!(stale[0].0.id, broken); + assert_eq!(stale[0].1, "symbol prune_on_write"); + } +} diff --git a/crates/mimir-core/src/import.rs b/crates/mimir-core/src/import.rs index 76f5d5e..f903ead 100644 --- a/crates/mimir-core/src/import.rs +++ b/crates/mimir-core/src/import.rs @@ -13,6 +13,12 @@ use crate::model::MemoryType; pub struct ImportStats { pub imported: usize, pub skipped_duplicates: usize, + /// Deliberately forgotten before, and not resurrected by an import. + /// Counted separately from duplicates because it means something + /// different: a duplicate is redundant, this one someone deleted on + /// purpose. Silently folding it into the duplicate count is how the + /// deletion becomes invisible. + pub skipped_forgotten: usize, } /// One parsed memory ready for insertion. @@ -48,6 +54,7 @@ fn store_all(conn: &Connection, items: Vec) -> Result { stats.imported += 1; } RememberOutcome::Duplicate(_) => stats.skipped_duplicates += 1, + RememberOutcome::Forgotten(_) => stats.skipped_forgotten += 1, } } Ok(stats) diff --git a/crates/mimir-core/src/inject.rs b/crates/mimir-core/src/inject.rs index 2d240c5..e2d304c 100644 --- a/crates/mimir-core/src/inject.rs +++ b/crates/mimir-core/src/inject.rs @@ -475,7 +475,9 @@ mod tests { ) .unwrap(); match out { - RememberOutcome::Created(node) | RememberOutcome::Duplicate(node) => node.id, + RememberOutcome::Created(node) + | RememberOutcome::Duplicate(node) + | RememberOutcome::Forgotten(node) => node.id, } } diff --git a/crates/mimir-core/src/lib.rs b/crates/mimir-core/src/lib.rs index 0281acb..1816cd9 100644 --- a/crates/mimir-core/src/lib.rs +++ b/crates/mimir-core/src/lib.rs @@ -15,6 +15,7 @@ pub mod error; #[cfg(any(test, feature = "eval"))] pub mod eval; pub mod format; +pub mod grounding; pub mod import; pub mod index; pub mod inject; diff --git a/crates/mimir-core/src/memory.rs b/crates/mimir-core/src/memory.rs index 0681917..6b1d737 100644 --- a/crates/mimir-core/src/memory.rs +++ b/crates/mimir-core/src/memory.rs @@ -13,11 +13,24 @@ use crate::store::{self, row_to_node, NODE_COLS}; /// as a near-duplicate (without --force). const NEAR_DUP_JACCARD: f64 = 0.85; +/// How many deliberate tombstones the reword pass compares against. A bound +/// so `remember` can never turn into a table scan on a store where someone +/// has forgotten a great deal; newest-first, because a recent deletion is +/// the one a re-add is most likely to be undoing. +const FORGOTTEN_SCAN_LIMIT: usize = 500; + #[derive(Debug)] pub enum RememberOutcome { Created(Node), /// Refused: the contained node is the existing near-duplicate. Duplicate(Node), + /// Refused: this text was deliberately forgotten and the tombstone is + /// still there. Re-adding it silently is how a deleted fact creeps + /// back in — an extractor re-reading the same source produces the same + /// text, and without this the only trace of the deletion is a node the + /// dedup path can't see. Takes `--force`, which is the point: coming + /// back has to be a decision someone made, not a default. + Forgotten(Node), } #[derive(Debug)] @@ -33,7 +46,20 @@ pub struct Remember { pub fn remember(conn: &Connection, args: Remember) -> Result { let hash = content_hash(&args.text); if !args.force { - if let Some(dup) = find_duplicate(conn, &args.text, &hash)? { + // Precedence matters. An exact *live* copy is an ordinary duplicate + // even when an older tombstone of the same text also exists — that + // is the shape of a fact someone deliberately brought back, and + // refusing it as "forgotten" forever would make `--force` a + // one-way door. Only when nothing live matches does the tombstone + // get to refuse. Near-duplicates rank last: a loose match against + // some other live memory must not mask an exact tombstone hit. + if let Some(dup) = find_exact(conn, &hash)? { + return Ok(RememberOutcome::Duplicate(dup)); + } + if let Some(gone) = find_forgotten(conn, &args.text, &hash)? { + return Ok(RememberOutcome::Forgotten(gone)); + } + if let Some(dup) = find_near_duplicate(conn, &args.text)? { return Ok(RememberOutcome::Duplicate(dup)); } } @@ -113,14 +139,34 @@ pub fn parse_expires_in(spec: &str, now: i64) -> Option { n.checked_mul(secs)?.checked_add(now) } -fn find_duplicate(conn: &Connection, text: &str, hash: &[u8]) -> Result> { - // Exact (normalized) content match, any scope. +/// SQL for tombstones that represent a *deliberate* deletion. +/// +/// `consolidate`'s decay pass also soft-deletes (with `meta.archived = 1`), +/// but nobody decided that — it fell below a strength threshold while idle. +/// Refusing to re-learn a fact because a background job archived it six +/// months ago would be wrong, and would teach people to pass `--force` by +/// reflex, which is exactly how the guard stops meaning anything. +const DELIBERATE_TOMBSTONE: &str = "kind = 'memory' AND deleted_at IS NOT NULL \ + AND COALESCE(json_extract(meta, '$.archived'), 0) != 1"; + +/// A memory someone deliberately forgot, whose text is being offered again. +/// +/// Two passes, because re-extraction takes two shapes. The exact +/// `content_hash` catches a source re-read unchanged. The token-overlap pass +/// catches a reword — a different extractor, or the same fact said again in +/// other words — which the hash cannot, since `tokens` keeps `file.rs` and +/// trailing punctuation whole and so treats "utc" and "utc." as different. +/// +/// The scan is over deliberate tombstones only, which stay rare in a real +/// store (people forget on purpose seldom, and decay archival is excluded), +/// so this is a few dozen short strings, not a table sweep. +fn find_forgotten(conn: &Connection, text: &str, hash: &[u8]) -> Result> { let exact = conn .query_row( &format!( "SELECT {NODE_COLS} FROM node - WHERE kind = 'memory' AND content_hash = ?1 AND deleted_at IS NULL - LIMIT 1" + WHERE {DELIBERATE_TOMBSTONE} AND content_hash = ?1 + ORDER BY deleted_at DESC LIMIT 1" ), [hash], row_to_node, @@ -129,7 +175,45 @@ fn find_duplicate(conn: &Connection, text: &str, hash: &[u8]) -> Result= NEAR_DUP_JACCARD { + return Ok(Some(node)); + } + } + Ok(None) +} + +/// Exact (normalized) content match against a *live* memory, any scope. +fn find_exact(conn: &Connection, hash: &[u8]) -> Result> { + Ok(conn + .query_row( + &format!( + "SELECT {NODE_COLS} FROM node + WHERE kind = 'memory' AND content_hash = ?1 AND deleted_at IS NULL + LIMIT 1" + ), + [hash], + row_to_node, + ) + .optional()?) +} + +/// Near-duplicate: high token overlap with the best FTS matches. +fn find_near_duplicate(conn: &Connection, text: &str) -> Result> { let query = SearchQuery { text: text.to_string(), scope: Scope::All, @@ -334,6 +418,134 @@ mod tests { } } + /// The failure this closes: `forget` tombstones a memory, recall stops + /// returning it, and then the same text is offered again — by an + /// importer, an extractor re-reading an unchanged source, or an agent + /// that saw the fact a second time. Dedup only ever looked at live + /// nodes, so the deletion left nothing that could catch the re-add and + /// the fact came back silently under a fresh id. + #[test] + fn forgotten_memory_is_not_silently_re_added() { + let conn = db::open_in_memory().unwrap(); + let text = "staging db password rotation runs from the ops cron at 03:00 UTC"; + let first = match remember_text(&conn, text) { + RememberOutcome::Created(n) => n, + other => panic!("expected Created, got {other:?}"), + }; + store::soft_delete(&conn, first.id).unwrap(); + + // Same fact, reformatted the way a different extractor would emit it. + match remember_text( + &conn, + "Staging DB password rotation runs from the OPS cron at 03:00 utc", + ) { + RememberOutcome::Forgotten(gone) => { + assert_eq!(gone.id, first.id); + assert!(gone.deleted_at.is_some(), "must carry the tombstone date"); + } + other => panic!("expected Forgotten, got {other:?}"), + } + + // Refusing is not the same as making it impossible: a human who + // means it can still bring it back, and gets a new live node. + let forced = remember( + &conn, + Remember { + text: text.into(), + mtype: MemoryType::Note, + tags: vec![], + project_id: None, + force: true, + }, + ) + .unwrap(); + let RememberOutcome::Created(revived) = forced else { + panic!("--force must override the tombstone"); + }; + assert_ne!(revived.id, first.id); + assert!(revived.deleted_at.is_none()); + } + + /// The hash pass alone would miss this: `tokens` keeps trailing + /// punctuation attached (so `file.rs` survives), which makes "utc" and + /// "utc." different tokens and the two texts different hashes. A + /// re-extraction that rewords even slightly is the common case, so the + /// guard has to survive it. + #[test] + fn forgotten_memory_is_not_re_added_under_a_reword() { + let conn = db::open_in_memory().unwrap(); + let first = match remember_text( + &conn, + "The staging database password rotation is handled by the ops cron at 03:00 UTC", + ) { + RememberOutcome::Created(n) => n, + other => panic!("expected Created, got {other:?}"), + }; + store::soft_delete(&conn, first.id).unwrap(); + match remember_text( + &conn, + "the STAGING database password rotation is handled by the ops cron at 03:00 utc.", + ) { + RememberOutcome::Forgotten(gone) => assert_eq!(gone.id, first.id), + other => panic!("expected Forgotten, got {other:?}"), + } + } + + /// Decay archival is not a decision anyone made. Treating it as one + /// would refuse re-learning facts a background job retired while idle, + /// and train everyone to pass `--force` by reflex — at which point the + /// guard on real deletions stops meaning anything. + #[test] + fn auto_archived_memory_does_not_block_re_adding() { + let conn = db::open_in_memory().unwrap(); + let text = "cargo nextest needs a separate install on the CI image"; + let first = match remember_text(&conn, text) { + RememberOutcome::Created(n) => n, + other => panic!("expected Created, got {other:?}"), + }; + // Exactly what consolidate's decay pass does. + conn.execute( + "UPDATE node SET deleted_at = 1, meta = json_set(meta, '$.archived', 1) WHERE id = ?1", + [first.id], + ) + .unwrap(); + assert!( + matches!(remember_text(&conn, text), RememberOutcome::Created(_)), + "decay archival must not masquerade as a deliberate forget" + ); + } + + /// A tombstone must not outrank a live memory: once the fact is back, + /// remembering it again is an ordinary duplicate, not a resurrection. + #[test] + fn live_duplicate_wins_over_an_older_tombstone() { + let conn = db::open_in_memory().unwrap(); + let text = "the proxy strips cache breakpoints above four"; + let first = match remember_text(&conn, text) { + RememberOutcome::Created(n) => n, + other => panic!("expected Created, got {other:?}"), + }; + store::soft_delete(&conn, first.id).unwrap(); + let revived = remember( + &conn, + Remember { + text: text.into(), + mtype: MemoryType::Note, + tags: vec![], + project_id: None, + force: true, + }, + ) + .unwrap(); + let RememberOutcome::Created(revived) = revived else { + panic!("expected Created"); + }; + match remember_text(&conn, text) { + RememberOutcome::Duplicate(d) => assert_eq!(d.id, revived.id), + other => panic!("expected Duplicate, got {other:?}"), + } + } + #[test] fn near_duplicate_refused_force_overrides() { let conn = db::open_in_memory().unwrap(); diff --git a/crates/mimir-core/src/model.rs b/crates/mimir-core/src/model.rs index a3a4d07..b78bc77 100644 --- a/crates/mimir-core/src/model.rs +++ b/crates/mimir-core/src/model.rs @@ -66,6 +66,34 @@ str_enum!(MemoryType { Summary => "summary", }); +// How sure the *author* was when they wrote it — never how useful it has +// since proved. +// +// Mimir already tracks how often a memory gets used (`strength`, which +// decays) and whether it has been retired (`superseded_by`, `expires_at`, +// and `grounding`). What it had no way to say is "I was guessing." Those +// are different questions, and folding them into one number is how a guess +// becomes canon: recall it enough and it outranks things that were checked, +// with nothing left to show it was ever uncertain. +// +// Author-declared at capture, deliberately — the same argument +// `expires_at` and `resolves_when` are built on. The person writing the +// memory is the only one who knows how sure they were, they know it at +// write time, and nobody ever backfills. The alternative, a confidence +// score inferred later by a model, is precisely the unfalsifiable policy +// label this is meant to replace: it would be computed from the text, so +// it could never contradict the text. +// +// Absent is a real state and is NOT the same as `Likely`. Defaulting an +// undeclared memory to the middle would put words in the author's mouth, +// the same "reject, don't mangle" call `parse_expires_in` and +// `sanitize_fires_when` already make. +str_enum!(MemoryConfidence { + Certain => "certain", + Likely => "likely", + Unsure => "unsure", +}); + str_enum!(Rel { Links => "links", Mentions => "mentions", @@ -149,6 +177,18 @@ impl Node { self.meta.get("resolves_when")?.as_str() } + /// What the author said about their own certainty, if they said + /// anything. `None` means undeclared, which is the common case and is + /// not a synonym for medium confidence. + /// + /// Read-only as far as ranking is concerned: nothing in `search` or + /// `learn` consults this. It exists so a reader — human or agent — can + /// tell a checked fact from a guess that has merely been recalled a + /// lot, which `strength` alone can never distinguish. + pub fn confidence(&self) -> Option { + self.meta.get("confidence")?.as_str()?.parse().ok() + } + /// True once a declared `expires_at` has passed. /// /// This is a *gate*, unlike strength decay — see `learn::effective_strength`, diff --git a/crates/mimir-core/src/secrets.rs b/crates/mimir-core/src/secrets.rs index 81a1462..ecf5dc4 100644 --- a/crates/mimir-core/src/secrets.rs +++ b/crates/mimir-core/src/secrets.rs @@ -74,10 +74,215 @@ static PATTERNS: Lazy> = Lazy::new(|| { ] }); +/// Which capture path a refusal came from, for the ledger. Not free text, +/// so a surface can't be silently renamed out of existing rows. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum Surface { + CliRemember, + CliEdit, + McpRemember, +} + +impl Surface { + pub fn label(self) -> &'static str { + match self { + Surface::CliRemember => "cli:remember", + Surface::CliEdit => "cli:edit", + Surface::McpRemember => "mcp:remember", + } + } +} + +/// Fingerprint of everything [`scan_capture`] looked at, so a secret +/// smuggled through tags is recognized on the next offer just as well as +/// one in the body. Same normalization as `memory::content_hash`, so a +/// reformat of the same value still collides. +pub fn capture_hash(text: &str, tags: &[String], fires_when: &[String]) -> Vec { + crate::memory::content_hash(&format!( + "{text}\u{1f}{}\u{1f}{}", + tags.join(" "), + fires_when.join(" ") + )) +} + +/// One row of the refusal ledger. +#[derive(Debug, Clone)] +pub struct Refusal { + pub kind: String, + pub surface: String, + pub first_seen: i64, + pub last_seen: i64, + pub count: i64, +} + +/// Record that a capture was turned away, keeping a fingerprint of the +/// text and never the text. +/// +/// The hash is the same normalized blake3 `memory::content_hash` uses, so +/// the ledger recognizes the identical value offered again through a +/// different surface, or after a reformat. What it deliberately cannot do +/// is give the value back: there is no path from these rows to the secret, +/// which is the entire reason the ledger is safe to keep at all. A +/// rejection record that stored what it rejected would just be a second +/// copy of the leak, in a table nobody thinks to audit. +/// +/// Best-effort by design: a failure to write the ledger must never turn a +/// successful refusal into an error, because the refusal is the part that +/// protects the user and the bookkeeping is not. +pub fn record_refusal( + conn: &rusqlite::Connection, + text_hash: &[u8], + kind: &str, + surface: Surface, + now: i64, +) { + let res = conn.execute( + "INSERT INTO refusal (hash, kind, surface, first_seen, last_seen, count) + VALUES (?1, ?2, ?3, ?4, ?4, 1) + ON CONFLICT(hash) DO UPDATE SET + last_seen = ?4, + count = count + 1, + kind = ?2, + surface = ?3", + rusqlite::params![text_hash, kind, surface.label(), now], + ); + if let Err(err) = res { + tracing::warn!(%err, "could not record refusal in the ledger"); + } +} + +/// Ledger rows, most recently offered first. +pub fn refusals(conn: &rusqlite::Connection, limit: usize) -> crate::error::Result> { + let mut stmt = conn.prepare( + "SELECT kind, surface, first_seen, last_seen, count FROM refusal + ORDER BY last_seen DESC LIMIT ?1", + )?; + let rows = stmt.query_map([limit as i64], |r| { + Ok(Refusal { + kind: r.get(0)?, + surface: r.get(1)?, + first_seen: r.get(2)?, + last_seen: r.get(3)?, + count: r.get(4)?, + }) + })?; + Ok(rows.collect::>>()?) +} + +/// (distinct values refused, total offers) since `since`. The two differ +/// exactly when something is being retried, which is the signal worth +/// surfacing: one secret offered forty times is an agent in a loop, not +/// forty careless captures. +pub fn refusal_counts(conn: &rusqlite::Connection, since: i64) -> crate::error::Result<(i64, i64)> { + Ok(conn.query_row( + "SELECT count(*), COALESCE(sum(count), 0) FROM refusal WHERE last_seen >= ?1", + [since], + |r| Ok((r.get(0)?, r.get(1)?)), + )?) +} + #[cfg(test)] mod tests { use super::*; + /// The property the whole ledger exists for: a record that something + /// was refused must not be a second copy of the thing refused. If this + /// ever fails, the ledger is a liability rather than an audit trail — + /// a table of plaintext secrets that nobody thinks to look in. + #[test] + fn ledger_never_contains_the_refused_value() { + let conn = crate::db::open_in_memory().unwrap(); + let secret = "AKIAIOSFODNN7EXAMPLE"; + let text = format!("deploy key is {secret}"); + let kind = scan(&text).expect("fixture must look like a secret"); + record_refusal( + &conn, + &capture_hash(&text, &[], &[]), + kind, + Surface::CliRemember, + 1_700_000_000, + ); + + // Sweep every text value in every column of every row. + let mut found = Vec::new(); + let tables: Vec = { + let mut s = conn + .prepare("SELECT name FROM sqlite_master WHERE type = 'table'") + .unwrap(); + let v = s + .query_map([], |r| r.get::<_, String>(0)) + .unwrap() + .collect::>>() + .unwrap(); + v + }; + for t in tables { + let mut stmt = match conn.prepare(&format!("SELECT * FROM \"{t}\"")) { + Ok(s) => s, + Err(_) => continue, // virtual/shadow tables we can't SELECT * + }; + let cols = stmt.column_count(); + let mut rows = stmt.query([]).unwrap(); + while let Some(row) = rows.next().unwrap() { + for i in 0..cols { + if let Ok(v) = row.get::<_, String>(i) { + if v.contains(secret) { + found.push(format!("{t}[{i}]")); + } + } + } + } + } + assert!( + found.is_empty(), + "refused secret leaked into the store at {found:?}" + ); + + // ...and the row is still genuinely there. + let (distinct, offers) = refusal_counts(&conn, 0).unwrap(); + assert_eq!((distinct, offers), (1, 1)); + assert_eq!(refusals(&conn, 10).unwrap()[0].kind, "an AWS access key"); + } + + /// A looping agent must show up as one row with a high count, not as a + /// flood of rows — that difference is the entire diagnostic value. + #[test] + fn repeat_offers_increment_rather_than_insert() { + let conn = crate::db::open_in_memory().unwrap(); + let text = "token ghp_000000000000000000000000000000000000"; + let kind = scan(text).expect("fixture must look like a secret"); + let hash = capture_hash(text, &[], &[]); + for i in 0..5 { + record_refusal(&conn, &hash, kind, Surface::McpRemember, 1_700_000_000 + i); + } + let (distinct, offers) = refusal_counts(&conn, 0).unwrap(); + assert_eq!((distinct, offers), (1, 5), "one value, five attempts"); + let row = &refusals(&conn, 10).unwrap()[0]; + assert_eq!(row.first_seen, 1_700_000_000); + assert_eq!(row.last_seen, 1_700_000_004); + } + + /// The same value reformatted, or moved from the body into a tag, is + /// still the same value — otherwise a retry loop looks like fresh + /// offers and the count stops meaning anything. + #[test] + fn fingerprint_survives_reformatting_and_tags() { + let secret = "AKIAIOSFODNN7EXAMPLE"; + let a = capture_hash(&format!("deploy key is {secret}"), &[], &[]); + let b = capture_hash(&format!("Deploy key is {secret}"), &[], &[]); + assert_eq!( + a, b, + "whitespace and case must not create a new fingerprint" + ); + + let in_tag = capture_hash("deploy notes", &[secret.to_string()], &[]); + let in_body = capture_hash("deploy notes", &[], &[]); + assert_ne!( + in_tag, in_body, + "a secret in tags must be fingerprinted too" + ); + } + #[test] fn detects_private_key_block() { let text = diff --git a/crates/mimir-core/src/store.rs b/crates/mimir-core/src/store.rs index 3b523a7..f4b6eda 100644 --- a/crates/mimir-core/src/store.rs +++ b/crates/mimir-core/src/store.rs @@ -409,6 +409,23 @@ pub fn set_expiry( Ok(()) } +/// Record the author's own certainty. Stored in `meta` for the same reason +/// as `set_expiry`: it replicates without a schema bump. +/// +/// No gate and no score — see [`crate::model::MemoryConfidence`]. Writing +/// this changes what a reader sees, never what recall returns. +pub fn set_confidence( + conn: &Connection, + id: i64, + confidence: crate::model::MemoryConfidence, +) -> Result<()> { + conn.execute( + "UPDATE node SET meta = json_set(meta, '$.confidence', ?2) WHERE id = ?1", + params![id, confidence.as_str()], + )?; + Ok(()) +} + /// Bind an existing keyed (possibly synced-shadow) project to this machine's /// local path + display name, clearing the shadow marker. Used when a project /// that first arrived via sync is opened locally.