Skip to content

feat(memory): add Klide-owned recall engine foundation - #68

Open
pierreprudh wants to merge 3 commits into
mainfrom
codex/memory-engine-foundation
Open

feat(memory): add Klide-owned recall engine foundation#68
pierreprudh wants to merge 3 commits into
mainfrom
codex/memory-engine-foundation

Conversation

@pierreprudh

@pierreprudh pierreprudh commented Aug 31, 2026

Copy link
Copy Markdown
Owner

Why

Klide already captures Runs, Transcripts, handoffs, and Skills, but agents had no native way to recall reviewed knowledge during a later Run. This PR makes Klide the owner of that learning layer. It does not add a third-party memory provider: Markdown remains the local source of truth, Rust owns policy and retrieval, and the future MCP surface is only an adapter for Delegate CLIs.

What ships in this foundation

  • Versioned Project Memory schema v1 with explicit knowledge kinds, review state, tags, typed source references, and supersession.
  • Backward-compatible reads: legacy Markdown loads as a reviewed handoff; entries declaring a future schema are not misinterpreted as v1.
  • Deterministic, offline, Workspace-scoped ranking. Every query term must match the same entry; normal recall excludes stale and superseded evidence.
  • Native memory_search and memory_read Harness Tools in Plan and Goal modes.
  • A distinct ReadProjectMemory / read_project_memory capability so Transcript evidence never mislabels recall as a Workspace-file or Conversation-history read.
  • Provenance in Tool result metadata, including source Runs, Transcript regions, commits, or Workspace-relative files and lines.
  • Draft generation now carries its source Run into the durable review flow.
  • Versioned Draft 2020-12 JSON Schemas and detailed architecture/schema documentation.

Architecture

Klide Memory Engine architecture

The key boundary is ownership: Kit and Missions call the Rust domain directly. A later embedded MCP server will expose these same operations to Delegates; it will not own storage, ranking, review state, or authorization.

Review lifecycle

Klide Project Memory lifecycle

A model can propose learning, but a draft is not durable Project Memory. Crossing memory_write is the acceptance boundary and Rust stamps the result reviewed. Normal recall hides stale and superseded entries while retaining them as inspectable evidence.

Schema changes

Field Contract
schemaVersion Currently 1; future versions are never parsed as v1
kind run, handoff, decision, convention, fact, failure, or pattern
reviewState proposed, reviewed, superseded, or stale
tags Small retrieval hints
sourceRefs Typed links to Runs, Transcripts, commits, or files/lines
supersedes Stable id of replaced knowledge, or null

Machine-readable contracts:

Retrieval contract

memory_search accepts a query, optional kind filters, a 1–20 result cap, and an explicit inactive-memory flag. Results contain a stable id, deterministic score, matched fields, bounded excerpt, parsed entry, and provenance. memory_read resolves that id through the Workspace boundary and returns authoritative Markdown plus structured metadata.

This PR intentionally does not auto-inject memory into prompts. Recall is an explicit Tool call, so its trigger, match reason, content, and evidence remain inspectable while the ranking and UX are proven.

Security and trust properties

  • No network call, external account, or remote memory service.
  • Memory ids are plain path components and reads remain under .klide/memory/.
  • Rust, not renderer input, assigns durable review state.
  • The local Markdown store is authoritative; any future FTS or embedding index is rebuildable.
  • The future MCP adapter must strip absolute paths and bind every request to the Workspace selected by Klide.

Validation

  • cargo test --manifest-path src-tauri/Cargo.toml --no-fail-fast615 passed
  • npm test62 files / 810 tests passed
  • npm run build — production TypeScript/Vite build passed
  • Draft 2020-12 meta-schema checks and example validation passed for both Tool calls
  • xmllint --noout passed for both SVG assets
  • git diff --check passed

Follow-up slices

  1. Typed recall/draft/publication events plus Context and Memory review UX.
  2. Embedded klide mcp memory stdio adapter over the same Rust engine, with Delegate registration and conformance tests.
  3. Local duplicate/staleness consolidation and reviewed Skill-draft learning.

Full rationale and sequence: MEMORY_ENGINE.md.

Review hardening (second commit)

A full review of the foundation surfaced ten correctness/trust findings; all are fixed in fix(memory): close the recall trust gaps from review, each behind a test.

flowchart LR
  subgraph runs ["Any Run"]
    K["Kit run<br/>(main checkout)"]
    W["Isolated run<br/>(linked worktree)"]
  end
  W -- ".git file → hop to<br/>the main checkout" --> STORE
  K --> STORE
  subgraph STORE [".klide/memory — one durable store"]
    R["reviewed"]
    X["proposed · stale · superseded<br/>unknown values fail closed"]
  end
  MW["memory_write"] -- "validates supersedes id ·<br/>demotes target in place ·<br/>sanitizes frontmatter" --> STORE
  R --> T["memory_search / memory_read"]
  X -. "includeInactive only,<br/>marked historical evidence" .-> T
  T -- "metadata without<br/>absolute paths" --> TR["durable Transcript"]
Loading
# Finding Fix
1 Worktree-isolated runs saw an empty store (.klide/memory is git-ignored, never copied) every memory command/tool hops through the worktree's .git file to the main checkout
2 Supersession recorded but never enforced — recall returned both the stale decision and its correction memory_write demotes the target's reviewState in place; search also honors supersedes pointers
3 Frontmatter injection could forge the Rust-stamped reviewState supersedes held to the id charset; all free-string values stripped of control characters
4 ASCII-only search silently missed French queries/content Unicode tokenizer + full to_lowercase folding
5 Absolute local paths stamped into the durable Transcript tool metadata drops path; relPath identifies entries
6 Unknown reviewState/kind failed open to reviewed unknown enum values fail closed to stale
7 Non-integer schemaVersion bypassed the future-schema guard; future entries silently vanished unparseable versions count as future; future entries surface as inert stale stubs
8 memory_read had no review-state gate inactive entries require includeInactive and return behind a [historical evidence] marker
9 Pre-PR localStorage drafts lacked the new required fields drafts are backfilled on load
10 Read-only tools created .klide/memory as a side effect (Plan mode must not write) read paths resolve-never-create; only memory_write creates

Gates: cargo test 620 ✓ · vitest 810 ✓ · npx tsc --noEmit

pierreprudh and others added 2 commits August 31, 2026 18:24
Make Klide the owner of durable Project Memory instead of introducing another memory provider. Version reviewed entries with knowledge kinds, typed source references, review state, tags, and supersession while preserving legacy Markdown compatibility.

Add deterministic workspace-local search and safe entry reads as native memory_search and memory_read Harness tools. Give recall its own ReadProjectMemory capability so Transcript evidence preserves the knowledge boundary and returned provenance.

Publish Draft 2020-12 entry/tool schemas, architecture and lifecycle documentation, an incremental MCP adapter roadmap, and two accessible website-ready SVG diagrams.

Validated with 615 Rust tests, 810 frontend tests, a production build, JSON Schema example validation, XML validation, and diff whitespace checks.
- Worktree runs hop to the main checkout: .klide/memory is git-ignored and
  never copied into a linked worktree, so isolated Race/Task/Mission runs
  recalled an empty store. All memory commands and tools now resolve the
  store through the worktree's .git file back to the project that owns it.
- Supersession is enforced, not just recorded: memory_write demotes the
  target entry's reviewState in place (surgical frontmatter edit, no
  re-render), and search also honors supersedes pointers from any entry.
- Frontmatter injection closed: supersedes must be a valid memory id, and
  every free-string frontmatter value is stripped of control characters so
  renderer/model input can't forge the Rust-stamped reviewState.
- Search is Unicode-aware: accented characters are term characters and fold
  with full to_lowercase, so French queries and content match.
- Tool metadata no longer stamps absolute local paths into the durable
  Transcript; relPath identifies entries.
- Unknown reviewState/kind values fail closed to stale; a non-integer
  schemaVersion counts as future; future entries surface as inert stale
  stubs instead of being silently skipped.
- memory_read gates on review state: inactive entries require
  includeInactive and return behind a historical-evidence marker.
- Read paths never create .klide/memory (Plan mode stays read-only);
  only memory_write does.
- Legacy localStorage drafts are backfilled with the new MemoryInput
  fields on load.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@pierreprudh
pierreprudh force-pushed the codex/memory-engine-foundation branch from b5e4b0c to 0d65be4 Compare September 1, 2026 09:46
The start-stage foot bar carried the same cycling policy note as a live
conversation, but before a run exists there is nothing for a standing note
to report on — it just crowded the one line of intent. Move it into the +
menu as rows (shown only when the run will actually be a Goal), so the bar
reads "+ · provider · model" and the note reappears in AiPanel's foot bar
once the first message lands.

GoalPolicyChoice gains a description clause so the rows and the note share
one text source.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant