From b0abe31dcd72238ed601cfcb4817accedd019fe2 Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 31 Aug 2026 02:03:09 +0000 Subject: [PATCH] =?UTF-8?q?chore(v3.1):=20release=20prep=20=E2=80=94=20ver?= =?UTF-8?q?sion=203.1.0,=20changelog,=20working=20release=20archives?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Phase 58 launch prep. The version had been stuck at 2.7.0 through the whole v3.0 and v3.1 line, and the release it would have cut was missing the CLI its own quickstart depends on. Version - workspace 2.7.0 -> 3.1.0 - memory-adapters pinned its own 2.7.0; now uses version.workspace so there is one source of truth Release archives actually usable - ship all four binaries (memory-daemon, memory-ingest, memory, memory-installer). memory and memory-installer were never in the archive, so a user on prebuilt binaries could not run the README quickstart at all - a missing binary now fails the release job instead of producing a silently partial archive - assets renamed agent-memory--: the archive stopped being just the daemon - docs/setup/quickstart.md rewritten to match: the old commands fetched per-binary tarballs that do not exist, with no version in the name, and untarred into ~/.local/bin where the tarball's own directory prefix would have broken the chmod that followed Documentation - CHANGELOG.md with an honest v3.1.0 entry, retractions included - docs/UPGRADING.md v3.1 section: the breaking OpenCode removal, rebuild-toc now exiting non-zero, and the index-directory behaviour change - docs/launch/ drafts: Substack post and Show HN / r/rust / r/LocalLLaMA copy Correction to a claim this milestone made - README and the draft upgrade notes said a rebuild would backfill event text into BM25. It will not. `admin rebuild-bm25` is a prune -- it removes documents below --min-level (default `day`, which would delete segment and grip docs) and re-indexes nothing; `rebuild-indexes` covers TOC nodes and grips, not events; clearing the index leaves the RocksDB checkpoint intact. There is no backfill path, so both documents now say so, and the command's help text and banner no longer call a prune a rebuild. Local: fmt, clippy -D warnings, 1205 workspace tests, 60 e2e tests, and doc all green on a clean target dir. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_014nSJ2c7F9mpmRCxfTAi214 --- .github/workflows/release.yml | 24 ++-- .planning/MILESTONES.md | 62 +++++++++ .planning/ROADMAP.md | 17 ++- .planning/STATE.md | 34 +++-- CHANGELOG.md | 100 ++++++++++++++ Cargo.toml | 2 +- README.md | 7 +- crates/memory-adapters/Cargo.toml | 2 +- crates/memory-daemon/src/cli.rs | 3 +- crates/memory-daemon/src/commands.rs | 8 +- docs/UPGRADING.md | 75 ++++++++++- docs/launch/launch-copy.md | 163 ++++++++++++++++++++++ docs/launch/v3.1-blog-post.md | 195 +++++++++++++++++++++++++++ docs/setup/quickstart.md | 22 ++- 14 files changed, 675 insertions(+), 39 deletions(-) create mode 100644 CHANGELOG.md create mode 100644 docs/launch/launch-copy.md create mode 100644 docs/launch/v3.1-blog-post.md diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index f14356a..e3e31c1 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -124,17 +124,25 @@ jobs: shell: bash run: | VERSION=${{ steps.version.outputs.version }} - ARCHIVE_DIR="memory-daemon-${VERSION}-${{ matrix.name }}" + ARCHIVE_DIR="agent-memory-${VERSION}-${{ matrix.name }}" mkdir -p "dist/${ARCHIVE_DIR}" - # Copy binaries + # Copy binaries. All four are required: the README quickstart uses + # `memory` for search and `memory-installer` for hook setup, so an + # archive missing either is a broken release, not a partial one. if [[ "${{ runner.os }}" == "Windows" ]]; then - cp target/${{ matrix.target }}/release/memory-daemon.exe "dist/${ARCHIVE_DIR}/" - cp target/${{ matrix.target }}/release/memory-ingest.exe "dist/${ARCHIVE_DIR}/" || true + EXT=".exe" else - cp target/${{ matrix.target }}/release/memory-daemon "dist/${ARCHIVE_DIR}/" - cp target/${{ matrix.target }}/release/memory-ingest "dist/${ARCHIVE_DIR}/" || true + EXT="" fi + for bin in memory-daemon memory-ingest memory memory-installer; do + src="target/${{ matrix.target }}/release/${bin}${EXT}" + if [[ ! -f "$src" ]]; then + echo "::error::release archive would be missing ${bin}${EXT} for ${{ matrix.name }}" + exit 1 + fi + cp "$src" "dist/${ARCHIVE_DIR}/" + done # Copy documentation cp LICENSE "dist/${ARCHIVE_DIR}/" || true @@ -145,7 +153,7 @@ jobs: shell: bash run: | VERSION=${{ steps.version.outputs.version }} - ARCHIVE_DIR="memory-daemon-${VERSION}-${{ matrix.name }}" + ARCHIVE_DIR="agent-memory-${VERSION}-${{ matrix.name }}" cd dist tar -czvf "${ARCHIVE_DIR}.tar.gz" "${ARCHIVE_DIR}" rm -rf "${ARCHIVE_DIR}" @@ -155,7 +163,7 @@ jobs: shell: pwsh run: | $VERSION = "${{ steps.version.outputs.version }}" - $ARCHIVE_DIR = "memory-daemon-${VERSION}-${{ matrix.name }}" + $ARCHIVE_DIR = "agent-memory-${VERSION}-${{ matrix.name }}" cd dist Compress-Archive -Path $ARCHIVE_DIR -DestinationPath "${ARCHIVE_DIR}.zip" Remove-Item -Recurse -Force $ARCHIVE_DIR diff --git a/.planning/MILESTONES.md b/.planning/MILESTONES.md index 25a5edb..5b845b1 100644 --- a/.planning/MILESTONES.md +++ b/.planning/MILESTONES.md @@ -1,5 +1,67 @@ # Project Milestones: Agent Memory +## v3.1 Make It True (Shipped: 2026-08-31) + +**Delivered:** no new capabilities. Four phases closing the gap between what the +project claimed and what it did, after a v3.0 verification document self-graded +17/17 on a benchmark that had never been run and a crate no shipped binary could +reach. + +**Phases completed:** 54, 54.5, 55, 56, 57 (5 phases, 14 plans + a cleanup phase) + +**Key accomplishments:** + +- `memory-orchestrator` made reachable: `RouteQuery` calls it, `memory search` + is a client of that RPC (`cargo tree -i` now shows a shipped dependent) +- Hybrid retrieval actually fuses BM25 and vector; BM25 outbox drain no longer a + no-op; event documents store text so previews and LLM reranking work +- Explainability reports what actually ran — a failed LLM rerank says + `rerank=heuristic`, `layers_attempted` lists only invoked layers, client stop + conditions reach the orchestrator instead of being echoed and ignored +- Performance measurement split setup from query: the retired "64.6s TOC + navigation" figure was ingest-time rollup; warm query p50 is 0.13 ms. + Percentiles withheld below 10 (p90) / 30 (p99) samples +- Honest benchmark harness: the substring metric is named `context_hit_rate`, + not "LOCOMO"; committed results are labelled mock-backend / mock-judge, and no + comparative accuracy claim ships anywhere +- Root README, LICENSE, positioning writeup, CHANGELOG — the repo had none +- Supported-surface tiering: Tier 1 (Claude Code, Codex) gates PRs, Tier 2 + (Gemini, Copilot) runs weekly. OpenCode stub deleted rather than shipped +- Release archives now contain all four binaries; previously the CLI the + quickstart depends on was not shipped at all + +**Process changes (bound to this milestone):** + +- Execution-evidence rule: run-dependent requirements must cite a committed + artifact produced by running them +- Reachability rule: a new crate is not complete until `cargo tree -i` shows a + dependent binary, or it is explicitly declared dormant +- `human_verification` items gating the milestone goal are blockers by default +- The v3.0 retrospective was written: *the code is strong; the claims were not* + +**Defects found by executing documentation rather than reading it:** + +- First-run daemon created no index directories, so it accepted events and + answered every query with an empty result set — successfully +- `admin rebuild-toc` printed a TODO and exited 0; `--dry-run` advertised a + rebuild that did nothing +- `admin rebuild-bm25` is a prune, not a rebuild — relabelled rather than renamed + +**Known Gaps:** + +- No backfill for events indexed before v3.1 (`text_preview` stays empty) +- No real-backend / real-judge LOCOMO run, so no comparative claim +- Vector retrieval quality is not benchmarked +- GitHub repo description, topics, Discussions, and a recorded demo are + maintainer actions, not code + +**Stats:** + +- 64,626 LOC Rust across 20 crates +- Timeline: 2026-08-30 → 2026-08-31 + +--- + ## v2.7 Multi-Runtime Portability (Shipped: 2026-03-22) **Delivered:** Rust-based multi-runtime installer that converts canonical Claude plugin source into runtime-specific installations for 6 targets, replacing 5 manually-maintained adapter directories with a single conversion pipeline. diff --git a/.planning/ROADMAP.md b/.planning/ROADMAP.md index 76d0824..074c3a6 100644 --- a/.planning/ROADMAP.md +++ b/.planning/ROADMAP.md @@ -12,7 +12,7 @@ - ✅ **v2.6 Cognitive Retrieval** — Phases 39-44 (shipped 2026-03-16) - ✅ **v2.7 Multi-Runtime Portability** — Phases 45-50 (shipped 2026-03-22) - **v3.0 Competitive Parity & Benchmarks** — Phases 51-53 + Phase 51.5 (in progress; Phase 51.5 merged 2026-04-28) -- **v3.1 Make It True** — Phases 54-58 (in progress; Phases 54, 54.5, 55, 56 merged 2026-08-30, Phase 57 executing) +- ✅ **v3.1 Make It True** — Phases 54-58 (shipped 2026-08-31) ## Phases @@ -285,13 +285,22 @@ Close the claim/reality gap, then open the shop window. No new capabilities. - [x] 56-02: LOCOMO adapter v2 (real schema, mock vs llm-judge) - [x] 56-03: Smoke artifacts + HOLD comparison marketing -### Phase 57: Shop Window & Positioning (3/3 plans) — IN EXECUTION 2026-08-30 +### Phase 57: Shop Window & Positioning (3/3 plans) — COMPLETE 2026-08-31 (PR #36) - [x] 57-01: Repo hygiene (root README, LICENSE, repository URL) - [x] 57-02: Positioning writeup vs Mem0 / Zep / MemMachine / Letta - [x] 57-03: Scope trim — Tier 1/Tier 2 surface; OpenCode stub deleted -### Phase 58: Launch (side quest) +### Phase 58: Launch (side quest) — IN EXECUTION 2026-08-31 -*Updated: 2026-08-30 — Phase 56 merged (#34); Phase 57 Shop Window in execution* +- [x] Version bumped to 3.1.0 (was still 2.7.0 across v3.0 and v3.1) +- [x] CHANGELOG.md and v3.1 upgrade notes +- [x] Release archives ship all four binaries; assets renamed `agent-memory-*` +- [x] Blog post and Show HN / reddit copy drafted in `docs/launch/` +- [ ] Tag `v3.1.0` (maintainer — publishes public binaries) +- [ ] Repo description, topics, Discussions (maintainer — repo settings) +- [ ] Recorded demo (maintainer) +- [ ] Post the blog and the launch threads (maintainer) + +*Updated: 2026-08-31 — Phase 57 merged (#36); v3.1 shipped; Phase 58 launch prep* diff --git a/.planning/STATE.md b/.planning/STATE.md index d9248a9..138da78 100644 --- a/.planning/STATE.md +++ b/.planning/STATE.md @@ -1,16 +1,16 @@ --- gsd_state_version: 1.0 milestone_name: Make It True -status: in_progress +status: shipping stopped_at: null -last_updated: "2026-08-30T22:30:00.000Z" -last_activity: 2026-08-30 — Phase 56 merged (#34); Phase 57 Shop Window & Positioning in execution +last_updated: "2026-08-31T01:30:00.000Z" +last_activity: 2026-08-31 — Phase 57 merged (#36); v3.1 shipped; Phase 58 launch prep (version 3.1.0, CHANGELOG, launch drafts) progress: total_phases: 6 - completed_phases: 4 + completed_phases: 5 total_plans: 14 - completed_plans: 11 - percent: 79 + completed_plans: 14 + percent: 100 --- # Project State @@ -20,17 +20,19 @@ progress: See: .planning/PROJECT.md (updated 2026-03-22) **Core value:** Agent can answer "what were we talking about last week?" without scanning everything -**Current focus:** v3.1 Phase 57 — Shop Window & Positioning (root README, LICENSE, positioning writeup, supported-surface tiering) +**Current focus:** v3.1 Phase 58 — Launch (side quest): version 3.1.0, CHANGELOG, release archive fixes, launch drafts. The tag and the public posts are maintainer actions. ## Current Position -Phase: 57 of 58 (Shop Window & Positioning) -Plan: 01-03 implemented on `claude/phase-54-toolchain-drift-3k4fer` -Status: Phases 54, 54.5, 55, 56 merged; Phase 57 in review -Last activity: 2026-08-30 — #34 merged; Phase 57 README/LICENSE/positioning/scope-trim +Phase: 58 of 58 (Launch — side quest, not a GSD phase) +Status: all v3.1 GSD phases merged (54, 54.5, 55, 56, 57). Launch prep in review. +Last activity: 2026-08-31 — #36 merged; version bumped 2.7.0 → 3.1.0 -Progress: [████████░░] 11/14 plans merged; Phase 57's 3 plans are implemented and in review -(Phase 58 is a side quest, not a GSD phase) +Progress: [██████████] 14/14 plans merged. v3.1 GSD work complete. + +Remaining launch steps are maintainer actions: tag `v3.1.0` (publishes public +binaries), set the repo description/topics/Discussions, record the demo, and +post the blog and launch threads. ## Out-of-band Work @@ -38,12 +40,13 @@ Progress: [████████░░] 11/14 plans merged; Phase 57's 3 plan | PR | What | Status | |---|---|---| -| _(none open)_ | | | +| _(Phase 58 launch prep)_ | version 3.1.0, CHANGELOG, release fix, launch drafts | Open | ### Recently Merged | PR | What | Merged | |---|---|---| +| #36 | Phase 57 Shop Window & Positioning | 2026-08-31 | | #34 | Phase 56 Honest Benchmarks | 2026-08-30 | | #35 | Phase 54.5 truth leaks + rustc 1.97 pin | 2026-08-30 | | #33 | Phase 55 Performance Truth | 2026-08-30 | @@ -64,3 +67,6 @@ Progress: [████████░░] 11/14 plans merged; Phase 57's 3 plan - Phase 57 tiering: Tier 1 = Claude Code + Codex CLI (PR gate); Tier 2 = Gemini + Copilot (weekly schedule) - Phase 57: OpenCode removed rather than archived — a converter whose methods return empty is a false success, not a gap - Phase 57: no comparative benchmark claim ships while the only committed results are mock-backend / mock-judge +- Phase 58: version is 3.1.0 — it had been stuck at 2.7.0 through the whole v3.0 and v3.1 line, and there are no tags in the repo +- Phase 58: release archives are `agent-memory--` and carry all four binaries; the CLI the quickstart needs was previously not shipped +- Phase 58: `admin rebuild-bm25` is a prune, not a rebuild — relabelled rather than renamed, and there is no event backfill path diff --git a/CHANGELOG.md b/CHANGELOG.md new file mode 100644 index 0000000..be2dcdc --- /dev/null +++ b/CHANGELOG.md @@ -0,0 +1,100 @@ +# Changelog + +Notable changes per release. Dates are the release date, not the merge date of +the last commit. + +The guiding rule for this file, after the v3.0 retrospective: **a change is +listed only if it is true of the shipped code.** Claims that turned out to be +aspirational are recorded as retractions, not quietly dropped. + +## v3.1.0 — Make It True (2026-08-31) + +No new capabilities. This release closes the gap between what the project +claimed and what it did. + +### Added + +- Root `README.md` and `LICENSE` (MIT) — the repository had neither +- `docs/positioning/agent-memory-vs-competition.md` — head-to-head against + Mem0, Zep, MemMachine and Letta, with a claims ledger and the platform-risk + argument stated up front +- `docs/verification/57-quickstart-transcript.md` — the quickstart executed on + a clean machine, including the three defects the first run exposed +- `docs/benchmarks.md` — what the performance harness measures and what it does + not +- Supported-surface tiering: Tier 1 (Claude Code, Codex CLI) gates every PR; + Tier 2 (Gemini, Copilot) runs on a weekly schedule +- Release archives now ship all four binaries (`memory-daemon`, + `memory-ingest`, `memory`, `memory-installer`); previously the CLI the + quickstart depends on was not in the release at all + +### Fixed + +- **First-run daemon indexed nothing.** A fresh store had no `db/search` or + `db/vector`, so the outbox indexing job never registered and every query + returned an empty result set with no error. `start_daemon` now creates them +- **`memory-orchestrator` was unreachable** from any shipped binary. `RouteQuery` + now calls it, and `memory search` is a client of that RPC +- **Hybrid search was not hybrid** — it now fetches BM25 and vector results and + fuses them +- **BM25 outbox drain was a no-op**; events are indexed, and misses warn and + increment a counter instead of passing silently +- **BM25 stored no text for events**, so previews were empty and the LLM + reranker was judging blank bodies. New event documents are `TEXT | STORED` +- **A successful LLM rerank was undone** by a salience re-sort afterwards +- **Explainability misreported what ran**: a failed LLM rerank said + `rerank=llm`, `layers_attempted` listed layers that returned nothing, and + client `stop_conditions` / `mode_override` were echoed back while being + ignored. All now report and behave truthfully +- Per-event grip full-scan on the BM25 drain (an O(n²) drain) removed +- Retrieval fan-out is concurrent unless explicitly sequential +- Query, prune and dedup share one HNSW handle; embedding dimension comes from + the embedder rather than a hardcoded 384 +- Lock poisoning is recovered and counted rather than panicking +- CI pinned to Rust 1.97 via `rust-toolchain.toml`, after a floating-stable + Clippy lint reddened `main` + +### Changed — now fails loudly instead of silently + +- `memory-daemon start --background` exits non-zero with guidance; background + daemonization is not implemented +- `memory-daemon admin rebuild-toc` exits non-zero; offline TOC rebuild is not + implemented and no longer prints a TODO and exits 0 +- `CrossEncoderReranker` returns an explicit `NotImplemented` error rather than + degrading quietly +- An unknown `--rerank` value is rejected rather than falling back to heuristic +- `admin rebuild-bm25` is relabelled: it prunes documents below `--min-level` + and re-indexes nothing, which is what it always did + +### Removed (breaking) + +- **OpenCode is no longer a supported runtime.** Every method of its converter + returned empty, so `memory-installer --agent opencode` exited 0 and wrote no + files. The converter, the `Runtime::OpenCode` variant, its tool mappings, its + bats suite and the archived plugin directory are gone. `--agent opencode` now + exits 2. The runtime-agnostic `memory-ingest --agent opencode` path is + unaffected. See [UPGRADING](docs/UPGRADING.md) + +### Benchmarks + +- The "64.6 second TOC navigation" figure is **retracted**. It timed + ingest-time summarization rollup and labelled it navigation. Warm query + `single.toc` p50 is 0.13 ms over 30 samples +- Percentiles are withheld below 10 samples (p90) and 30 samples (p99) instead + of being interpolated from 3 +- Committed results in `benchmarks/results/` are **mock-backend and mock-judge** + and are labelled as such. No comparative accuracy claim ships anywhere in this + repository until a real-backend, real-judge run is committed beside it + +### Known gaps + +- No backfill for events indexed before v3.1 — their `text_preview` stays empty +- Vector search requires the embedding model download on first daemon start; + with no network the daemon warns and runs BM25-only +- Ingest to searchable is a ~1 minute scheduled outbox drain, not synchronous + +## Earlier releases + +Milestones v1.0 through v3.0 predate this file. Their scope is recorded in +`.planning/MILESTONES.md` and `.planning/ROADMAP.md`, and upgrade notes for +v2.0.0 through v2.2.0 are in [docs/UPGRADING.md](docs/UPGRADING.md). diff --git a/Cargo.toml b/Cargo.toml index ae21fa1..1d97571 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -24,7 +24,7 @@ members = [ ] [workspace.package] -version = "2.7.0" +version = "3.1.0" edition = "2021" license = "MIT" repository = "https://github.com/SpillwaveSolutions/agent-memory" diff --git a/README.md b/README.md index b90efdd..9581775 100644 --- a/README.md +++ b/README.md @@ -86,6 +86,10 @@ cargo build --release -p memory-daemon -p memory-ingest -p memory-cli export PATH="$PWD/target/release:$PATH" ``` +Prefer not to compile? Each release ships one archive per platform containing +all four binaries — see +[docs/setup/quickstart.md](docs/setup/quickstart.md#option-b-prebuilt-binaries). + ### 2. Start the daemon The daemon runs in the foreground. There is no built-in background mode — use @@ -159,7 +163,7 @@ is experimental. | Passive hook capture → `memory-ingest` | **Solid** | Covered by the bats CLI suites on Linux + macOS | | TOC build and drill-down navigation | **Solid** | Year → Month → Week → Day → Segment → Grip | | Grips / provenance | **Solid** | Excerpts link back to the events they came from | -| BM25 keyword search (Tantivy) | **Solid** | Exact tokens, no stemming (`jwt` does not match `JWTs`). Indexes built before v3.1 do not store text — rebuild to get event previews | +| BM25 keyword search (Tantivy) | **Solid** | Exact tokens, no stemming (`jwt` does not match `JWTs`). Events indexed before v3.1 have empty `text_preview` and there is no backfill command — see [UPGRADING](docs/UPGRADING.md) | | Vector search (HNSW + Candle) | **Solid** | First daemon start downloads the embedding model; with no network the daemon warns and runs BM25-only | | Topic graph | **Works** | Clustering quality is not benchmarked | | Hybrid fusion + `RouteQuery` orchestration | **Works** | Wired end-to-end in Phase 54; explainability reports what actually ran | @@ -215,6 +219,7 @@ unaffected by tiering. | [docs/verification/57-quickstart-transcript.md](docs/verification/57-quickstart-transcript.md) | The transcript of this quickstart being run on a clean machine, defects and all | | [docs/positioning/agent-memory-vs-competition.md](docs/positioning/agent-memory-vs-competition.md) | Head-to-head vs Mem0 / Zep / MemMachine / Letta | | [docs/UPGRADING.md](docs/UPGRADING.md) | Version-to-version migration notes | +| [CHANGELOG.md](CHANGELOG.md) | What changed per release, including retractions | ## Contributing diff --git a/crates/memory-adapters/Cargo.toml b/crates/memory-adapters/Cargo.toml index 1ea4c5f..7fac15f 100644 --- a/crates/memory-adapters/Cargo.toml +++ b/crates/memory-adapters/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "memory-adapters" -version = "2.7.0" +version.workspace = true edition = "2021" description = "Agent adapter SDK for multi-agent memory integration" license = "MIT" diff --git a/crates/memory-daemon/src/cli.rs b/crates/memory-daemon/src/cli.rs index 6d6e33b..41ce1c7 100644 --- a/crates/memory-daemon/src/cli.rs +++ b/crates/memory-daemon/src/cli.rs @@ -306,7 +306,8 @@ pub enum AdminCommands { dry_run: bool, }, - /// Rebuild BM25 index with level filtering + /// Prune BM25 documents below a TOC level. Despite the name this does not + /// re-index anything -- it only removes documents below `--min-level`. RebuildBm25 { /// Minimum TOC level to keep: segment, day, week, month, year #[arg(long, default_value = "day")] diff --git a/crates/memory-daemon/src/commands.rs b/crates/memory-daemon/src/commands.rs index 6c2fb0f..d111b90 100644 --- a/crates/memory-daemon/src/commands.rs +++ b/crates/memory-daemon/src/commands.rs @@ -1532,7 +1532,8 @@ fn handle_prune_vectors( /// Handle the rebuild-bm25 command. /// -/// Rebuilds the BM25 index keeping only documents at or above the specified level. +/// Despite the name, this prunes rather than rebuilds: it removes documents +/// below `min_level` and re-indexes nothing. fn handle_rebuild_bm25(db_path: &str, min_level: &str, search_path: Option) -> Result<()> { use memory_search::{SearchIndex, SearchIndexConfig, SearchIndexer}; @@ -1554,8 +1555,9 @@ fn handle_rebuild_bm25(db_path: &str, min_level: &str, search_path: Option-.tar.gz` and contain all four binaries + (`memory-daemon`, `memory-ingest`, `memory`, `memory-installer`). Previously + the CLI the quickstart depends on was not shipped at all. +- **Runtimes are tiered.** Claude Code and Codex CLI (Tier 1) gate every PR; + Gemini and Copilot (Tier 2) moved to a weekly scheduled CI run. Tiering is + not removal — the Tier 2 converters and their tests are unchanged. +- **Explainability now reports what actually ran.** A failed LLM rerank reports + `rerank=heuristic` rather than `rerank=llm`, and `layers_attempted` lists + only layers that were invoked. +- **Toolchain is pinned** to Rust 1.97 via `rust-toolchain.toml`. + +### Not changed + +Storage format, gRPC API, config file schema, and hook payloads are unchanged. +No data migration is required. + +--- + ## v2.1.0 to v2.2.0 (Multi-Agent Ecosystem) **Release Focus:** Cross-agent discovery, multi-adapter support, and ecosystem documentation @@ -337,6 +409,7 @@ memory-daemon admin rebuild-index --type all | Version | Release Date | Key Changes | |---------|--------------|-------------| +| v3.1.0 | 2026-08-31 | Phase 54-57: Make It True — orchestrator wired, honest benchmarks, README/LICENSE, OpenCode removed | | v2.2.0 | TBD | Phase 18-23: Multi-agent ecosystem, cross-agent discovery, CLOD format | | v2.1.0 | TBD | Phase 16-17: Ranking enhancements, index lifecycle | | v2.0.0 | 2026-02-01 | Topic graph, vector search, hybrid search | @@ -344,4 +417,4 @@ memory-daemon admin rebuild-index --type all --- -*Last Updated: 2026-02-06* +*Last Updated: 2026-08-31* diff --git a/docs/launch/launch-copy.md b/docs/launch/launch-copy.md new file mode 100644 index 0000000..931b209 --- /dev/null +++ b/docs/launch/launch-copy.md @@ -0,0 +1,163 @@ +# Launch copy — v3.1.0 + +Drafts for Phase 58. Nothing here has been posted. Read the notes at the bottom +before posting anything. + +--- + +## Show HN + +**Title** (80 char limit; the guideline is "Show HN: " plus a plain description, +no adjectives, no exclamation marks): + +``` +Show HN: Agent-Memory – local-first memory for AI coding agents, zero-token capture +``` + +**Text:** + +``` +Every memory layer I looked at has the same capture contract: the agent decides +what is worth remembering and calls an API. That costs tokens on every turn, and +it gets skipped exactly when the context window is under pressure — which is when +memory matters most. + +Agent-Memory captures passively instead. CLI hooks (SessionStart, UserPromptSubmit, +PostToolUse, Stop) pipe events into an append-only RocksDB log. The agent is not in +the loop and pays nothing. Retrieval is a time-hierarchical table of contents the +agent drills down — Year → Month → Week → Day → Segment → Grip — rather than a +similarity search over everything, so answering "what were we talking about last +week?" doesn't mean loading last week into context. + +It's Rust. Everything is local: the event log, a Tantivy BM25 index, an HNSW vector +index, and Candle embeddings all run on your machine. Nothing leaves it unless you +turn on LLM summarization or reranking and supply a key. + +The README has a status table saying which parts are solid, which are experimental, +and which are not implemented, because the last four phases of this project were +spent finding out that my own verification process had been grading claims instead +of behavior. Some specifics, since they're more interesting than the pitch: + +- A crate I'd certified as "integrated" was unreachable from any shipped binary +- A benchmark showing 64.6s TOC navigation was a harness defect — it timed + ingest-time summarization and labelled it navigation. Real warm p50 is 0.13ms +- The quickstart in my own README returned zero results for content just ingested, + on a fresh install, with exit code 0 and no error + +There is no benchmark comparison in the repo. The only committed results are +mock-backend runs that test the harness, not the retrieval quality, and I'm not +publishing a LoCoMo number until I've run it properly. + +https://github.com/SpillwaveSolutions/agent-memory +``` + +--- + +## r/rust + +**Title:** + +``` +Agent-Memory 3.1: local-first conversational memory for AI coding agents (Rust, RocksDB + Tantivy + HNSW) +``` + +**Body:** + +``` +Local-first memory layer for AI coding CLIs — the agent's conversation history +becomes a queryable store instead of evaporating at the end of a session. + +Stack, since that's what this sub actually cares about: + +- **RocksDB** for the append-only event log (immutable; summaries derived out of band) +- **Tantivy** for BM25 keyword retrieval +- **usearch** HNSW for vector retrieval, with **Candle** running the embedding model + locally — no Python, no inference service +- **tonic**/gRPC daemon, **clap** CLI, **tokio-cron-scheduler** for the background + rollup and indexing jobs +- Toolchain pinned via `rust-toolchain.toml` after a floating-stable Clippy lint + reddened main overnight + +The design bit I'd be interested in feedback on: retrieval is *navigational* rather +than purely similarity-based. A time hierarchy (Year → Month → Week → Day → Segment) +is built by scheduled rollup jobs, and queries drill down it, reading a summary at +each level. Three layers (BM25, vector, topic graph) feed a fusion step. The intent +is that answering "what did we decide about auth last month" reads a handful of +summary nodes rather than embedding-searching the entire corpus. + +This release added no features. It was four phases of closing the gap between what +the project claimed and what it did — including a benchmark number that turned out +to be timing the wrong thing by five orders of magnitude, and a first-run bug where +the daemon accepted events and answered every query with an empty result set, +successfully, because the index directories didn't exist yet. + +MIT. Builds on stable 1.97. Feedback on the retrieval architecture very welcome. + +https://github.com/SpillwaveSolutions/agent-memory +``` + +--- + +## r/LocalLLaMA + +**Title:** + +``` +Agent-Memory: fully local memory for coding agents — RocksDB + BM25 + local embeddings, nothing phones home +``` + +**Body:** + +``` +Built this because every agent memory option I found was a hosted API, and I do +client work where the conversation history is the sensitive artifact. + +Everything runs on your box: + +- Append-only event log (RocksDB) +- BM25 keyword index (Tantivy) +- Vector index (HNSW) with embeddings generated locally by Candle — + all-MiniLM-L6-v2 downloaded once, then no network +- gRPC daemon + CLI + +No account, no service, no telemetry. LLM summarization and LLM reranking are +optional and off unless you supply a key; without one it falls back to a heuristic +ranker and *says so* in the response payload rather than pretending. + +Capture is passive — CLI hooks pipe events in, so the model never spends tokens +deciding what to remember and can't skip it when the context fills up. Works +alongside Claude Code and Codex CLI today (Gemini and Copilot are best-effort). + +Honest status, since this sub can smell marketing: BM25 works well and is exact-token +(no stemming). Vector search works but the retrieval quality is not benchmarked — the +only committed benchmark artifacts are mock-backend runs that test the harness, and I +am not publishing a LoCoMo number until I've run a real one. The README has a table +listing what's solid, what's experimental, and what's not implemented. + +MIT, Rust, Linux/macOS. + +https://github.com/SpillwaveSolutions/agent-memory +``` + +--- + +## Before posting + +1. **Tag `v3.1.0` first** so the release links resolve and Show HN visitors can + download binaries rather than compile. +2. **Set the GitHub repo description and topics** (`ai-agents`, `memory`, `rust`, + `claude-code`, `local-first`) and enable Discussions. A Show HN landing on a + repo with no description reads as abandoned. +3. **Post the blog post first**, then link it from the HN thread as a comment + rather than submitting it as the HN URL — the repo is the better submission. +4. **Sequence, don't shotgun.** HN and one subreddit on day one; the second + subreddit later. Simultaneous posts read as a launch campaign and get less + patience from commenters. +5. **Be around for the first three hours.** The platform-risk question ("won't + Anthropic just build this in?") will come up; the answer is in + `docs/positioning/agent-memory-vs-competition.md` and should be given in your + own words, not pasted. +6. **Do not add a benchmark claim under pressure.** If someone asks how it scores + on LoCoMo, the answer is "I haven't run it properly, so I'm not going to quote a + number" — which is a stronger answer here than a weak score, and the whole + reason the last milestone existed. diff --git a/docs/launch/v3.1-blog-post.md b/docs/launch/v3.1-blog-post.md new file mode 100644 index 0000000..a5cadc6 --- /dev/null +++ b/docs/launch/v3.1-blog-post.md @@ -0,0 +1,195 @@ +# My AI pipeline graded an unrun benchmark 17/17. Here's what fixing that took. + +*Draft for Hightower's AI Harness Engineering. Status: ready for a human edit +pass. Every number and quote in here is checkable in the repo — links inline.* + +--- + +There is a failure mode in agentic development that no amount of code review +catches, because the code is fine. + +I build [agent-memory](https://github.com/SpillwaveSolutions/agent-memory), a +local-first conversational memory system for AI coding agents. It runs a +GSD-style planning loop: milestones decompose into phases, phases into plans, +each plan produces a VERIFICATION document that checks requirements off. Sixty +thousand lines of Rust, a hundred-odd phases of history, the whole thing driven +by agents working through planning files. + +At the end of milestone v3.0, the verification document said **17 of 17 +requirements met, no gaps.** + +The benchmark it certified had never been run. The crate it certified as +"integrated" was not reachable from any binary that ships. `cargo tree -i +memory-orchestrator` returned nothing — no shipped binary depended on it. It +was 3,000 lines of retrieval orchestration that existed, compiled, passed its +own unit tests, and could not be called by anything a user runs. + +Nothing lied. That is the part worth sitting with. + +## Verification by existence + +Every check in that document was true as written. "Benchmark suite implemented" +— there was a benchmark suite. "Orchestrator integrated" — there was an +integration module with tests. "Retrieval reranking implemented" — there was a +reranker. + +The checks tested for the *existence of structures*, not the *occurrence of +events*. A struct exists. A flag is defined. A test file is present. Each is +trivially confirmable by an agent reading the codebase, and none of them tells +you whether the thing ever ran. + +This is what I now call **verification by existence**, and it is the natural +attractor for any agentic pipeline that grades its own work. The agent writing +the verification has the code in context. Confirming a struct exists costs one +grep. Confirming a benchmark produced a number requires running it, waiting, +and reading an artifact that may not be there. Under any pressure at all — +context, time, a long plan — the loop converges on the cheap check. Not through +deception. Through gradient. + +The tell was there the whole time: **17/17, no gaps.** Real engineering work +does not come out clean. A verification document with no unchecked boxes is +evidence about the checker, not the code. + +## The four rules + +The fix was not better prompting. It was changing what counts as a check. + +**1. Execution-evidence rule.** Any requirement whose truth depends on +something running — a benchmark, an end-to-end behavior, a performance number, +a quickstart — must cite a committed artifact produced by actually running it. +A results file. A CI run. A transcript. The existence of a struct, a flag, or a +test file verifies the existence of a struct, a flag, or a test file. + +**2. Reachability rule.** A phase that adds a crate is not complete until +`cargo tree -i ` shows a dependent binary — or the phase explicitly +declares the crate dormant with a tracking requirement. "Integrated" now has a +mechanical definition that an agent cannot argue its way around. + +**3. `human_verification` items are blockers by default** when they gate the +milestone's stated goal. In v3.0 the *entire goal* was the benchmark, and the +benchmark item was waved through as a nice-to-have. + +**4. Write the retrospective.** The v3.0 retrospective did not exist until this +milestone. Its first line now reads: *the code is strong; the claims were not.* + +## What four phases of enforcement actually turned up + +I ran a milestone — "Make It True" — that added no features. Only closing the +gap between claim and reality. Here is what came out, because the specifics are +more useful than the principle. + +**The orchestrator got wired.** `RouteQuery` now calls it, and the CLI is a +client of that RPC. `cargo tree -i memory-orchestrator` returns a path to a +shipped binary. Rule 2, applied once, closed a gap that had been open for a +whole milestone. + +**Hybrid search was not hybrid.** It fetched one layer. It now fetches BM25 and +vector and fuses them. + +**The BM25 outbox drain was a no-op.** Events went into the queue and nothing +came out. Now they are indexed, and a miss warns and increments a counter +instead of passing silently. + +**The 65-second number was a harness defect.** A recorded benchmark showed TOC +navigation at 64,576 ms p50 — catastrophic for a system whose entire pitch is +"answer questions about last week without scanning everything." The timer +bracketed *ingest-time summarization rollup* of a 240-event segment and labelled +the result "navigation." The actual warm query p50 is **0.13 ms**. The p90 and +p99 in that old file were interpolated from three samples; the harness now +refuses to emit p90 below 10 samples and p99 below 30. + +Note the direction. Honest measurement did not make the numbers worse. It made +them *five orders of magnitude better* and retracted a claim that had been +quietly poisoning my own roadmap. + +**Then a second-order problem.** With the orchestrator wired, the explainability +payload — the thing that tells you which retrieval layers ran and how results +were ranked — was itself lying. A failed LLM rerank reported `rerank=llm`. +`layers_attempted` listed layers that had returned nothing. Client-supplied +stop conditions were echoed back in the response while being ignored by the +code. The feature whose entire job was *telling the truth about what ran* was +not telling the truth about what ran. + +That is the thing about this failure mode. It has a second floor. + +## The one that got me + +Last phase. Shop window: write a README, a license, a positioning document. +Make the repo legible to a stranger. Low risk, mostly prose. + +I wrote the README first, then ran its quickstart verbatim on a clean container +— no toolchain, no store, nothing cached. Build. Start the daemon. Add two +facts. Search for one of them. + +```console +$ memory add --content "We chose RS256 over HS256 for the auth service JWTs" --agent claude +{"status":"ok","results":{"created":true,"event_id":"01M1ABYF8RR4F8JCGGE2Z0V864"}} + +$ memory search "which JWT signing algorithm did we pick" --top 5 +{"status":"ok","results":[],"meta":{"retrieval_ms":0,"confidence":0.0}} +``` + +Empty. `confidence 0.0`. No error, no warning, exit code zero. + +A fresh store has no index directories. The outbox indexing job only registers +if its directory already exists. So a first-run daemon accepted events forever +and answered every single query with nothing — successfully. + +Every unit test passed. Every integration test passed. CI was green on both +operating systems. The product did not work if you had never used it before. + +It gets better. The remedy the daemon itself printed in its startup log — +"run rebuild-indexes to initialize the search index" — failed on the RocksDB +lock while the daemon was running, and reported "No documents found" when it +wasn't, because it indexes summary nodes rather than raw events. And a sibling +command, `admin rebuild-toc`, printed "TOC rebuild not yet fully implemented" +and **exited 0**, while its `--dry-run` mode advertised "to actually rebuild, +run without --dry-run." + +Three commands. Three different ways of reporting success while doing nothing. + +None of this is exotic. It is the most ordinary bug in software: the +happy path nobody walks because everybody's machine is already warm. What made +it invisible for months was that my verification loop had no step that started +from nothing. + +## The generalization + +If you are running an agentic pipeline that produces its own verification, the +question is not whether your agents are honest. Mine were. The question is +whether your checks can be satisfied without the thing happening. + +Three that transfer: + +- **Grade on artifacts, not assertions.** "The benchmark is implemented" is + free. "Here is `results.json` with a timestamp and a git SHA" is not. Make + the expensive one the only one that counts. +- **Reachability is mechanical — so mechanize it.** "Is this integrated?" is a + judgment call an agent will resolve generously. `cargo tree -i` is not. + Every ecosystem has an equivalent. Use it. +- **Start from nothing, regularly.** Not a test fixture. An actual clean + machine, following your actual documentation, in order, adding nothing. It is + the single highest-yield hour I spent across four phases. + +And one that is less about tooling: **a verification document with no unchecked +boxes is a red flag, not a milestone.** The current one has four items in a +section headed "Not done, stated rather than waved through." That is what a +real one looks like. + +## Where the project is + +v3.1 ships today. No new features — the whole milestone was closing this gap. +The [README](https://github.com/SpillwaveSolutions/agent-memory) has a status +table that says which parts are solid, which are experimental, and which are +not implemented. The +[positioning writeup](https://github.com/SpillwaveSolutions/agent-memory/blob/main/docs/positioning/agent-memory-vs-competition.md) +compares it against Mem0, Zep, MemMachine and Letta, including a section on +where they are ahead. + +It makes no benchmark comparison. The only committed results are mock-backend +and mock-judge runs, which measure whether the harness works, not whether the +memory is good. MemMachine publishes 0.9169 on LoCoMo. I publish nothing on +LoCoMo, because I have not run it properly, and the point of the last four +phases was to stop doing exactly that. + +The quickstart works now. I know, because I ran it. diff --git a/docs/setup/quickstart.md b/docs/setup/quickstart.md index 6a64f24..ee4fb21 100644 --- a/docs/setup/quickstart.md +++ b/docs/setup/quickstart.md @@ -57,15 +57,26 @@ cargo build --release - [ ] Download the latest release for your platform - [ ] Unpack to a local bin directory +One archive per platform contains all four binaries: `memory-daemon`, +`memory-ingest`, `memory` (the CLI), and `memory-installer`. Assets are named +`agent-memory--.tar.gz`, and the tarball unpacks into a +directory of that name. + ```bash mkdir -p ~/.local/bin -PLATFORM=$(uname -s | tr '[:upper:]' '[:lower:]') -ARCH=$(uname -m) -curl -L "https://github.com/SpillwaveSolutions/agent-memory/releases/latest/download/memory-daemon-${PLATFORM}-${ARCH}.tar.gz" | tar xz -C ~/.local/bin -curl -L "https://github.com/SpillwaveSolutions/agent-memory/releases/latest/download/memory-ingest-${PLATFORM}-${ARCH}.tar.gz" | tar xz -C ~/.local/bin -chmod +x ~/.local/bin/memory-daemon ~/.local/bin/memory-ingest +VERSION=3.1.0 +case "$(uname -s)" in Darwin) OS=macos ;; Linux) OS=linux ;; esac +case "$(uname -m)" in x86_64|amd64) ARCH=x86_64 ;; arm64|aarch64) ARCH=aarch64 ;; esac +ASSET="agent-memory-${VERSION}-${OS}-${ARCH}" + +curl -fL "https://github.com/SpillwaveSolutions/agent-memory/releases/download/v${VERSION}/${ASSET}.tar.gz" \ + | tar xz -C /tmp +install -m 0755 /tmp/${ASSET}/memory-daemon /tmp/${ASSET}/memory-ingest \ + /tmp/${ASSET}/memory /tmp/${ASSET}/memory-installer ~/.local/bin/ ``` +Verify the download against the release's `SHA256SUMS.txt` if you want to. + ### 3) Add binaries to PATH - [ ] Ensure `~/.local/bin` is on your PATH @@ -77,6 +88,7 @@ export PATH="$HOME/.local/bin:$PATH" > Verify now (optional): > - `memory-daemon --version` > - `memory-ingest --version` +> - `memory --version` ### 4) Create a minimal config (single-agent defaults)