Skip to content

feat: local prune of synced sessions - #333

Open
upsetbit wants to merge 6 commits into
masterfrom
feat/local-prune
Open

feat: local prune of synced sessions#333
upsetbit wants to merge 6 commits into
masterfrom
feat/local-prune

Conversation

@upsetbit

Copy link
Copy Markdown
Contributor

What

Adds prosa prune: sessions already pushed to the server and inactive beyond a window (default 30d) get their local raw copy deleted, while the store keeps pointing at the remote location. Pruned sessions stay fully listed, searchable, and viewable offline — turns, FTS, usage, tools, and analytics all live in store.db and are untouched. Only raw reads change: prosa show --raw (and the non-TTY pipe default) stream a pruned session's raw from the server via the existing GetRaw RPC.

Nothing is ever pruned automatically. prosa sync now prints an advisory-only summary line (count + reclaimable disk) when prunable sessions exist; the --json summary carries prunable_sessions / prunable_bytes.

How

  • Migration 0011_push_prune: sync_state gains pushed_at / pushed_hash / remote_uri (transport state — until now the CLI had no local record that a session reached the server); sessions gains pruned_at.
  • Push state: every confirmed push records the server-acknowledged hash and S3 URI; sessions pushed before these columns existed are backfilled from the server manifest during reconcile, so one ordinary sync populates everything.
  • Safety model: a prune candidate needs a locally recorded push whose hash still matches the raw, and the server manifest re-confirms every id (hash + current projection version) at prune time before any file is deleted. No login or unreachable server ⇒ hard error, nothing deleted. The DB flips pruned_at before the file is removed (a failed delete reverts), so sync can never chase a missing raw.
  • Sync fences: reconcile excludes pruned rows from the catch-up work set (including under --overwrite and after future ProjectionVersion bumps); the denoise sweep skips them; pushSession short-circuits them. A re-import of a changed source file rewrites the raw and un-prunes the row atomically.
  • Zero proto/server changesPushResponse.raw_uri and GetRaw already existed; the CLI just starts using them.

Scope note

This amends INTENT.md §Out of scope: retention stays never automatic, but explicit, server-verified pruning now exists; the push-only posture is unchanged for sync (a pruned raw streams on read but never re-enters the local store). AGENTS.md, docs/concepts.md, docs/architecture/{store,cli}.md, docs/usage.md, and the rendering contract are updated to match.

Upgrade note

Migration 0011 means read-only commands (prosa, show, search, analytics) return the usual ErrStoreNeedsMigration guidance after upgrading until the first write command (prosa sync) runs — same behavior as every prior migration.

Verification

  • Unit tests across store (migration, candidate filters, guarded flip, advisory, un-prune), sync (push recording, reconcile exclusion + backfill), prune (dry-run, hash mismatch, offline, idempotency, NDJSON shape), and show (pruned raw fallback, login error, rendered annotation).
  • just ci green (fmt, vet, lint, race tests, gen-check, tidy-check).
  • Full-stack e2e against a throwaway Postgres + MinIO + prosa-server: sync pushes, prune deletes and records the real s3:// URIs, list/search/show stay local, show --raw streams byte-identical raw from the server, and a follow-up sync converges with pruned rows excluded.

🤖 Generated with Claude Code

upsetbit and others added 6 commits August 31, 2026 16:22
Migration 0011 adds pushed_at/pushed_hash/remote_uri to sync_state and
pruned_at to sessions. Session gains PrunedAt; the session upsert clears
pruned_at so a re-import is an explicit un-prune. New store APIs cover
push-state recording, prune candidate selection, the guarded prune flip,
and the advisory count. The reconcile manifest carries prune/push state
and the denoise sweep skips pruned rows.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012b7bHAgVdeT5x7mhNu5suQ
Every successful or idempotent push now records the server-confirmed
hash and raw_uri in sync_state. Reconcile excludes pruned rows from the
catch-up work set (even under --overwrite, since there is no raw file
to read) and backfills pushed state for sessions the server manifest
already holds, covering pushes made before these columns existed.
pushSession short-circuits pruned sessions with a skipped outcome.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012b7bHAgVdeT5x7mhNu5suQ
prune deletes the local raw copies of sessions inactive beyond
--older-than (default 30d) whose push the server re-confirms via the
manifest at prune time. The row flips to pruned before the file is
removed so a failed delete can revert; empty YYYY/MM shard dirs are
swept afterwards. Supports --dry-run, --limit, and NDJSON output via
the global --json flag. Requires a logged-in server; offline prune is
an error before anything is deleted.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012b7bHAgVdeT5x7mhNu5suQ
After the denoise pass, sync counts sessions pushed more than a week
ago and inactive beyond 30 days, then advertises the total and the
reclaimable bytes. Plain and TTY summaries print an advisory-only Prune
line when the count is positive; the --json summary always carries
prunable_sessions and prunable_bytes for a stable shape. The query is
local, so the advisory also works offline.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012b7bHAgVdeT5x7mhNu5suQ
show --raw and the non-TTY pipe default reroute pruned sessions to the
existing GetRaw streaming path instead of opening the deleted local
file; a missing login yields an actionable error before any RPC. The
rendered metadata line annotates a pruned raw path, and the timeline
--json payload carries pruned_at.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012b7bHAgVdeT5x7mhNu5suQ
INTENT drops the blanket veto on retention: pruning exists but stays
explicit and server-verified, and the pull-down exclusion now names the
one read-back (streaming a pruned raw via GetRaw). Concepts gains the
pruned lifecycle state; store.md documents migration 0011, the push
state columns, and the new store APIs; cli.md, usage.md, README, and
the rendering contract cover the prune command, the sync Prune advisory
line, and show's remote raw streaming for pruned sessions.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012b7bHAgVdeT5x7mhNu5suQ
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant