Context
Raised by Gemini Code Assist on #82 (declined there to keep parity with upstream semble, which also computes sizes eagerly).
CspIndex::load_from_disk recomputes file_sizes for every file referenced by the index whenever the stored root is a still-present local directory. That is an O(N) read of the whole source tree on every cached load, only to feed the file_chars side of token-savings telemetry.
Proposal
- For local-path indexes, compute per-file sizes lazily in
save_search_stats (or a helper) for the unique file_paths in the actual results, with a small per-index memo.
- Keep the eager capture for
from_git, whose shallow clone is deleted before any search runs.
- Preserve the
savings.jsonl record shape and UTF-16 accounting.
Notes
- Upstream reference:
SembleIndex.__init__ → _compute_file_sizes(root); load_from_disk passes root_path from metadata. A lazy variant is a deliberate divergence, so document it in .please/docs/references/semble.md.
- Benchmark cached-load time before/after on a mid-size repo to confirm the win.
Context
Raised by Gemini Code Assist on #82 (declined there to keep parity with upstream semble, which also computes sizes eagerly).
CspIndex::load_from_diskrecomputesfile_sizesfor every file referenced by the index whenever the storedrootis a still-present local directory. That is an O(N) read of the whole source tree on every cached load, only to feed thefile_charsside of token-savings telemetry.Proposal
save_search_stats(or a helper) for the uniquefile_paths in the actual results, with a small per-index memo.from_git, whose shallow clone is deleted before any search runs.savings.jsonlrecord shape and UTF-16 accounting.Notes
SembleIndex.__init__→_compute_file_sizes(root);load_from_diskpassesroot_pathfrom metadata. A lazy variant is a deliberate divergence, so document it in.please/docs/references/semble.md.