Context
Follow-up from #84 / #91 (incremental reindexing, ADR-0005). The per-file manifest keys reuse on the file's content hash only. A tree-sitter grammar or chunker change (e.g. bumping DESIRED_CHUNK_LENGTH_CHARS is covered, but a grammar upgrade or a chunking-heuristic fix is not) leaves unchanged files with chunks produced by the old chunker until their bytes change.
Proposal
- Add a
chunkerFingerprint (hash of the chunker version + grammar set) to manifest.json.
manifest_compatible fails closed when it differs, forcing a full rebuild.- Bump
INDEX_SCHEMA_VERSION only if the field cannot be defaulted for older manifests.
Acceptance
- A grammar/chunker change invalidates the incremental seed (test in
cache_orchestrator/tests.rs).
.please/docs/decisions/0005-per-file-hash-manifest-incremental-reindex.md documents the new key.
Context
Follow-up from #84 / #91 (incremental reindexing, ADR-0005). The per-file manifest keys reuse on the file's content hash only. A tree-sitter grammar or chunker change (e.g. bumping
DESIRED_CHUNK_LENGTH_CHARSis covered, but a grammar upgrade or a chunking-heuristic fix is not) leaves unchanged files with chunks produced by the old chunker until their bytes change.Proposal
chunkerFingerprint(hash of the chunker version + grammar set) tomanifest.json.manifest_compatiblefails closed when it differs, forcing a full rebuild.INDEX_SCHEMA_VERSIONonly if the field cannot be defaulted for older manifests.Acceptance
cache_orchestrator/tests.rs)..please/docs/decisions/0005-per-file-hash-manifest-incremental-reindex.mddocuments the new key.