Skip to content

fix(rvf-runtime): clear the deletion bit when an id is re-ingested - #759

Open
ohdearquant wants to merge 2 commits into
ruvnet:mainfrom
ohdearquant:fix/undelete-on-reingest
Open

fix(rvf-runtime): clear the deletion bit when an id is re-ingested#759
ohdearquant wants to merge 2 commits into
ruvnet:mainfrom
ohdearquant:fix/undelete-on-reingest

Conversation

@ohdearquant

@ohdearquant ohdearquant commented Aug 2, 2026

Copy link
Copy Markdown
Contributor

Fixes #748.

delete(ids) sets bits in the deletion bitmap. ingest_batch never cleared them, so re-inserting a
deleted id left the bit set. Two consequences, and the second is data loss:

  1. query() skips the id, so the newly ingested vector is unreachable even though it is stored.
  2. compact() treats the id as dead and physically removes it, destroying a payload that was
    written after the deletion.

DeletionBitmap::clear_ids already existed for exactly this; its only caller was its own unit test.

The fix

One line, placed immediately after the loop that inserts the vectors, which is the point where the
id becomes live again:

self.deletion_bitmap.clear_ids(&valid_ids);

Verified at main, not inherited from the report

The issue confirms 0.2.0 and says the interaction is "present in 0.3.0". I reproduced it at main
rather than taking that forward, because the same crate had a separate issue that was already fixed
by the time it was filed. Both halves reproduce:

test without the fix with the fix
reingest_after_delete_makes_the_id_visible_again FAILED — "id 42 stayed soft-deleted after re-ingest, so the new payload is unreachable" ok
compact_keeps_a_vector_reingested_after_delete FAILED — "compaction dropped a vector that had been re-ingested after deletion" ok

Both are in the PR. They were written before the fix and observed failing first, then observed
passing, and the failure was re-confirmed afterwards by removing the added line and watching exactly
those two go red while the other 243 stayed green. A regression test that passes with the fix
reverted would not be guarding anything.

Full suite at the current head: 328 passed, 0 failed across the rvf-runtime library (246) plus integration and doc targets. cargo fmt --all --check clean.

One thing I checked and deliberately did not change

delete() also drops the id from a COW child's membership_filter, so my first version of this
patch re-added it there for symmetry. That turned out to be unnecessary: re-ingesting makes the
vector child-owned, and the query paths find it through self.vectors without consulting membership.
I confirmed it by writing a COW branch/delete/re-ingest/query test and watching it pass both with and
without that block, which means the block was doing nothing. It is not in this PR, and neither is
that test — it passes on unmodified main, so shipping it alongside the two real ones would have
made it look like a regression guard when it characterises behaviour that was already correct.

Mentioning it because "does this need the same treatment for COW children" is the first question a
reviewer should ask about this fix, and the answer is no, measured rather than assumed.

Note on lint

cargo clippy on this crate reports three pre-existing errors, all in rvf-types (impl can be derived ×2, missing Default for Sha256). They are untouched by this PR. Worth knowing why they
survive: rvf-runtime is not a workspace member, so cargo clippy --workspace and
cargo test --workspace never reach it or its path deps. Happy to open that separately if useful.


Note on this PR's CI. Tests (core-and-rest) is red on every branch in this repository,
including main: the job is cancelled at its 240-minute cap while still compiling and never
reaches the test phase. #786 restores the exclusion list that the shard's packages: value loses
to a shell comment, #784 unblocks the ruvector-filter test target that the compiler cannot
finish, and #787 fixes a deadlock waiting behind both. That failure is not caused by this branch.

delete(ids) sets bits in the deletion bitmap and ingest_batch never cleared
them, so re-inserting a deleted id left it tombstoned. query() then skips the
id, making the newly written vector unreachable, and compact() reads the stale
bit as dead and physically removes the vector, destroying a payload that was
written after the deletion.

DeletionBitmap::clear_ids already existed for this; its only caller was its own
unit test. The call goes immediately after the insert loop, which is where the
id becomes live again.

Adds two regression tests, one per consequence. Both were observed failing
before the fix and passing after, and removing the added line again turns
exactly those two red while the other 243 stay green.
@ohdearquant
ohdearquant marked this pull request as draft August 2, 2026 14:37
@ohdearquant
ohdearquant marked this pull request as ready for review August 3, 2026 18:08
Add a regression test that ingests id 42, deletes it, re-ingests it with
a distinguishable replacement vector, closes the store, reopens it from
disk, and asserts the query returns the replacement rather than a
soft-deleted (or stale) result. The two existing re-ingest tests only
covered the in-memory and compaction paths; neither closed and reopened
the store, so the persisted deleted_ids manifest snapshot was unpinned.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug(rvf-runtime): delete() bitmap is permanent — re-ingest does not undelete; compact drops re-ingested data

1 participant