crates/ruvector-diskann/src/reuse.rs:412 fails intermittently in CI:
test reuse::tests::recall_trigger_holds_under_no_drift ... FAILED
thread panicked at crates/ruvector-diskann/src/reuse.rs:412:9:
assertion failed: t.probe_recall(&v) >= 0.9
Seen on https://github.com/ruvnet/RuVector/actions/runs/30867347732/job/91861986023 (994 of 995 passed). Passes 6/6 locally on darwin-arm64 with --all-features.
Mechanism
The test's fixture is deterministic — reuse.rs contains no RNG at all. The graph build is not:
graph.rs:101 — order.shuffle(&mut rand::thread_rng())
graph.rs:346, graph.rs:370 — rand::thread_rng()
pq.rs:59 — rand::thread_rng()
So RecallTrigger::build produces a different graph every run, probe_recall is a random variable, and assert!(recall >= 0.9) fails whenever the draw lands low. The comment above the assertion ("same vectors → recall ~1.0") describes the expectation, not a guarantee.
Why it matters beyond this test
A hard threshold on an unseeded random process fails at some rate forever, and a CI job that is red for reasons unrelated to the change under review trains reviewers to merge past red. That cost is paid by every future PR, not just this one.
Options
- Seed the graph build for tests — thread a seedable RNG (
StdRng::seed_from_u64) through GraphBuilder/pq, so the test is deterministic and a failure means a real regression.
- Assert a distribution rather than a draw — run N builds and require the recall median (or a lower bound like p05) to clear 0.9, so the test measures the property it actually cares about.
- Lower the threshold — cheapest, and the worst: it hides the variance instead of describing it, and the next unlucky draw still fails.
Option 1 is preferable; it is the only one that makes a failure mean something.
Found while verifying that this failure was not caused by #791, which touches no diskann file.
🤖 Generated with claude-flow
https://claude.ai/code/session_01WfyBMvexdN4bhWVaqSXKvR
crates/ruvector-diskann/src/reuse.rs:412fails intermittently in CI:Seen on https://github.com/ruvnet/RuVector/actions/runs/30867347732/job/91861986023 (994 of 995 passed). Passes 6/6 locally on darwin-arm64 with
--all-features.Mechanism
The test's fixture is deterministic —
reuse.rscontains no RNG at all. The graph build is not:graph.rs:101—order.shuffle(&mut rand::thread_rng())graph.rs:346,graph.rs:370—rand::thread_rng()pq.rs:59—rand::thread_rng()So
RecallTrigger::buildproduces a different graph every run,probe_recallis a random variable, andassert!(recall >= 0.9)fails whenever the draw lands low. The comment above the assertion ("same vectors → recall ~1.0") describes the expectation, not a guarantee.Why it matters beyond this test
A hard threshold on an unseeded random process fails at some rate forever, and a CI job that is red for reasons unrelated to the change under review trains reviewers to merge past red. That cost is paid by every future PR, not just this one.
Options
StdRng::seed_from_u64) throughGraphBuilder/pq, so the test is deterministic and a failure means a real regression.Option 1 is preferable; it is the only one that makes a failure mean something.
Found while verifying that this failure was not caused by #791, which touches no diskann file.
🤖 Generated with claude-flow
https://claude.ai/code/session_01WfyBMvexdN4bhWVaqSXKvR