Switch to foldhash-portable and add generic BuildHasher support - #24
Conversation
bb4c6fe to
a5f00cc
Compare
There was a problem hiding this comment.
Pull request overview
This PR updates qfilter’s hashing to use foldhash-portable (portable/stable across platforms) and extends the public API to support custom hashers via a generic S: BuildHasher on Filter and Builder.
Changes:
- Replace
xxhash-rustwithfoldhash-portableand introduceStableBuildHasheras the default stable hasher. - Add generic hasher support (
S) toFilter/Builder, plus_with_hasherconstructors andwith_hasher()to swap hashers. - Update docs, tests, and CI (including cross-target runs) to validate cross-platform stability.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
src/stable_hasher.rs |
Switch stable hashing implementation to foldhash-portable and add StableBuildHasher. |
src/lib.rs |
Add S: BuildHasher generic support across Filter/Builder, hasher swapping, and updated hashing paths/tests. |
README.md |
Update documentation for new hashing backend and custom-hasher caveats. |
Cargo.toml |
Swap dependency from xxhash-rust to foldhash-portable with portable feature. |
.github/workflows/ci.yml |
Add cross-target test job to validate portability/stability. |
Comments suppressed due to low confidence (2)
src/lib.rs:2269
- The
merge/merge_sortedsignatures useother: &Filter<impl AsRef<[u8]>, impl Sized>.impl Sizedadds no constraint (all type params areSizedby default) and makes the API harder to read. Consider using explicit generic parameters instead (e.g.,B2: AsRef<[u8]>, S2) so it's clear this accepts any buffer/hasher types.
*self = self.merge_sorted(keep_duplicates, other)?;
} else if other.fingerprint_size() >= self.fingerprint_size() {
// Different fingerprint sizes: truncation changes sort order,
// so fall back to one-by-one insertion.
let max_count = if keep_duplicates { u64::MAX } else { 1 };
src/lib.rs:1284
Builder<S>is already declared underimpl<S: Default> Builder<S>, so the extrawhere S: Defaultoninsert_fingerprintis redundant noise. Dropping thewhereclause will simplify the signature without changing bounds.
match self.insert_impl(duplicate, hash) {
Ok(inserted) => Ok(inserted),
Err(Error::CapacityExceeded) => {
*self = self.filter.rebuild_grown()?;
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
a5f00cc to
22cb89f
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 5 out of 5 changed files in this pull request and generated 5 comments.
Comments suppressed due to low confidence (1)
src/lib.rs:2349
rebuild_grown()returnsBuilder<S>, but the builder it constructs ultimately usesS::default()(viaBuilder::with_qr/Filter::with_qr_generic) instead of preservingself.build_hasher. For custom hashers this means an automatic grow will change the hasher/seed and invalidate futurecontains/insert/removecalls that acceptT: Hash. Ensure rebuild uses the samebuild_hasherasself(e.g. clone/move it into the rebuilt filter).
fn rebuild_grown(&self) -> Result<Builder<S>, Error> {
let max = self.max_qbits.ok_or(Error::CapacityExceeded)?;
if max <= self.qbits {
return Err(Error::CapacityExceeded);
}
let qbits = self.qbits.checked_add(1).ok_or(Error::CapacityExceeded)?;
let rbits = NonZeroU8::new(self.rbits.get() - 1).ok_or(Error::CapacityExceeded)?;
let mut inserter = Builder::with_qr(qbits, rbits, self.max_qbits)?;
// Use insert_impl directly: the new filter has 2x capacity so growth cannot happen.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
22cb89f to
1aa5e96
Compare
1aa5e96 to
4aca91f
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 7 out of 7 changed files in this pull request and generated 2 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
b4d1fbc to
96ac716
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 7 out of 7 changed files in this pull request and generated 1 comment.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| fn merge_sorted( | ||
| &self, | ||
| keep_duplicates: bool, | ||
| other: &Filter<impl AsRef<[u8]>>, | ||
| ) -> Result<Filter, Error> { | ||
| other: &Filter<impl AsRef<[u8]>, impl Sized>, | ||
| ) -> Result<Self, Error> { |
There was a problem hiding this comment.
Same issue as merge: impl Trait inside Filter<...> generic arguments (Filter<impl AsRef<[u8]>, impl Sized>) is not supported on stable Rust and will not compile. Prefer explicit generic parameters for other (e.g. B2, S2) and return Result<Self, Error> as you already do.
2a2e4ba to
4b42dad
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 7 out of 7 changed files in this pull request and generated 4 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Replace xxhash-rust with foldhash-portable (portable feature) - Add S: BuildHasher generic parameter to Filter, FilterRef, and Builder - Preserve hasher across grow/shrink/merge via S: Clone - Add _with_hasher constructors for Filter; Builder::new(filter) takes an empty Filter - Add Filter::with_hasher() for swapping hasher (e.g. after deserialization) - Add truncate_to_fingerprint() and compute_fingerprint_with_hasher() public API - Remove compute_fingerprint() from public API (now test-only) - Export StableBuildHasher for use with fingerprint APIs - Remove StableHasher LE normalization (foldhash-portable handles it) - Keep usize/isize → u64/i64 widening for 32-bit compatibility - Add cross-platform hash stability test (verified on x86_64 LE + PowerPC 32-bit BE) - Add cross-platform CI job (powerpc, i686, aarch64) - Document T: Hash stability caveats and custom hasher usage
4b42dad to
06a6fcf
Compare
Summary
S: BuildHashergeneric parameter toFilter,FilterRef, andBuilderS: Clone_with_hasherconstructors for Filter;Builder::new(filter)takes an empty FilterFilter::with_hasher()for swapping hasher (e.g. after deserialization)truncate_to_fingerprint()andcompute_fingerprint_with_hasher()public APIcompute_fingerprint()from public API (now test-only)StableBuildHasherfor use with fingerprint APIsT: Hashstability caveats and custom hasher usage