Skip to content

Switch to foldhash-portable and add generic BuildHasher support - #24

Merged
arthurprs merged 1 commit into
masterfrom
foldhash-portable
Mar 29, 2026
Merged

Switch to foldhash-portable and add generic BuildHasher support#24
arthurprs merged 1 commit into
masterfrom
foldhash-portable

Conversation

@arthurprs

@arthurprs arthurprs commented Mar 29, 2026

Copy link
Copy Markdown
Owner

Summary

  • Replace xxhash-rust with foldhash-portable (portable feature)
  • Add S: BuildHasher generic parameter to Filter, FilterRef, and Builder
  • Preserve hasher across grow/shrink/merge via S: Clone
  • Add _with_hasher constructors for Filter; Builder::new(filter) takes an empty Filter
  • Add Filter::with_hasher() for swapping hasher (e.g. after deserialization)
  • Add truncate_to_fingerprint() and compute_fingerprint_with_hasher() public API
  • Remove compute_fingerprint() from public API (now test-only)
  • Export StableBuildHasher for use with fingerprint APIs
  • Remove StableHasher LE normalization (foldhash-portable handles it)
  • Keep usize/isize → u64/i64 widening for 32-bit compatibility
  • Add cross-platform hash stability test (verified on x86_64 LE + PowerPC 32-bit BE)
  • Add cross-platform CI job (powerpc, i686, aarch64)
  • Document T: Hash stability caveats and custom hasher usage

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates qfilter’s hashing to use foldhash-portable (portable/stable across platforms) and extends the public API to support custom hashers via a generic S: BuildHasher on Filter and Builder.

Changes:

  • Replace xxhash-rust with foldhash-portable and introduce StableBuildHasher as the default stable hasher.
  • Add generic hasher support (S) to Filter/Builder, plus _with_hasher constructors and with_hasher() to swap hashers.
  • Update docs, tests, and CI (including cross-target runs) to validate cross-platform stability.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 6 comments.

Show a summary per file
File Description
src/stable_hasher.rs Switch stable hashing implementation to foldhash-portable and add StableBuildHasher.
src/lib.rs Add S: BuildHasher generic support across Filter/Builder, hasher swapping, and updated hashing paths/tests.
README.md Update documentation for new hashing backend and custom-hasher caveats.
Cargo.toml Swap dependency from xxhash-rust to foldhash-portable with portable feature.
.github/workflows/ci.yml Add cross-target test job to validate portability/stability.
Comments suppressed due to low confidence (2)

src/lib.rs:2269

  • The merge/merge_sorted signatures use other: &Filter<impl AsRef<[u8]>, impl Sized>. impl Sized adds no constraint (all type params are Sized by default) and makes the API harder to read. Consider using explicit generic parameters instead (e.g., B2: AsRef<[u8]>, S2) so it's clear this accepts any buffer/hasher types.
            *self = self.merge_sorted(keep_duplicates, other)?;
        } else if other.fingerprint_size() >= self.fingerprint_size() {
            // Different fingerprint sizes: truncation changes sort order,
            // so fall back to one-by-one insertion.
            let max_count = if keep_duplicates { u64::MAX } else { 1 };

src/lib.rs:1284

  • Builder<S> is already declared under impl<S: Default> Builder<S>, so the extra where S: Default on insert_fingerprint is redundant noise. Dropping the where clause will simplify the signature without changing bounds.
        match self.insert_impl(duplicate, hash) {
            Ok(inserted) => Ok(inserted),
            Err(Error::CapacityExceeded) => {
                *self = self.filter.rebuild_grown()?;

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/lib.rs
Comment thread src/lib.rs
Comment thread src/lib.rs
Comment thread .github/workflows/ci.yml Outdated
Comment thread src/lib.rs Outdated
Comment thread src/lib.rs Outdated

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 5 comments.

Comments suppressed due to low confidence (1)

src/lib.rs:2349

  • rebuild_grown() returns Builder<S>, but the builder it constructs ultimately uses S::default() (via Builder::with_qr / Filter::with_qr_generic) instead of preserving self.build_hasher. For custom hashers this means an automatic grow will change the hasher/seed and invalidate future contains/insert/remove calls that accept T: Hash. Ensure rebuild uses the same build_hasher as self (e.g. clone/move it into the rebuilt filter).
    fn rebuild_grown(&self) -> Result<Builder<S>, Error> {
        let max = self.max_qbits.ok_or(Error::CapacityExceeded)?;
        if max <= self.qbits {
            return Err(Error::CapacityExceeded);
        }
        let qbits = self.qbits.checked_add(1).ok_or(Error::CapacityExceeded)?;
        let rbits = NonZeroU8::new(self.rbits.get() - 1).ok_or(Error::CapacityExceeded)?;
        let mut inserter = Builder::with_qr(qbits, rbits, self.max_qbits)?;
        // Use insert_impl directly: the new filter has 2x capacity so growth cannot happen.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/lib.rs Outdated
Comment thread src/lib.rs Outdated
Comment thread src/lib.rs Outdated
Comment thread src/lib.rs Outdated
Comment thread src/lib.rs Outdated

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 2 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/stable_hasher.rs Outdated
Comment thread src/lib.rs
@arthurprs
arthurprs force-pushed the foldhash-portable branch 2 times, most recently from b4d1fbc to 96ac716 Compare March 29, 2026 20:21
@arthurprs
arthurprs requested a review from Copilot March 29, 2026 20:22

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 1 comment.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/lib.rs Outdated
Comment on lines +2243 to +2247
fn merge_sorted(
&self,
keep_duplicates: bool,
other: &Filter<impl AsRef<[u8]>>,
) -> Result<Filter, Error> {
other: &Filter<impl AsRef<[u8]>, impl Sized>,
) -> Result<Self, Error> {

Copilot AI Mar 29, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same issue as merge: impl Trait inside Filter<...> generic arguments (Filter<impl AsRef<[u8]>, impl Sized>) is not supported on stable Rust and will not compile. Prefer explicit generic parameters for other (e.g. B2, S2) and return Result<Self, Error> as you already do.

Copilot uses AI. Check for mistakes.
@arthurprs
arthurprs force-pushed the foldhash-portable branch 2 times, most recently from 2a2e4ba to 4b42dad Compare March 29, 2026 20:56
@arthurprs
arthurprs requested a review from Copilot March 29, 2026 21:14

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 4 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/lib.rs
Comment thread src/lib.rs
Comment thread src/lib.rs Outdated
Comment thread src/lib.rs
- Replace xxhash-rust with foldhash-portable (portable feature)
- Add S: BuildHasher generic parameter to Filter, FilterRef, and Builder
- Preserve hasher across grow/shrink/merge via S: Clone
- Add _with_hasher constructors for Filter; Builder::new(filter) takes an empty Filter
- Add Filter::with_hasher() for swapping hasher (e.g. after deserialization)
- Add truncate_to_fingerprint() and compute_fingerprint_with_hasher() public API
- Remove compute_fingerprint() from public API (now test-only)
- Export StableBuildHasher for use with fingerprint APIs
- Remove StableHasher LE normalization (foldhash-portable handles it)
- Keep usize/isize → u64/i64 widening for 32-bit compatibility
- Add cross-platform hash stability test (verified on x86_64 LE + PowerPC 32-bit BE)
- Add cross-platform CI job (powerpc, i686, aarch64)
- Document T: Hash stability caveats and custom hasher usage
@arthurprs
arthurprs merged commit 740515c into master Mar 29, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants