feat(core,wasm): add a lattice-simd distance backend that vectorizes on wasm32 by ohdearquant · Pull Request #762 · ruvnet/RuVector

ohdearquant · 2026-08-02T12:35:28Z

The gap

ruvector-core's f32 euclidean, cosine, and dot distance functions have two backends: SimSIMD on native, scalar otherwise; Manhattan never had a SimSIMD path and uses its own simd_intrinsics dispatch (see its section below).
SimSIMD's call sites are gated off on wasm32 (the dependency itself still resolves there), so
browser builds without this PR's feature take the scalar path — a 4x-unrolled loop for
euclidean, a single-pass loop for cosine, and iter().zip().map().sum() for dot product.

That path matters more on wasm than anywhere else, because hnsw is unavailable there (mmap), so
wasm search is brute force and distance is close to the whole query cost.

The change

A third backend behind an off-by-default lattice-simd feature, routing the four public f32
distance adapters through lattice-embed's kernels, which do compile to simd128 on wasm32
(euclidean, cosine, and dot in the opening commits; Manhattan joined in a later commit, see
its own section below). ruvector-wasm gets a
matching passthrough feature beside its existing simd.

For euclidean, cosine, and dot the three cfg arms are mutually exclusive and exhaustive —
lattice-simd wins where both are enabled, otherwise SimSIMD on non-wasm, otherwise scalar —
so exactly one compiles for any feature/target combination. Manhattan's dispatch is two-arm
(it never had a SimSIMD path): lattice-simd, otherwise the existing simd_intrinsics
dispatch with its scalar fallback. Default native builds are untouched and still use SimSIMD
(or, for Manhattan, simd_intrinsics); default wasm32 builds remain scalar.

Kernels only, not the model stack

The dependency is now pinned default-features = false, which is what makes a kernels-only
dependency possible: native is the feature that pulls lattice-inference, so with defaults off
there is no model, tokenizer, or download stack.

Verified rather than assumed, with a positive control on the same invocation:

cargo tree ... --features lattice-simd | grep -c lattice-inference   ->  0
cargo tree ... --features lattice-simd | grep -c lattice-embed       ->  1

lattice-embeddings now forwards native and download explicitly, so its behaviour is unchanged.

Cost, stated plainly

lattice-embed requires Rust >= 1.93. Enabling either lattice feature raises the effective
MSRV for whoever turns it on, the same shape as the existing simd-avx512 feature requiring

= 1.89. The workspace declares Rust 1.77, and this PR does not change that; one caveat worth
stating is that the default feature set already includes simd-avx512, whose own documented
requirement is >= 1.89, so the practical default-build floor is set by that pre-existing feature
rather than by anything here. That is the real price of this feature and it is why it is opt-in.

One deliberate behavioural difference

Cosine only. The scalar path treats denom <= 1e-8 as degenerate and returns 1.0; the lattice
kernels test norms against exactly 0.0. Vectors with tiny but nonzero norms therefore get a
computed ratio rather than a saturated 1.0. Exact zero vectors agree on both, and a test covers it.

Verification

cargo test -p ruvector-core --features lattice-simd --lib: 232 passed, 0 failed. Scope
note: the lattice kernels serve the generic distance path (the four public adapters in
distance.rs, used by FlatIndex — the path wasm search actually takes); index/hnsw.rs
dispatches its own simd_intrinsics kernels directly for every metric and is unaffected by
this feature. The default-feature run
stays green, so the SimSIMD path is unaffected.
New test_backend_matches_scalar_reference checks whichever backend compiled in against naive
scalar references across a 13-dimension sample spanning 1 through 768, chosen to straddle
4/8/16-lane widths and their remainders so tail handling is exercised. It is backend-independent, so it now covers the
SimSIMD path too.
Mutation-checked per adapter, not as one patch, since mutating a patch as a unit only
certifies its best line. Dropping the 1.0 - on cosine, the negation on dot, swapping
euclidean for its squared variant, or doubling the Manhattan result each makes the test fail;
reverting each returns it to green. Each mutation also leaves the default-feature run green,
which is what proves it stayed inside its own cfg block instead of breaking the crate outright.

wasm32, measured rather than asserted

A successful build is not evidence that anything vectorized, so the check is now a committed,
fail-closed script rather than a one-off measurement: scripts/check_wasm_simd.sh builds
ruvector-wasm with --features lattice-simd twice — with and without
RUSTFLAGS='-C target-feature=+simd128' — disassembles each artifact with wasm-objdump,
and counts SIMD128 opcode mnemonics. It asserts its prerequisites before building, asserts
the artifact is non-empty before counting (a failed build cannot read as a zero count), and
requires the with-flag count to exceed the control's. Current-head counts:

`simd128` flag	`lattice-simd`	SIMD128 opcodes
on	on	1007
off	on	0

The zero control shows the SIMD128 acceleration is absent: without -C target-feature=+simd128
the lattice kernels take their scalar fallback. That control build is not a no-op of the feature —
the route still goes through lattice-embed, whose scalar fallback carries the exact-zero norm
guard rather than the 1e-8 saturation (the tiny-norm difference documented and tested above). Earlier revisions of this description quoted
474/462/12 from a pre-Manhattan one-off measurement; the committed check's current-head counts
above supersede them.

Not addressed here

ruvector-core does not currently build for wasm32-unknown-unknown on its own under
--no-default-features --features memory-only, because getrandom needs its js feature. This
reproduces on an unmodified checkout with this feature absent, so it is independent of this change.
ruvector-wasm supplies the js-enabled getrandom and builds fine, which is the configuration
measured above.

Manhattan

Manhattan was the one metric that bypassed this split entirely and called
simd_intrinsics directly. That dispatch covers x86_64 and aarch64 and falls through to scalar
everywhere else, wasm32 included, so the module's claim that lattice-simd is the only backend
that vectorizes on wasm was true for three metrics and false for this one.

lattice-embed 0.7.1 adds a runtime-dispatched L1 kernel (AVX-512F, AVX2, NEON, wasm32 SIMD128,
each with a scalar fallback), so Manhattan now joins the lattice-simd override: its two
mutually exclusive arms are lattice-simd and the existing simd_intrinsics dispatch, and the
wasm-vectorization claim holds for all four. The pin moves to 0.7.1 because
simd::manhattan_distance does not exist in 0.7.0, which makes it a build requirement rather
than a tidy-up.

test_backend_matches_scalar_reference covered euclidean, cosine and dot but not Manhattan,
despite documenting itself as covering every metric. It now covers all four, with a scalar L1
reference alongside the existing three. Default builds keep the simd_intrinsics path unchanged.

Touches the same lattice-embed pin as the dependency bump PR, so whichever lands second will need
a one-line rebase.

Measurement

Apple silicon Mac mini, macOS, aarch64, rustc 1.93.0, ruvector-core's own
distance_metrics bench target. Criterion --measurement-time 10.

A two-arm on/off comparison cannot answer the question this PR raises. With the
crate's default features the "off" side is not scalar: default pulls simd,
simd pulls simsimd, so off is SimSIMD. Whether an on/off delta means "this
backend is fast" or "SimSIMD is not vectorising in this build" is not decidable
from two arms, so all three were measured against the same scalar baseline
(--no-default-features --features storage,hnsw,api-embeddings,parallel, the
opt-out documented in Cargo.toml).

group	scalar	SimSIMD (default)	lattice-embed
euclidean/128	18.3 ns	12.91 ns (-29.6%)	7.03 ns (-61.2%)
euclidean/384	50.5 ns	52.49 ns (+3.9%)	18.88 ns (-62.7%)
euclidean/768	102 ns	109.45 ns (+7.5%)	38.57 ns (-62.3%)
euclidean/1536	210 ns	264.33 ns (+26.1%)	82.59 ns (-60.7%)
cosine/128	58.7 ns	15.85 ns (-73.0%)	10.53 ns (-82.1%)
cosine/384	196.6 ns	57.28 ns (-70.8%)	27.51 ns (-85.9%)
cosine/768	421.9 ns	119.24 ns (-71.6%)	53.31 ns (-87.6%)
cosine/1536	848.8 ns	277.89 ns (-67.4%)	96.01 ns (-88.6%)
dot/128	26.55 ns	11.79 ns (-55.7%)	6.65 ns (-75.2%)
dot/384	117.5 ns	51.62 ns (-56.3%)	17.66 ns (-84.9%)
dot/768	296.6 ns	102.27 ns (-65.5%)	37.14 ns (-87.5%)
dot/1536	676.5 ns	247.25 ns (-63.5%)	73.14 ns (-89.2%)
batch 1000x384	100.88 us	72.37 us (-28.3%)	62.42 us (-38.1%)

SimSIMD is working: roughly 3-4x scalar on cosine and 2-3x on dot. So the delta against it
is a real backend-to-backend difference, not a broken control.

One result here is worth separating from this PR entirely, because it is a
property of the current default path and would still be true if this PR were
closed: SimSIMD's euclidean is a net regression against scalar at 384
dimensions and above on aarch64 (+3.9%, +7.5%, +26.1%), and only wins at 128.
Cosine and dot behave normally. Worth a look independently of which backend this
crate ends up preferring.

Conditions, stated rather than certified

The host runs a browser whose CPU draw fluctuates, and idle was sampled every
20s inside each measured arm rather than around it. Per-arm minimum idle: scalar
64%, SimSIMD 33%, lattice 9%. That does not meet a quiet-machine bar and I am
not claiming it does.

What carries the result instead is agreement across independent runs. A separate
two-arm run of the default-vs-lattice comparison, taken earlier under a
different load profile, measured cosine/768 at 50.98 ns, dot/1536 at 75.09 ns
and batch at 62.29 us; this run measured 53.31 ns, 73.14 ns and 62.42 us. Two
runs with different noise agreeing to within a few percent, on effects of 60-89%,
is a stronger argument than either run's idle trace. A reviewer who wants a
certified-quiet run on a dedicated host should ask for one before this is
treated as a benchmark rather than a direction.

What is not measured

This PR's stated value is wasm32, where SimSIMD's call sites are compiled out and the distance
path is otherwise scalar. That claim is unmeasured. No declared bench target
in this repository runs under wasm32, so the wasm32 comparison has no harness to
run in, and nothing above speaks to it. The numbers here are native aarch64 and
answer a different question: whether enabling this backend costs anything on
native. It does not; it is faster on every group measured.

…on wasm32 `ruvector-core`'s f32 distance functions have two backends: SimSIMD on native, and a scalar fallback. SimSIMD is excluded from wasm32, so every browser build takes the scalar path — an unrolled loop for euclidean, a single-pass loop for cosine, and a plain `iter().zip().map().sum()` for dot product. That path is also on the hot line for wasm search, since `hnsw` is unavailable there (mmap), leaving brute-force scans where distance dominates query cost. This adds a third backend behind a new off-by-default `lattice-simd` feature, routing the three f32 functions through `lattice-embed`'s kernels, which do compile to `simd128` on wasm32. `ruvector-wasm` gets a matching passthrough feature next to its existing `simd` one. Precedence is explicit: `lattice-simd` wins where both it and `simd` are enabled, and the three `cfg` arms are mutually exclusive and exhaustive, so exactly one compiles for any feature/target combination. Default builds are untouched and still use SimSIMD. The dependency is now pinned `default-features = false`. That is what makes a kernels-only dependency possible: `native` is the feature that pulls `lattice-inference`, so with defaults off there is no model, tokenizer, or download stack in the tree. Verified with `cargo tree`: the `lattice-simd` build resolves `lattice-embed` and zero `lattice-inference`. `lattice-embeddings` now forwards `native` and `download` explicitly so its behaviour is unchanged. MSRV: `lattice-embed` needs Rust >= 1.93, so enabling either lattice feature raises the effective MSRV for whoever turns it on, in the same shape as the existing `simd-avx512` feature requiring >= 1.89. The default build stays on the workspace 1.77. One deliberate behavioural difference, in cosine only. The scalar path treats a denominator <= 1e-8 as degenerate and returns 1.0; the lattice kernels test the norms against exactly 0.0. Vectors with tiny but nonzero norms therefore get a computed ratio here rather than a saturated 1.0. Exact zero vectors agree, and that agreement is covered by a test. Verification: - `cargo test -p ruvector-core --features lattice-simd --lib`: 230 passed, 0 failed. Default-feature run stays green, so the SimSIMD path is unaffected. - New `test_backend_matches_scalar_reference` checks whichever backend is compiled in against naive scalar references across dimensions 1..768, chosen to straddle 4/8/16-lane widths and their remainders. It is backend-independent, so it now also covers the SimSIMD path. - Mutation-checked per adapter rather than as one patch: dropping the `1.0 -` on cosine, the negation on dot product, or swapping euclidean for its squared variant each makes that test fail, and reverting each returns it to green. - wasm32-unknown-unknown builds of `ruvector-wasm` with `RUSTFLAGS="-C target-feature=+simd128"`. Disassembling the artifact and counting vector opcodes: 474 with the feature on, 12 with it off, and 0 when the feature is on but the target-feature flag is absent. So the kernels are genuinely vectorized rather than merely compiled, and without the flag they fall back to scalar as documented. Not addressed here: `ruvector-core` does not currently build for wasm32-unknown-unknown on its own under `--no-default-features --features memory-only`, because `getrandom` needs its `js` feature. That reproduces on an unmodified checkout with this feature absent, so it is independent of this change; `ruvector-wasm` supplies the `js`-enabled `getrandom` and builds fine. Manhattan distance is left on `simd_intrinsics` and is not part of this change.

Formatting only, no behaviour change. Brings the new test helper in line with the repo's rustfmt settings so the Rustfmt check passes.

Manhattan was the one metric that bypassed the backend split entirely and called `simd_intrinsics` directly. That dispatch covers x86_64 and aarch64 and falls through to scalar everywhere else, wasm32 included, so this module's claim that `lattice-simd` is the only backend that vectorizes on wasm held for three metrics and not for this one. lattice-embed 0.7.1 adds a runtime-dispatched L1 kernel (AVX-512F, AVX2, NEON, wasm32 SIMD128, each with a scalar fallback), so Manhattan now joins the same mutually exclusive cfg split as the other three and the claim is true for all four. The pin moves to 0.7.1 because `simd::manhattan_distance` does not exist in 0.7.0. `test_backend_matches_scalar_reference` covered euclidean, cosine and dot but not Manhattan, despite documenting itself as covering every metric. It now covers all four, with a scalar L1 reference alongside the existing three. Default builds keep the `simd_intrinsics` path unchanged.

The lockfile update that added the lattice-embed pins also moved tempfile's getrandom edge from 0.3.4 to 0.4.3. That bump is unrelated to this change and is not required by it: `cargo check -p ruvector-core --features lattice-simd --locked` resolves cleanly with the edge back on 0.3.4. `rand 0.10.1` keeps its own `getrandom 0.4.3` edge, which predates this branch and is untouched here.

- Document which call sites use the feature-gated distance adapters (generic/FlatIndex path) versus index/hnsw.rs, which dispatches its own kernels directly regardless of the lattice-simd feature. - Add a lattice-simd-gated regression test asserting each of the four public adapters (euclidean, cosine, dot product, manhattan) is bit-exact against lattice_embed's kernels, so a silent fallback to the scalar/SimSIMD path is caught even though the existing tolerance test would still pass. - Add a test documenting the intentional tiny-norm cosine divergence: the scalar path saturates cosine distance at 1.0 below a 1e-8 norm guard, while the lattice-simd kernel only short-circuits on an exactly-zero norm.

The tiny-norm cosine coverage previously called a hand-copied mirror of the scalar branch instead of the compiled production cosine_distance, so a regression in the real branch could leave the mirror green. The scalar assertion now calls cosine_distance itself, gated on the same cfg as its scalar branch, so it only asserts when that branch is the one actually compiled in. Also corrects the comment describing the saturation threshold: it guards the product of the two vector norms (norm_a_sq.sqrt() * norm_b_sq.sqrt()), not an individual norm. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

ruvector-wasm's lattice-simd feature claims to vectorize on wasm32 via lattice-embed's simd128 kernels, but nothing in the repo reproduced that build-time. Add scripts/check_wasm_simd.sh, which builds ruvector-wasm for wasm32-unknown-unknown with and without RUSTFLAGS='-C target-feature=+simd128' (feature lattice-simd enabled in both), disassembles each .wasm with wasm-objdump, and counts SIMD128 opcodes. It fails closed: missing wasm32 target or wasm-objdump aborts before any build runs, a missing or empty .wasm artifact is treated as a failure rather than a zero count, and the with-flag build must both emit at least one SIMD128 opcode and carry strictly more than the without-flag control. On the current head this measures 1007 SIMD128 opcodes with the flag and 0 without it.

check_wasm_simd.sh could exit 0 with `== PASS ==` on a genuinely broken build: `cargo build` ran inside a `$(...)` command substitution where a non-final failing command doesn't reliably abort the script under `errexit`, and because the per-arm target dirs persist across runs, a stale artifact from a prior successful build satisfied the downstream non-empty check. Fix by checking the build's exit status explicitly inside build_artifact (and again at both call sites) and by deleting the expected artifact before each build attempt. Separately, `grep -Ec ... || true` folded a real grep failure (status 2+) into the same bucket as a legitimate zero-match count (status 1), producing a blank count that bash arithmetic silently treats as zero — indistinguishable from the control arm's expected zero-opcode result. Capture grep's real exit status, fail loudly on anything other than 0/1, and validate both opcode counts are numeric before any arithmetic. Also corrects the `lattice-simd` feature comment in crates/ruvector-wasm/Cargo.toml: the simsimd dependency is not excluded on wasm32 (only its call sites are cfg-gated on `not(target_arch = "wasm32")`), and disabling the flag does not make the feature a no-op — distance calls still run through lattice-embed's own scalar fallback, whose exact-zero cosine norm guard differs from ruvector-core's own `denom <= 1e-8` scalar guard. Comment-only change; build graph is unchanged.

check_wasm_simd.sh's build_artifact() deleted the previous artifact with an unchecked `rm -f` before invoking cargo. If rm failed (e.g. EBUSY, a permissions error, or a full/read-only filesystem), the function continued into a cache-fresh cargo build that could exit 0 without rewriting the artifact, leaving the script to disassemble the stale file and report PASS on a build that never actually produced the artifact it was checking. Check the exit status of rm and fail closed: if ! rm -f "$artifact_path"; then echo "FAIL: could not remove stale artifact '$artifact_path'." >&2 return 1 fi Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

- Cargo.toml lattice-simd feature comment: SimSIMD's call sites are gated off on wasm32, not the dependency itself (it still resolves there per cargo tree). Match the accurate phrasing in ruvector-wasm/Cargo.toml. - distance.rs module docs: scope the three-backend description (lattice-simd/simd/scalar) to Euclidean, cosine, and dot. Manhattan uses simd_intrinsics's x86_64/aarch64 dispatch by default and was incorrectly implied to follow the same three-way split.

ohdearquant added 2 commits August 2, 2026 08:34

style(core): rustfmt the distance backend test helper

0db0d72

Formatting only, no behaviour change. Brings the new test helper in line with the repo's rustfmt settings so the Rustfmt check passes.

This was referenced Aug 2, 2026

feat(spann): add an optional lattice-simd distance backend #763

Open

feat(router-core): add an optional lattice-simd distance backend #765

Open

ohdearquant marked this pull request as draft August 2, 2026 14:37

ohdearquant added 2 commits August 2, 2026 12:29

ohdearquant marked this pull request as ready for review August 3, 2026 18:08

ohdearquant and others added 6 commits August 3, 2026 15:07

Navigation Menu

feat(core,wasm): add a lattice-simd distance backend that vectorizes on wasm32 - #762

feat(core,wasm): add a lattice-simd distance backend that vectorizes on wasm32#762
ohdearquant wants to merge 10 commits into
ruvnet:mainfrom
ohdearquant:feat/wasm-simd128-core-distance

ohdearquant commented Aug 2, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

ohdearquant commented Aug 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

The gap

The change

Kernels only, not the model stack

Cost, stated plainly

One deliberate behavioural difference

Verification

wasm32, measured rather than asserted

Not addressed here

Manhattan

Measurement

Conditions, stated rather than certified

What is not measured

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

ohdearquant commented Aug 2, 2026 •

edited

Loading