ggml-quants: vectorize the per-block extremes scan in the reference quantizers

[codspeed-hq]

What

Every legacy reference quantizer starts each block with a scan for its extremes, written as a sequential search:

for (int j = 0; j < qk; j++) {
    const float v = x[i*qk + j];
    if (amax < fabsf(v)) { amax = fabsf(v); max = v; }   // q4_0 / q5_0 / q8_K
}

Each element depends on the two selects of the previous one, so the compiler cannot vectorize it: on x86-64 at -O2 (the level used by RelWithDebInfo and the CI benchmark build) quantize_row_q4_0_ref spends 10 scalar instructions per element in this scan - about 75% of the whole function - while the quantize/pack loop right below it is already auto-vectorized.

This change replaces the scan with a branchless signed min/max reduction over four independent accumulator groups (ggml_block_min_max), which the compiler collapses into minps/maxps: 6 instructions per 4 elements instead of 10 per element. The absolute maximum follows from max(|mn|, |mx|), and the element reaching it from whichever side is larger.

Applied to the q4_0, q4_1, q5_0, q5_1, q8_0, q8_1 and q8_K reference quantizers.

Correctness - bit-identical output

The sequential scan has an observable tie-break: when a block contains both +amax and -amax, the first one becomes the block scale, and the two choices produce different (though equally accurate) encodings. ggml_block_amax therefore detects that exact case (mx == -mn) and falls back to a short scan, so it reproduces the original element for element. Signed zeros are handled through fabsf, so an all-zero block still yields the same d.

Verified in this sandbox with an A/B harness that builds ggml at the base commit and at this commit and hashes the produced blocks byte for byte:

• 2520/2520 comparisons bit-identical, 0 mismatches - 12 quantization types (q4_0, q4_1, q5_0, q5_1, q8_0, q4_K, q5_K, q6_K, q2_K, q3_K, iq4_nl, plus quantize_row_q8_K_ref called directly) x 210 data patterns of 16384 elements each: benchmark data, all zeros, all negative zeros, constant positive/negative, exact +-v ties (both orders, and with the tie late in the block), denormal and 1e30 magnitudes, and 200 randomized rounds sweeping magnitude (1e-10 .. 1e9), sparsity and symmetry.
• tests/test-quantize-fns.cpp passes for all 35 types.

Measured impact (CodSpeed, CPU simulation)

Base run vs head run of the micro suite (benchmarks/bench_ggml.cpp):

Benchmark	Base	Head	Change
quantize_chunk[q4_0]	246.6 µs	139.3 µs	+77.09%
quantize_chunk[q4_1]	213.5 µs	144.7 µs	+47.54%
quantize_chunk[q5_0]	390.5 µs	282.8 µs	+38.08%
quantize_chunk[q8_0]	175.3 µs	135.5 µs	+29.35%

The 11 other benchmarks are unchanged (byte-for-byte identical measurements) - no regressions.

Scope note

This lands on the model-quantization path (llama-quantize / ggml_quantize_chunk), not on the matmul kernels that dominate the walltime macro suite. quantize_row_q8_K_ref is also on the inference path (activation quantization for q4_K/q5_K/q6_K matmuls), but it accounts for only ~0.7% of mul_mat[q4_k], so the gain there (71.72 ms -> 71.58 ms) is below the noise floor and not claimed. The change is architecture-independent: the same scan is scalar on every target, so aarch64 benefits from the same fmin/fmax vectorization.

[codspeed-hq]

Merging this PR will improve performance by 46.71%

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 4 improved benchmarks
✅ 24 untouched benchmarks

Performance Changes

	Mode	Benchmark	BASE	HEAD	Efficiency
⚡	Simulation	quantize_chunk[q4_0]	246.7 µs	140.2 µs	+75.95%
⚡	Simulation	quantize_chunk[q4_1]	213.6 µs	144.8 µs	+47.53%
⚡	Simulation	quantize_chunk[q5_0]	386 µs	279.2 µs	+38.25%
⚡	Simulation	quantize_chunk[q8_0]	175 µs	135.6 µs	+29.1%

Tip

Curious why this is faster? Comment @codspeedbot explain why this is faster on this PR, or directly use the CodSpeed MCP with your agent.

---

_{Comparing codspeed-optim-vectorize-the-per-block-extremes-scan-in-the-refer-1785394478758 (15ee065) with master (46819c9)}