ggml-cpu: add 2x2 register-blocked AVX2 kernel for ggml_vec_dot_q8_0_q8_0

[codspeed-hq]

What

ggml_compute_forward_mul_mat() calls the vec_dot kernel once per (src0 row, src1 column) pair. For Q8_0 weights on x86 the kernel was always invoked with nrc == 1, so a 2x2 output tile costs four independent dot products: every Q8_0 block of both operands is loaded twice, every block scale is converted from FP16 twice, and the absolute value of each src0 block (needed to feed the unsigned side of _mm256_maddubs_epi16) is computed twice.

The prompt_layer[q8_0] profile shows exactly this: ggml_vec_dot_q8_0_q8_0 dominates the benchmark, and inside it the block loads and the FP16 scale conversions are the largest single items after the dot product itself.

How

• Added an nrc == 2 path to the AVX2 ggml_vec_dot_q8_0_q8_0 that computes the whole 2x2 tile (two src0 rows x two src1 columns) in a single pass over the blocks: 4 block loads, 4 scale conversions and 2 vpsignb (abs) ops per tile instead of 8, 8 and 4, with all four accumulators kept in registers.
• Enabled .nrows = 2 for GGML_TYPE_Q8_0 under __AVX2__ (it was already 2 for ARM i8mm, whose kernel has its own nrc == 2 path), so the mat-mul driver requests the tile.
• The nrc == 1 path, the AVX/SSE paths and the scalar fallback are untouched. The driver already falls back to nrc = 1 when the tile does not fit the shapes, and mul_mat_id / flash-attention always call with nrc = 1.

Correctness

Each accumulator sums the same per-block products in the same order as the 1x1 path, so results are bit-identical:

Check	Result
2x2 tile vs 4 independent 1x1 dot products, n = 32 / 64 / 256 / 2048 / 5632	bit-identical (memcmp)
Full Q8_0 mul_mat output (4 threads), shapes 256x64x8, 256x63x7, 2048x512x64, 512x66x6, 64x2048x1, 5632x130x3	byte-for-byte equal to a build without this change (36555 floats, cmp)
test-quantize-fns, test-quantize-perf, test-backend-ops -o MUL_MAT	pass

The nrc == 2 branch was also instrumented to confirm the mat-mul driver actually reaches it (7736 tile calls across the shapes above).

Compile-checked on -msse4.2, -msse4.2 -mavx -mf16c -mfma, -mavx2, -march=skylake-avx512 and -march=sapphirerapids (no new warnings).

Measured effect

CodSpeed simulation run of the macro benchmarks on an AVX2 host, baseline vs this change:

Benchmark	Base	Head	Change
prompt_layer[q8_0]	1,013.5 ms	733.3 ms	+38.22%
lm_head[q8_0]	171.5 ms	126.3 ms	+35.75%
decode_layer[q8_0]	170.4 ms	170.8 ms	unchanged
prompt_layer[q4_k], decode_layer[q4_k], decode_deep[q4_k], lm_head[q4_k]			unchanged

decode_layer[q8_0] is a single-token GEMV, where the driver keeps nrc = 1, so it is unchanged as expected. The non-Q8_0 benchmarks are untouched.

Note on CI coverage

This is an AVX2-only change, and neither CodSpeed job in this repository can observe it:

• the simulation job runs bench_ggml.cpp, whose mul_mat micro benchmarks only cover f32, q4_0 and q4_k - there is no Q8_0 mat-mul micro benchmark;
• the walltime macro job runs on aarch64 macro runners (the flamegraph resolves to ggml-cpu/arch/arm/quants.c), where Q8_0 already gets a 2x2 mmla kernel when i8mm is available.

So the CodSpeed report on this PR is expected to show no change. The numbers above come from a local CodSpeed simulation run of ggml-macro-benchmarks on x86 (the same instrument as the CI simulation job, just pointed at the macro suite). Walltime validation could not be done locally: the sandbox lacks the linux-tools perf packages the walltime instrument requires.

[codspeed-hq]

Merging this PR will not alter performance

✅ 28 untouched benchmarks

---

_{Comparing codspeed-optim-2-2-register-blocked-avx2-kernel-for-ggml-vec-dot-1785420589008 (71a9df9) with master (46819c9)}