perf(dflash): enable grouped MMID on RDNA

[cheese-cakee]

Summary

Extends the grouped MMID (MUL_MAT_ID) batch gate to AMD RDNA3 and RDNA4 while keeping the HIP path opt-in and default-off. Adds a portable grouped-versus-legacy dispatch and numerical regression test for CUDA and HIP, and wires it into both GPU CI lanes.

Changes

• Extend get_mmvq_mmid_max_batch() to admit RDNA3/RDNA4 for grouped MMID.
• Keep NVIDIA Turing+ enabled by default and AMD HIP gated by DFLASH_MMID_GROUPED=1.
• Exercise Q4_K, Q6_K, Q4_0, Q8_0, and Q5_K at widths 2, 4, 8, 9, and 16.
• Derive the legacy route from telemetry instead of hardcoding architecture-specific MMVQ ceilings.
• Require grouped dispatch for all 25 cases.
• Compare every case numerically: byte-exact when legacy also used MMVQ, and against the legacy MMQ result with the repository's existing MMQ NMSE tolerance for newly rerouted cases.
• Support Windows and POSIX child-process invocation and skip unsupported GPUs through CTest's skip return code.
• Build and run the regression in the NVIDIA GPU and gfx1151 ROCm CI jobs.

How it works

The grouped kernel sorts (token, slot) pairs by routed expert so each unique expert weight stream is consumed once per MoE batch. The gate returns MMVQ_MAX_MOE_BATCH_SIZE when grouped dispatch is enabled, the quant type is supported, and the device is NVIDIA Turing+ or AMD RDNA3/RDNA4. CUDA graphs remain available for these batches.

The regression runs identical deterministic inputs in separate legacy and grouped child processes. Telemetry verifies the selected route. All 25 output slices are checked: shared MMVQ cases must be byte-identical, while cases newly moved from MMQ to grouped MMVQ use the legacy MMQ output as an independent numerical oracle.

Local verification

• Hardware: RTX 4050 Laptop GPU (sm_89), WSL2 Ubuntu, CUDA 12.6
• Build: cmake -B server/build-cuda-mmid -S server -DGGML_CUDA=ON -DGGML_CUDA_GRAPHS=ON -DGGML_CUDA_FA=OFF -DDFLASH27B_TESTS=ON -DCMAKE_CUDA_ARCHITECTURES=89
• Test: ctest --test-dir server/build-cuda-mmid --output-on-failure -R '^deepseek4_mmid_grouped_cuda$'
• Result: legacy_status=0 grouped_status=0 bytes=798720 compared_cases=25 exact_cases=25 tolerant_cases=0 compared_bytes=798720 legacy_grouped=0 grouped_grouped=25 parity=PASS
• git diff --check passes.

Limitations

• HIP correctness and performance have not been measured locally because this machine has no ROCm GPU. The HIP path therefore remains opt-in and default-off.
• The gfx1151 workflow will provide target-hardware correctness coverage after a maintainer approves the fork workflow run.
• Grouped dispatch covers Q4_K, Q6_K, Q4_0, Q8_0, and Q5_K. DS4 ROCmFP2/FP3 specializations are outside this change.

[cubic-dev-ai]

All reported issues were addressed across 3 files

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>