rocm: fix build on gfx1201 (RDNA4, R9700) — gate the WMMA v1 kernel#599
Open
cm999club wants to merge 1 commit into
Open
rocm: fix build on gfx1201 (RDNA4, R9700) — gate the WMMA v1 kernel#599cm999club wants to merge 1 commit into
cm999club wants to merge 1 commit into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #521.
Problem
Building on an AMD Radeon AI PRO R9700 (gfx1201, RDNA4 discrete GPU) fails
with:
matmul_q8_0_f32_batch_wmma_4w_kernelinrocm/ds4_rocm_q8.cuhuses__builtin_amdgcn_wmma_f32_16x16x16_f16_w32, which is the RDNA3 (WMMA v1)builtin. RDNA4 (gfx12xx) uses a different WMMA v2 intrinsic, so the compiler
can't select the RDNA3 builtin for a gfx1201 target and the build aborts.
Fix
This is a compile/runtime workaround, not an RDNA4 WMMA v2
implementation:
ds4_rocm_runtime.cuh: atds4_gpu_init(), detect the arch fromprop.gcnArchNameand set a newg_wmma_v1flag (0 on anygfx12*target, 1 otherwise).
ds4_rocm_q8.cuh: guard the RDNA3 WMMA v1 builtin calls insidematmul_q8_0_f32_batch_wmma_4w_kernelwith#if !defined(__gfx1200__) && !defined(__gfx1201__), so the kernel stillcompiles for gfx12xx targets (the guarded body becomes a no-op there)
instead of failing at builtin selection.
ds4_rocm_matmul.cuh: gate dispatch into that kernel ong_wmma_v1, sogfx12xx never calls into the no-op kernel at runtime — it falls back to
the existing non-WMMA matmul path instead.
Net effect: the build succeeds on gfx1201, and inference runs correctly,
but the accelerated WMMA-4w kernel is not used there — gfx12xx runs on the
existing fallback path. A real RDNA4 WMMA v2 kernel would recover that
performance; this PR intentionally scopes to "unbreak the build and run
correctly" rather than attempting the v2 intrinsic without hardware to
validate the accumulator/layout semantics against.
Testing
Running in production on the reported hardware (R9700, 32GB VRAM, ROCm
7.2.2) for two weeks with
--rocm --ssd-streaming, serving DeepSeek V4Flash inference daily without correctness issues.
Happy to adjust the detection (e.g. a proper
HIP_ARCH/target macro insteadof the
gcnArchNamesubstring check) if there's a preferred convention inthe codebase — I didn't see one, so I went with what's inspectable at
runtime.