[Fix] conv3d fp8: guard padded M rows in the epilogue store

[jiacao-amd]

Problem

tests/kernels/test_conv3d_implicit_fp8.py::test_conv3d_fp8_vs_fp8cast_reference[1-96-4-8-9-96-1-1] fails intermittently on linux-flydsl-mi355-1, roughly once every few days. Two observed instances:

• https://github.com/ROCm/FlyDSL/actions/runs/31016862619/job/92345704976
• https://github.com/ROCm/FlyDSL/actions/runs/30984871048/job/92239635970

Both reported the identical rel_err 3.075e-01 (0.3075297176837921), which ruled out a plain data race on the accumulators and pointed at a deterministic set of clobbered output elements.

Root cause

The FP8 epilogue masks stores on the column bound only:

buffer_ops.buffer_store(out.to(fx.BFloat16), y_rsrc, off_ncdhw, mask=col_valid)

For n == 1 the output address fast path is off = col * dhw + row. The M rows added to pad npq up to a whole TILE_M are not out of bounds under that mapping — when dhw == npq they alias real elements of the next column:

col=0, row=288  ->  off = 288  ==  col=1, row=0

So they are perfectly legal addresses, not something the hardware OOB check suppresses. Those padded rows carry a zero accumulator (A is zeroed by the im2col mask), so they race legitimate stores issued by another block and silently zero part of the output. Whichever block writes last wins — hence the intermittency.

For shape (1,96,4,8,9,96) with padding=1: npq=288, TILE_M=128 → grid_m=3, 96 padded rows, and dhw == npq == 288, making 32.99% of the output clobberable. 3.075e-01 is the deterministic error when every clobberable element loses the race.

Fix

Add the row bound to the store mask, mirroring what the BF16 kernel and the FP8 split-K path already do.

Verification

On gfx950 (MI355X), same shape, 200 iterations:

	failures	worst rel_err
before	12/200	3.075e-01
after	0/200	1.11e-05

The set of zeroed offsets was checked against the predicted padded-row alias set and matched exactly (9120 offsets, all such offsets in predicted set: True).

tests/kernels/test_conv3d_implicit_fp8.py + tests/kernels/test_conv3d_implicit.py: 33 passed.

I swept all 7 parametrized shapes in the test file — only this one triggers the bug. The others either have padrows == 0 or dhw != npq, so the padded rows do not alias into the live region. The exposure is narrow, but when it hits it silently corrupts output rather than failing loudly.

Notes

• The BF16 kernel is unaffected: kernels/conv/conv3d_implicit.py already guards rows via _row_chk = npq % TILE_M != 0.
• The FP8 split-K path already checks row < npq for its atomics ("Atomics ignore hardware OOB suppression; guard explicitly"). Only the non-atomic FP8 store was missing the equivalent check.

[jiacao-amd]

Perf impact: none on aligned shapes, negligible on unaligned

The first version added the row compare unconditionally. Pushed a follow-up that gates it on row_chk = npq % TILE_M != 0, matching _row_chk in the BF16 kernel — shapes with no padded rows now generate no extra instruction at all.

Verified at the ISA level rather than by wall-clock, because this box has all 8 GPUs pegged at 100% by other tenants and run-to-run variance on an identical binary (82–290 us on the same shape) swamped any real signal. Instruction-level comparison is the trustworthy measurement here.

Aligned shape (npq=768, row_chk=False) — dumped 21_final_isa.s before vs after:

IDENTICAL ISA  (1767 instructions both)

Byte-for-byte identical, so aligned shapes pay exactly nothing.

Unaligned shape (npq=288, row_chk=True) — the guard is active:

metric	before	after
v_mfma	168	168
buffer_store_short	32	32
ds_read	252	252
v_cmp	94	96
v_cndmask	81	81
s_barrier	21	21
VGPR / SGPR	120 / 60	120 / 60
spills	0 / 0	0 / 0

The cost is 2 v_cmp in the epilogue. The math loop is untouched, occupancy is unchanged (same VGPR/SGPR, no spills), and the remaining textual diff is scheduler reordering.

Correctness re-confirmed after the change: 0/200 failures, worst rel_err 1.11e-05. 33 conv3d tests pass.