Skip to content

feat(ascend): add MiniMax-H3 fused RoPE and MindIE SD compile backend - #1471

Open
blian6 wants to merge 1 commit into
ModelTC:mainfrom
blian6:main
Open

feat(ascend): add MiniMax-H3 fused RoPE and MindIE SD compile backend#1471
blian6 wants to merge 1 commit into
ModelTC:mainfrom
blian6:main

Conversation

@blian6

@blian6 blian6 commented Sep 1, 2026

Copy link
Copy Markdown

Summary

Enable an optional MindIE-SD compile backend and a fused partial split-half RoPE for MiniMax-H3 on Ascend NPU (USP4). The MindIE compile fuses the remaining DiTBlock ops (adaln / swiglu / residual-gate) on top of the runtime rms/rope fusion, and keeping the Ulysses A2A on the eager path removes an HCCL alltoallv regression. Kernel-level (CANN profiler, rank0): kernel count -15.3%, kernel time -9.9%, communication -35%.

Performance (kernel-level)

Measured with CANN profiler (warmup 5 + 1 profiled step, rank0, 768p t2av, USP4, 4x Ascend950PR). Compile vs the runtime-fused config (npu_rms_norm + minimax_h3_npu_rope, no compile):

metric (5s) runtime-fused MindIE compile change
kernel count 3600 3050 -15.3%
kernel time 3968ms 3576ms -9.9%
communication 962ms 621ms -35%

The DiTBlock op chains are replaced by three fused kernels, and the Ulysses A2A regression is removed:

  • new fused kernels: swiglu 50x32ms, gather_scale_shift 200x19ms, gather_residual_gate 100x11ms
  • eliminated op chains: Mul -90%, Add -77%, Silu -99%, IndexSelect -87%
  • hcom_alltoallv eliminated: tracing dist.all_to_all_single infers split_sizes=[1,1,...] and degrades HCCL to the variable-length alltoallv path (200-270ms/call); with the collective kept on the eager path it stays on the fixed-size alltoall and overlaps better with the compiled compute (-35% communication)

15s: swiglu 93.7ms + gather_scale_shift 55.5ms + gather_residual_gate 32.3ms replace the 99.4ms old Silu/split/Add/Mul chain; kernel count -550 (-15.3%), kernel time -333ms (-0.7%). Communication dominates the 15s kernel time (57.6%, 28002ms/48584ms) — a Ulysses-level bottleneck that grows linearly with sequence length and is outside compile's reach.

Known residual: the gate-msa residual (residual + gate * attn_out) spans a graph-break boundary (its index_select lands in subgraph 1, the attention output in subgraph 6) and cannot be fused by a single pattern; a dedicated 2D residual-gate kernel was prototyped and reverted (its contiguous-copy and launch overhead outweighed the ~15ms/step fusion saving). All other DiTBlock ops are fused.

Changes

  • Add minimax_h3_npu_rope (Ascend): fuses the 96/128 rotate-half rotary part through MindIE-SD rotary_position_embedding (input guards live in mindiesd); falls back to TorchRealRope when mindiesd is absent
  • BaseTransformerInfer: support compile_backend ("default" | "mindie") and compile_dynamic config keys; reuse ONE MindieSDBackend instance to avoid BACKEND_MATCH recompilation and silent eager fallback
  • Keep TorchUlyssesA2A.exchange out of the compiled graph (torch._dynamo.disable) so HCCL keeps the fast fixed-size alltoall. This changes compiled-graph behavior for all Ulysses + compile users across platforms (the collective now runs eager)
  • Add ascend_npu configs: minimax_h3_t2av_sp_compile_5s.json, minimax_h3_t2av_sp_compile_15s.json

Usage

mindiesd is an optional out-of-tree dependency. Without it, both the compile backend and the fused rope degrade gracefully to the default torch.compile / TorchRealRope paths. Enable the feature in the model config:

"rope_type": "minimax_h3_npu_rope",
"use_compile": true,
"compile_backend": "mindie"

Verification

  • mindiesd op path numerically matches rotate-half semantics (bf16, within 1 ULP of the fp32 reference); no-mindiesd fallback equals TorchRealRope bitwise
  • ruff check and ruff format --check pass on all changed files

## Summary

Enable an optional MindIE-SD compile backend and a fused partial
split-half RoPE for MiniMax-H3 on Ascend NPU (USP4). On a 5s t2av 768p
run this measures -12%~-15% wall-clock vs the torch-native baseline
(rank0): rms/rope runtime fusion plus MindIE compile-side adaln/swiglu/
gate fusion, and Ulysses A2A kept on the eager path.

## Changes

- Add `minimax_h3_npu_rope`: fuses the 96/128 rotate-half rotary part
  through MindIE-SD `rotary_position_embedding` (input guards live in
  mindiesd); falls back to `TorchRealRope` when mindiesd is absent
- `BaseTransformerInfer`: support `compile_backend` ("default"|"mindie")
  and `compile_dynamic` config keys; reuse ONE MindieSDBackend instance
  to avoid BACKEND_MATCH recompilation and silent eager fallback
- Keep `TorchUlyssesA2A.exchange` out of the compiled graph
  (`torch._dynamo.disable`) so HCCL keeps the fast fixed-size alltoall.
  This changes compiled-graph behavior for all Ulysses + compile users
  across platforms (the collective now runs eager)
- Add ascend_npu configs: `minimax_h3_t2av_sp_compile_5s.json`,
  `minimax_h3_t2av_sp_compile_15s.json`

## Usage

mindiesd is an optional out-of-tree dependency. Without it, both the
compile backend and the fused rope degrade gracefully to the default
torch.compile / TorchRealRope paths.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant