Skip to content

[Bug][Export]: Qwen3-ASR export can preserve rope_type=linear and disable MRoPE at runtime #142

Description

@suharvest

Describe the bug

Exporting the public Qwen3-ASR checkpoint with TensorRT-Edge-LLM v0.9.1 /
current main can produce a runtime LLM config with:

{
  "rope_scaling": {
    "type": "linear",
    "rope_type": "linear",
    "factor": 1.0,
    "mrope_section": [24, 20, 20]
  }
}

The checkpoint contains MRoPE sections, but the current normalization helper
does not canonicalize linear + mrope_section to MRoPE. The C++ runtime does
not select its MRoPE path for this exported config. The engine can build, but
runtime execution then lacks mropeCosSinOut, producing a silent
configuration/correctness failure instead of a clean export error.

Steps/Code to reproduce bug

Installation method:

Built from source at
7f061f21f0a581ba234a1e233c9315b89d8e47d6.

Model:

Qwen/Qwen3-ASR-0.6B@5eb144179a02acc5e5ba31e748d22b0cf3e303b0

Export flow:

tensorrt-edgellm-quantize llm \
  --model_dir Qwen/Qwen3-ASR-0.6B \
  --output_dir ./qwen3-asr-int4 \
  --quantization int4_awq

tensorrt-edgellm-export llm \
  --model_dir ./qwen3-asr-int4 \
  --output_dir ./qwen3-asr-onnx

Inspect the generated LLM config.json. Both rope_scaling.type and
rope_scaling.rope_type remain linear while mrope_section is present.

Current normalization:

def normalize_rope_scaling_for_runtime(rope_scaling: Any) -> Any:
"""Normalize HF MRoPE / rope_parameters metadata to the shape expected by
the C++ runtime.
"""
if not isinstance(rope_scaling, dict):
return rope_scaling
normalized = dict(rope_scaling)
if "mrope_section" in normalized:
rope_type = normalized.get("type") or normalized.get("rope_type")
if rope_type in (None, "default", "mrope"):
normalized["type"] = "default"
normalized["rope_type"] = "default"
# rope_parameters (transformers v5) carries "rope_type" without "type";
# propagate the alias so Python callers keyed off "type" still work.
# C++ collectRopeConfig() accepts either key.
if "type" not in normalized and "rope_type" in normalized:
normalized["type"] = normalized["rope_type"]
return normalized

Expected behavior

For Qwen3-ASR, an exported rope config containing mrope_section should use
the canonical MRoPE semantics expected by the C++ runtime. The exporter
should preserve all unrelated rope parameters, leave other model families
unchanged, and be idempotent for already-canonical MRoPE configs.

System information (x86 Host with GPU)

  • Container used: source virtual environment
  • OS: WSL2 / Linux
  • CPU architecture: x86_64
  • GPU: NVIDIA GeForce RTX 3060
  • GPU memory: 12 GB
  • Number of GPUs: 1
  • Library versions:
    • Python: 3.12
    • TensorRT Edge-LLM: 7f061f21f0a581ba234a1e233c9315b89d8e47d6
    • CUDA: 13.0 build environment
    • PyTorch: 2.12.0+cu130
    • ModelOpt: 0.44.0

A focused config-level regression test and minimal exporter fix are available
for a follow-up PR after issue approval. Device validation used a freshly
built SM87 engine; only the exported rope config changed.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions