Describe the bug
Exporting the public Qwen3-ASR checkpoint with TensorRT-Edge-LLM v0.9.1 /
current main can produce a runtime LLM config with:
{
"rope_scaling": {
"type": "linear",
"rope_type": "linear",
"factor": 1.0,
"mrope_section": [24, 20, 20]
}
}
The checkpoint contains MRoPE sections, but the current normalization helper
does not canonicalize linear + mrope_section to MRoPE. The C++ runtime does
not select its MRoPE path for this exported config. The engine can build, but
runtime execution then lacks mropeCosSinOut, producing a silent
configuration/correctness failure instead of a clean export error.
Steps/Code to reproduce bug
Installation method:
Built from source at
7f061f21f0a581ba234a1e233c9315b89d8e47d6.
Model:
Qwen/Qwen3-ASR-0.6B@5eb144179a02acc5e5ba31e748d22b0cf3e303b0
Export flow:
tensorrt-edgellm-quantize llm \
--model_dir Qwen/Qwen3-ASR-0.6B \
--output_dir ./qwen3-asr-int4 \
--quantization int4_awq
tensorrt-edgellm-export llm \
--model_dir ./qwen3-asr-int4 \
--output_dir ./qwen3-asr-onnx
Inspect the generated LLM config.json. Both rope_scaling.type and
rope_scaling.rope_type remain linear while mrope_section is present.
Current normalization:
|
def normalize_rope_scaling_for_runtime(rope_scaling: Any) -> Any: |
|
"""Normalize HF MRoPE / rope_parameters metadata to the shape expected by |
|
the C++ runtime. |
|
""" |
|
if not isinstance(rope_scaling, dict): |
|
return rope_scaling |
|
|
|
normalized = dict(rope_scaling) |
|
if "mrope_section" in normalized: |
|
rope_type = normalized.get("type") or normalized.get("rope_type") |
|
if rope_type in (None, "default", "mrope"): |
|
normalized["type"] = "default" |
|
normalized["rope_type"] = "default" |
|
# rope_parameters (transformers v5) carries "rope_type" without "type"; |
|
# propagate the alias so Python callers keyed off "type" still work. |
|
# C++ collectRopeConfig() accepts either key. |
|
if "type" not in normalized and "rope_type" in normalized: |
|
normalized["type"] = normalized["rope_type"] |
|
return normalized |
Expected behavior
For Qwen3-ASR, an exported rope config containing mrope_section should use
the canonical MRoPE semantics expected by the C++ runtime. The exporter
should preserve all unrelated rope parameters, leave other model families
unchanged, and be idempotent for already-canonical MRoPE configs.
System information (x86 Host with GPU)
- Container used: source virtual environment
- OS: WSL2 / Linux
- CPU architecture: x86_64
- GPU: NVIDIA GeForce RTX 3060
- GPU memory: 12 GB
- Number of GPUs: 1
- Library versions:
- Python: 3.12
- TensorRT Edge-LLM:
7f061f21f0a581ba234a1e233c9315b89d8e47d6
- CUDA: 13.0 build environment
- PyTorch: 2.12.0+cu130
- ModelOpt: 0.44.0
A focused config-level regression test and minimal exporter fix are available
for a follow-up PR after issue approval. Device validation used a freshly
built SM87 engine; only the exported rope config changed.
Describe the bug
Exporting the public Qwen3-ASR checkpoint with TensorRT-Edge-LLM
v0.9.1/current
maincan produce a runtime LLM config with:{ "rope_scaling": { "type": "linear", "rope_type": "linear", "factor": 1.0, "mrope_section": [24, 20, 20] } }The checkpoint contains MRoPE sections, but the current normalization helper
does not canonicalize
linear + mrope_sectionto MRoPE. The C++ runtime doesnot select its MRoPE path for this exported config. The engine can build, but
runtime execution then lacks
mropeCosSinOut, producing a silentconfiguration/correctness failure instead of a clean export error.
Steps/Code to reproduce bug
Installation method:
Built from source at
7f061f21f0a581ba234a1e233c9315b89d8e47d6.Model:
Export flow:
Inspect the generated LLM
config.json. Bothrope_scaling.typeandrope_scaling.rope_typeremainlinearwhilemrope_sectionis present.Current normalization:
TensorRT-Edge-LLM/tensorrt_edgellm/checkpoint/checkpoint_utils.py
Lines 54 to 72 in 7f061f2
Expected behavior
For Qwen3-ASR, an exported rope config containing
mrope_sectionshould usethe canonical MRoPE semantics expected by the C++ runtime. The exporter
should preserve all unrelated rope parameters, leave other model families
unchanged, and be idempotent for already-canonical MRoPE configs.
System information (x86 Host with GPU)
7f061f21f0a581ba234a1e233c9315b89d8e47d6A focused config-level regression test and minimal exporter fix are available
for a follow-up PR after issue approval. Device validation used a freshly
built SM87 engine; only the exported rope config changed.