feat(models): add Qwen3.8 Flash Next support

[mrcolo]

feat(models): add Qwen3.8 Flash Next support

Summary

Adds native single-GPU offload serving for the official Qwen/Qwen3.8-Flash-Next-FP8 checkpoint (Qwen4ExpForConditionalGeneration). The implementation covers Qwen4-Exp execution, host-resident PLE and routed experts, checkpoint loading, overlap-safe decoding, and OpenAI-compatible parser selection.

What's included

• Qwen4-Exp runtime: hyper-connections, Gated DeltaNet layers, centered RMSNorm semantics, and exact dense Qwen Sparse Attention within the supported prefix.
• Per-Layer Embedding (PLE): mmap-backed FP8 n-gram tables, per-request convolution state, overlap-safe in-flight token handling, and incremental suffix hashing during decode.
• Optional PLE preload: FREETOKEN_QWEN4_PLE_PRELOAD=1 copies PLE shards into host RAM to avoid mmap page-fault latency on hosts with sufficient memory.
• Checkpoint loading: projection fusion, official 128x128 block-FP8 routed experts, host expert banks, and support for raw Hugging Face configs when installed Transformers predates Qwen4-Exp.
• Runtime constraints: automatically selects the naive cache and disables CUDA graphs because PLE performs host-side gathers and owns request state.
• Expert copy kernels: supports the checkpoint's non-128-byte-aligned 200-byte and 400-byte FP8 scale rows and registers its AOT shapes.
• Server integration: model registration plus automatic Qwen reasoning and qwen3_coder tool-call parser selection.

Context support

The checkpoint natively advertises 262,144 tokens. This integration intentionally caps total sequence length at 2,048 tokens because sparse QSA beyond that point is not implemented. Within the first 2,048 tokens, dense causal attention is exactly equivalent to the checkpoint's QSA selection budget.

Run command

ft serve \
  --model Qwen/Qwen3.8-Flash-Next-FP8 \
  --moe-backend offload \
  --moe-cache-auto \
  --kv-reserve-tokens 2048 \
  --max-seq-len-override 2048

Resource profile

• Official FP8 checkpoint: approximately 173 GiB on disk.
• Routed-expert banks: 122 GB decimal (about 113.6 GiB) of host memory.
• PLE tables: approximately 48 GiB, mmap-backed by default.
• Validated GPU: one NVIDIA RTX A6000 (48 GiB).

Validation

• FREETOKEN_ALLOW_CUDA_MISMATCH=1 uv run pytest -q tests/models/test_qwen4_exp.py tests/models/test_qwen4_exp_raw_config.py tests/kernels/test_pinned_tensor.py tests/server/test_parser_auto_selection.py tests/engine/test_attention_backend_matrix.py — 87 passed, 8 CUDA-only tests skipped on the local host.
• Checked all 152,089 tensors across the official checkpoint's 131 safetensors indexes: 854/854 resident keys covered, with no missing keys, unexpected keys, or shape mismatches.
• Loaded the complete official checkpoint on one RTX A6000 with the offload backend and CUDA 13 runtime.
• Sent sequential OpenAI-compatible multi-token chat completions through the live server; both returned HTTP 200 and exactly A6000_OK, exercising overlap decode beyond the first token.
• Warm smoke request: 65 prompt tokens plus 45 completion tokens in 8.57 seconds end-to-end. The first cold request, including PLE page faults and kernel warmup, took 212 seconds.

Scope

No Dockerfiles, deployment manifests, credentials, or cluster-specific packaging are included.