Skip to content

feature/qwen35 (DeltaNet) rehab checklist before it can merge #235

Description

@michalharakal

feature/qwen35 (5637da3, 5 commits) adds a Qwen3.5 hybrid runtime — Qwen35Runtime.kt (+637) interleaving Gated-DeltaNet (SSM) layers with full-attention layers — plus a Qwen byte-level BPE tokenizer, the Qwen3.5 chat template, and tool-call parsing. ~2,440 insertions across 20 files.

The work is genuinely valuable and shouldn't be lost. Commit 4af7e99 documents a real debugging arc, enumerating 9 specific DeltaNet bugs fixed by cross-referencing llama.cpp and the NVlabs GatedDeltaNet reference (3-way QKV split, channel-major conv1d indexing, delta-rule recurrence, softplus(alpha+dt_bias)*ssm_a decay, per-head L2 norm, Q/K head repetition 16→32, output path order, sigmoid gate). Qwen35RuntimeTest.kt (+365) covers these with 11 unit tests on a tiny synthetic model (dim=16, 4 layers, vocab=8) needing no GGUF — that test suite is the branch's real asset.

But it is not mergeable as-is. Newest commit is 2026-04-11 (~97 days old) and it sits 473 commits behind develop.

Checklist

  • Rebase from 473 behind. Merge-base is 545f06a (2026-04-21).
  • Re-verify against engine 0.36.0. The branch was written against engine 0.19.1 — 17 minor versions back. It has never been built against the current engine.
  • Strip 24 diagnostic printlns. Commit 33efe46 is explicitly titled "add init-time dimensional validation and diagnostic prints". Residue in Qwen35Runtime.kt init includes println("Qwen35Runtime: dim=$dim nHeads=$nHeads ..."), println(" ssmQkvDim=... headRatio=..."), println(" GGUF shapes: attn_qkv=$qkvShape ..."), println(" matmul convention: ..."). Some other printlns are legitimate CLI/demo output — check case by case.
  • Resolve the rename collision. The branch edits llm-inference/llama/.../LlamaWeightLoader.kt (±114); develop has renamed it to DecoderGgufWeightLoader.kt. This is a rename/edit conflict git will not resolve cleanly.
  • tests/smoke/SMOKE-TEST.md is deleted on develop — the branch modifies it (modify/delete conflict).
  • tests/smoke/smoke-models.json and llm-runtime/kllama/.../cli/Main.kt also conflict (content).
  • Also verify transformer-core interplay — the branch predates the transformer-core module entirely (added 7c8e4c8, 2026-07-03).

Notes

  • Conflict surface is small (4 conflicts, 8 overlapping files) — the risk is staleness and the engine gap, not merge mechanics.
  • Purely additive API-wise: zero removed public declarations.
  • Explicitly out of scope for 0.36.1, which ships BGE embeddings + vec2text beam search only.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions