Skip to content

Add QDQ build feature - #461

Open
justinchuby with Copilot wants to merge 2 commits into
mainfrom
copilot/feature-validate-qdq-with-gemma-4
Open

Add QDQ build feature#461
justinchuby with Copilot wants to merge 2 commits into
mainfrom
copilot/feature-validate-qdq-with-gemma-4

Conversation

Copilot AI commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

Adds an opt-in build feature for emitting QDQ operators when validating quantized Gemma 4-style exports. --features qdq forces quantized MatMulNBits weights through the existing standard ONNX QDQ lowering path.

  • CLI feature

    • Registers qdq in the cargo-style --features set.
    • Passes the toggle through mobius build.
  • Build / optimization plumbing

    • Adds qdq to build() and build_from_module().
    • Updates optimize_model() so com.microsoft::MatMulNBits is inlined to DequantizeLinear + MatMul when requested, independent of EP defaults.
  • Docs and coverage

    • Documents qdq in README and CLI reference.
    • Adds focused tests for CLI pass-through and CPU-side QDQ lowering.
mobius build --model google/gemma-4-12B out/ \
  --features qdq

Copilot AI linked an issue Aug 7, 2026 that may be closed by this pull request
Co-authored-by: justinchuby <11205048+justinchuby@users.noreply.github.com>
@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

Copilot AI changed the title [WIP] Validate QDQ with gemma 4 Add QDQ build feature Aug 7, 2026
Copilot AI requested a review from justinchuby August 7, 2026 05:18
@justinchuby
justinchuby requested a lite review from Copilot August 7, 2026 05:50
@justinchuby
justinchuby marked this pull request as ready for review August 7, 2026 05:50
@justinchuby
justinchuby requested a review from a team August 7, 2026 05:50
@github-actions

github-actions Bot commented Aug 7, 2026

Copy link
Copy Markdown

🏗️ Architecture Diff

Comparing 95671443c9c4b4

Model Sub-model Changes Status

No architecture changes detected.


Legend: ⚪ No change · 🔵 Minor (attrs/inits) · 🟡 Moderate (nodes added/removed) · 🔴 Major (interface changed)

@github-actions

github-actions Bot commented Aug 7, 2026

Copy link
Copy Markdown

Performance Comparison

Comparing 95671443c9c4b4

Model Metric Baseline Current Delta
bert (feature-extraction) model_size_bytes 359 KB 359 KB +0.0%
bert (feature-extraction) num_nodes 60 60 +0.0%
falcon model_size_bytes 364 KB 364 KB +0.0%
falcon num_nodes 68 68 +0.0%
gemma2 model_size_bytes 428 KB 428 KB +0.0%
gemma2 num_nodes 107 107 +0.0%
gpt2 model_size_bytes 388 KB 388 KB +0.0%
gpt2 num_nodes 54 54 +0.0%
llama model_size_bytes 425 KB 425 KB +0.0%
llama num_nodes 62 62 +0.0%
llama (static-cache) model_size_bytes 425 KB 425 KB +0.0%
llama (static-cache) num_nodes 58 58 +0.0%
mamba (ssm-text-generation) model_size_bytes 296 KB 296 KB +0.0%
mamba (ssm-text-generation) num_nodes 98 98 +0.0%
phi3 model_size_bytes 421 KB 421 KB +0.0%
phi3 num_nodes 60 60 +0.0%
phi3 (static-cache) model_size_bytes 421 KB 421 KB +0.0%
phi3 (static-cache) num_nodes 56 56 +0.0%
qwen2 model_size_bytes 425 KB 425 KB +0.0%
qwen2 num_nodes 62 62 +0.0%
qwen2 (static-cache) model_size_bytes 425 KB 425 KB +0.0%
qwen2 (static-cache) num_nodes 58 58 +0.0%
qwen3_5_moe (hybrid-text-generation) model_size_bytes 506 KB 506 KB +0.0%
qwen3_5_moe (hybrid-text-generation) num_nodes 275 275 +0.0%
qwen3_5_text (hybrid-text-generation) model_size_bytes 458 KB 458 KB +0.0%
qwen3_5_text (hybrid-text-generation) num_nodes 129 129 +0.0%
qwen3_5_vl (hybrid-qwen-vl) model_size_bytes 977 KB 977 KB +0.0%
qwen3_5_vl (hybrid-qwen-vl) num_nodes 431 431 +0.0%
t5 (seq2seq) model_size_bytes 836 KB 836 KB +0.0%
t5 (seq2seq) num_nodes 166 166 +0.0%
whisper (speech-to-text) model_size_bytes 1008 KB 1008 KB +0.0%
whisper (speech-to-text) num_nodes 128 128 +0.0%

No performance regressions.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Validate QDQ with gemma 4

3 participants