Skip to content

Q/DQ and QONNX parsing, GPU INT8 and FP8 GEMM/MatMul - #44

Open
shaunlee8 wants to merge 7 commits into
gpu/alpakafrom
quantize
Open

Q/DQ and QONNX parsing, GPU INT8 and FP8 GEMM/MatMul#44
shaunlee8 wants to merge 7 commits into
gpu/alpakafrom
quantize

Conversation

@shaunlee8

@shaunlee8 shaunlee8 commented Jun 30, 2026

Copy link
Copy Markdown
Collaborator

First-class quantization and low-precision pipeline for SOFIE. Covers GEMM/MatMul operators from Q/DQ ONNX and QONNX graphs. Detects GEMM/MatMul regions by inspecting local Q nodes after the base graph is built.

  • Support for QONNX parsing and Q/DQ ONNX parsing.
  • Simple CPU lowering path for quantized GEMM added.
  • Per-tensor and per-channel weight quantization support.
  • Generate optimized INT8 and FP8 GEMM and MatMul calls through cuBLASLt.
  • cuBLASLt exact, padded, flattened tensor shapes supported.
  • External binary weight storage for INT8/FP8.
  • Test coverage for QONNX, Q/DQ, codegen, and GEMM/MatMul execution.

@shaunlee8 shaunlee8 changed the title feat: QONNX parsing and basic lowering feat: QONNX parsing and CPU/GPU lowering Jul 2, 2026
@shaunlee8
shaunlee8 marked this pull request as ready for review July 6, 2026 08:31
@shaunlee8 shaunlee8 changed the title feat: QONNX parsing and CPU/GPU lowering Q/DQ and QONNX parsing, GPU INT8 and FP8 GEMM/MatMul Jul 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant