perf(gguf): sm_120 dispatch thresholds + upstream int8-MMA MMQ port (Ornith) - #196

Open

lucaspirola wants to merge 17 commits into

FlashML-org:mainfrom

lucaspirola:ornith-sm120-gguf-mmq-phase2

Commits on Aug 24, 2026

feat(laguna): native GGUF support for poolside Laguna (S/XS), unvalidated e2e

probe
and
claude
committed
d67ebcd
feat(kvcache): 8-bit KV cache (q8_0 / fp8_e4m3) behind --kv-cache-dtype

probe
and
claude
committed
38ea7d9
docs(laguna): correct handover — KV-quant work is committed, not pending

probe
and
claude
committed
a885149
fix(laguna): chunk moe_vec by grid-z/memory, enable Q3_K + IQ2_S; validate XS

probe
and
claude
committed
c20b200
docs(laguna): correct hybrid-backend scope in handover

probe
and
claude
committed
7da0e5d
feat(kvcache): packed signed int4 KV cache behind --kv-cache-dtype int4

probe
and
claude
committed
55c882b
fix(kvcache): round fp8 KV to the e4m3 grid before the native cast

probe
and
claude
committed
1b02ccb
feat(kvcache): 8-bit KV cache (q8_0 / fp8_e4m3) behind --kv-cache-dtype

probe
and
claude
committed
df37035
feat(kvcache): packed signed int4 KV cache behind --kv-cache-dtype int4

probe
and
claude
committed
4281579
fix(kvcache): round fp8 KV to the e4m3 grid before the native cast

probe
and
claude
committed
fbbc342
feat(laguna): support mixed INT4 safetensors
probe
committed
7c26f8f
feat(models): add Nemotron-H and Ornith GGUF support
probe
committed
d49d9aa
Merge feat/int4-kv-cache
probe
committed
b62cdc5

Commits on Aug 25, 2026

perf(attention): accelerate Ornith INT4 long context
probe
committed
88b33b9
fix(ornith): make Q4 Claude Code serving coherent
probe
committed
99324b8
Merge fork/main: Ornith INT4 serving + long-context attention
probe
committed
f7a450a

Commits on Aug 26, 2026

perf(gguf): sm_120 dispatch thresholds + upstream int8-MMA MMQ port

probe
and
claude
committed
242c37a