Skip to content

speed-bench: add M5 Max (Q2+Q4) results#587

Open
thomasjohnflaherty wants to merge 2 commits into
antirez:mainfrom
thomasjohnflaherty:speed-bench-m5-max
Open

speed-bench: add M5 Max (Q2+Q4) results#587
thomasjohnflaherty wants to merge 2 commits into
antirez:mainfrom
thomasjohnflaherty:speed-bench-m5-max

Conversation

@thomasjohnflaherty

Copy link
Copy Markdown

Benchmark results for Apple M5 Max (128 GB unified memory), Metal backend, mixed Q2+Q4 DeepSeek V4 Flash GGUF, at commit efdadd4.

Canonical sweep from the speed-bench README: ctx 2048..65536, step 2048, 128 greedy decode tokens per frontier. Headlines: 485 t/s prefill / 39 t/s gen @2k, 238 t/s prefill / 21.5 t/s gen @65k.

CSV is in the current ds4-bench output format (includes the gen_first_ms / steady-state columns); plot_speed.py renders it as-is.

Apple M5 Max, 128 GB unified memory, macOS (Metal backend), mixed
Q2+Q4 DeepSeek V4 Flash GGUF. Canonical sweep from the README:
ctx 2048..65536 step 2048, 128 greedy decode tokens per frontier.
…nt vs antirez#584

Renames the M5 Max CSV/SVG to encode the Q2+Q4 quant mix so this run can coexist
with the pure-IQ2XXS M5 Max results in antirez#584 rather than colliding on the shared
m5_max.* filename. Content unchanged; SVG name follows the plot_speed.py _ts.svg
convention derived from the CSV basename.
@thomasjohnflaherty thomasjohnflaherty changed the title speed-bench: add M5 Max results speed-bench: add M5 Max (Q2+Q4) results Jul 22, 2026
@thomasjohnflaherty

Copy link
Copy Markdown
Author

Heads-up that this overlaps with #584, which also adds M5 Max results at the same two paths (speed-bench/m5_max.csv and m5_max_ts.svg), so the two would collide on identical filenames.

They're not redundant, though: #584 is pure IQ2XXS full-residency, this PR is a mixed Q2+Q4 GGUF. The kvcache_bytes column is identical across both, so KV precision and the ctx 2048→65536 sweep are apples-to-apples — the divergence is purely the weight quant. The two curves cross: this run has higher prefill throughout (485 vs 340 t/s @2k), while #584's decode pulls ahead past ~32k. Having both quant points on the same machine class seems useful.

I've renamed mine to m5_max_q2q4.csv / m5_max_q2q4_ts.svg so both merge without collision; #584 can keep m5_max.*. Happy to fold into a single combined file with a quant column instead if you'd prefer.

Methodology note for comparability: these are steady-state numbers (the gen_steady_tps column) with a warm first token (gen_first_ms ~28 ms @2k), all greedy, --gen-tokens 128, at efdadd4 (current upstream main).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant