ggml: add support for MXFP8 CPU

[michaelw9999]

Overview

This PR adds MXFP8 support (CPU only) and includes accelerated x86, ARM, and generic fallback kernels.
Blackwell specific CUDA kernels with native MXFP8 support are ready and to immediately follow.
Support for converting existing Huggingface MXFP8 checkpoints to GGML is included.

MXFP8 is a OCP MX block scaled quantization format at 8.25bpw; native hardware acceleration is currently supported on Blackwell.
It uses one FP8 (E4M3) scale per 32 weights.

The block_mxfp8 form used in this implementation has a block size of 256 with a subb;ock size of 32.

#define QK_MXFP8 256
#define QK_MXFP8_SUB 32
typedef struct {
    uint8_t qs[QK_MXFP8 / QK_MXFP8_SUB][QK_MXFP8_SUB];
    uint8_t e[QK_MXFP8 / QK_MXFP8_SUB];
} block_mxfp8;

This requires no padding and is perfectly aligned for GPU kernels.

Additional information

Significant testing and evaluation of MXFP8 with FP8 was described in #25336 .
Testing showed the best performance and quality came from MXFP8 standalone as its own native format.

The use case justification for bringing in MXFP8 into llama.cpp:

• Some existing checkpoints already exist on Huggingface and can be converted [The quantizer was left out of this initial PR]
• MXFP8 is a universal, open format that likely will get future hardware blockscaling acceleration (or even further accelerated) on other upcoming platforms
• Initial POC CUDA testing demonstrated that MXFP8 had faster prefill than Q8 or BF16, with nearly matched tg. Further optimization may push this ahead. MXFP8 is an excellent option for creating Blackwell-specific model quantizations using mixed NVFP4/MXFP8 layers for overall best performance and quality

Quality

The results below are from Qwen3.5-4B Base using WikiText-2, using full KLD/PPL evaluation against complete BF16 logits.

Format	PPL	PPL(Q) / PPL(base)	Mean KLD	RMS Δp	Same top token
BF16	8.348271	1.0	—	—	—
Q8_0	8.352002	1.000447	0.000845	0.836 %	97.814 %
MXFP8	8.427964	1.009546	0.009009	2.613 %	94.110 %

Performance

CPU results are for only a basic CPU implementation included in this initial PR. A higher performing implementation is possible, but is not trivial and would expand this PR, so can be added at a later time.

Type	Size	pp512	tg128
BF16	7.84 GiB	51.46	4.42
Q8_0	4.16 GiB	65.69	7.01
MXFP8	4.04 GiB	9.58	4.35

Example MXFP8 GGUF models

• Qwen3.5-0.8B MXFP8
• Qwen3.5-4B Base MXFP8
• Qwen3.5-4B Instruct MXFP8
• Qwen3.5-9B Instruct MXFP8

Requirements

• I have read and agree with the contributing guidelines: Yes
• AI usage disclosure:
  Yes, AI was used to optimize and generate the CPU kernels. The generated code was exhaustively pruned and hand edited to be concise, minimal, and match llama.cpp style and standards. Verifying and validating the model outputs were done manually with llama-cli and llama-server.