michaelw9999/Qwen3.5-4B-MXFP6-GGUF overview

---

base_model:

• Qwen/Qwen3.5-4B

---

Changed GGML TYPE to 50, latest mxfp6 build is required

This model is a proof of concept example of MXFP6.<BR>It was quantized with my

<B>still experimental</B> <A HREF="https://github.com/michaelw9999/advanced-gguf-quantizer">advanced-gguf-quantizer</A> tool.<BR>

The CPU version of my llama.cpp PR is at<BR>

https://github.com/ggml-org/llama.cpp/pull/22671

That PR is currently CPU only, and it is very slow, but is the first step.<BR>

The fast CUDA version is available on my mxfp6-cuda fork.

MXFP6 has superior accuracy to NVFP4 and I am working on getting it "almost as fast".

The latest update just posted has the following results:

====== Perplexity statistics ======
Mean PPL(Q)                   :   9.998305 ±   0.071790
Mean PPL(base)                :   9.943929 ±   0.071459
Cor(ln(PPL(Q)), ln(PPL(base))):  99.73%
Mean ln(PPL(Q)/PPL(base))     :   0.005453 ±   0.000528
Mean PPL(Q)/PPL(base)         :   1.005468 ±   0.000531
Mean PPL(Q)-PPL(base)         :   0.054377 ±   0.005275

====== KL divergence statistics ======
Mean    KLD:   0.015374 ±   0.000300
Maximum KLD:  19.809378
99.9%   KLD:   0.629969
99.0%   KLD:   0.094831
95.0%   KLD:   0.037901
90.0%   KLD:   0.026535
Median  KLD:   0.008529
10.0%   KLD:   0.000474
 5.0%   KLD:   0.000135
 1.0%   KLD:   0.000017
 0.1%   KLD:   0.000001
Minimum KLD:  -0.000077

====== Token probability statistics ======
Mean    Δp: -0.174 ± 0.009 %
Maximum Δp: 94.303%
99.9%   Δp: 20.150%
99.0%   Δp:  8.865%
95.0%   Δp:  4.176%
90.0%   Δp:  2.423%
75.0%   Δp:  0.462%
Median  Δp: -0.003%
25.0%   Δp: -0.753%
10.0%   Δp: -3.020%
 5.0%   Δp: -4.931%
 1.0%   Δp: -9.745%
 0.1%   Δp: -24.106%
Minimum Δp: -99.902%
RMS Δp    :  3.437 ± 0.044 %
Same top p: 93.487 ± 0.064 %