---

base_model:

• Qwen/Qwen3.6-27B

tags:

• llama.cpp
• mxfp6
• nvfp4
• MTP
• qwen3.6

---

Changed GGML TYPE to 50, latest mxfp6 build is required

Model Card for Model ID

This is my best model yet! It offers incredibly fast performance with incredible quality.<BR>

This is the <B>MXFP6</B> enhanced version of michaelw9999/Qwen3.6-27B-NVFP4-MTP-GGUF.<p>It was quantized with my work-in-progress advanced-gguf-quantizer<BR>

<BR><U><b>Why add MXFP6 tensors into the NVFP4?</b></u><BR>

MXFP6 has vastly superior quality compared to NVFP4.<br>

Only a relatively small portion of this model is MXFP6, though; it's used to boost the quality up, where needed.<br>Most of the model is NVFP4, some small numbers of tensors are Q4_K.<br>

How much MXFP6 to inject to get close to BF16 in real world usage and benchmarks vs raw numbers remains TBD.<br>

This model is almost as fast as the primary NVFP4 model, and just a tiny bit bigger, but the results are further improved:

See this chart (thanks to@raulgdm) with the NVFP4/MXFP6 blend's improvement over the all original model:<BR>

<div class="bench-chart">

<style>

.bench-chart {

font-family: Inter, system-ui, -apple-system, BlinkMacSystemFont, "Segoe UI", sans-serif;

color: #111827;

max-width: 1400px;

}

.bench-chart table {

border-collapse: collapse;

width: 100%;

font-size: 14px;

line-height: 1.35;

}

.bench-chart caption {

caption-side: top;

text-align: left;

font-weight: 700;

font-size: 18px;

margin-bottom: 10px;

}

.bench-chart th,

.bench-chart td {

border: 1px solid #d1d5db;

padding: 8px 10px;

text-align: right;

white-space: nowrap;

}

.bench-chart th:first-child,

.bench-chart td:first-child {

text-align: left;

}

.bench-chart thead th {

background: #f3f4f6;

font-weight: 700;

}

.bench-chart .ours {

background: #ecfdf5;

font-weight: 700;

}

.bench-chart .note {

margin-top: 8px;

font-size: 12px;

color: #4b5563;

}

</style>

<table>

<caption>Qwen3.6-27B Quantization Benchmarks</caption>

<thead>

<tr>

<th>Benchmark</th>

<th>Samples</th>

<th>Q6_K (150K)</th>

<th>NVFP4-Q8_0 (27/05)</th>

<th>NVFP4 1f (29/05)</th>

<th>NVFP4 1f updated (30/05)</th>

<th class="ours">NVFP4/MXFP6 (w/ repack)</th>

</tr>

</thead>

<tbody>

<tr><td>GSM8K</td><td>51</td><td>88.2%</td><td>86.3%</td><td>88.2%</td><td>88.2%</td><td class="ours">96.1%</td></tr>

<tr><td>ARC Easy</td><td>30</td><td>100%</td><td>100%</td><td>100%</td><td>100%</td><td class="ours">100%</td></tr>

<tr><td>HellaSwag</td><td>30</td><td>63.3%</td><td>66.7%</td><td>73.3%</td><td>73.3%</td><td class="ours">86.7%</td></tr>

<tr><td>PIQA</td><td>20</td><td>100%</td><td>100%</td><td>100%</td><td>100%</td><td class="ours">100%</td></tr>

<tr><td>HumanEval pass@1</td><td>164</td><td>85.4%</td><td>84.8%</td><td>86.0%</td><td>85.4%</td><td class="ours">87.2%</td></tr>

<tr><td>Average</td><td>&mdash;</td><td>87.9%</td><td>88.2%</td><td>90.4%</td><td>90.4%</td><td class="ours">94.0%</td></tr>

<tr><td>PPL ratio (WikiText-2 full KLD)</td><td>&mdash;</td><td>&mdash;</td><td>&mdash;</td><td>1.13</td><td>1.105</td><td class="ours">1.0269</td></tr>

</tbody>

</table>

<div class="note">

PPL ratio = Mean PPL(Q)/PPL(base) = 1.026890 from the full KLD/PPL run.

</div>

</div>

<br>For more context, see:

<br><br>https://github.com/ggml-org/llama.cpp/discussions/22498 and https://github.com/ggml-org/llama.cpp/discussions/23853<BR><BR>

<U>You <B>must</B> install the <B>unofficial</B> MXFP6 CUDA build of llama.cpp for this model to work.<br><BR></U>

For best performance and quality, use NVFP4-Repack with MXFP6 from:<br>

https://github.com/michaelw9999/llama.cpp/tree/nvfp4repack_mxfp6_cuda<br>

Without NVFP4 repack:<br>

https://github.com/michaelw9999/llama.cpp/tree/mxfp6-cuda<br><BR>These branches are updated regularly.<BR>

NVFP4 repack preloads all tensors into a CUDA tile to boost speed. It is a tiny bit slower on first load, then provides ~10% prefill boost with a small reduction in token gen seen on larger models, and an increase on smaller models.<br>

However, it also enables NVFP4 <b>input scale</b>, which boosts model correctness.<br><br><br>

<B><U>Model Performance on 5090 with MXFP6-CUDA branch</U></B><br>

This model should be faster than the NVFP4-only version, as the MTP tensors are MXFP6.

| qwen35 27B NVFP4-MXFP6|  15.23 GiB |    27.32 B | CUDA       |  99 | pp512 |       5180.83 ± 2.14  |
| qwen35 27B NVFP4-MXFP6|  15.23 GiB |    27.32 B | CUDA       |  99 | tg128 |         76.56 ± 0.27

<B><U>Model Performance on 5090 (with NVFP4 Repack)</U><B>

| qwen35 27B NVFP4-MXFP6|  15.23 GiB |    27.32 B | CUDA       |  99 |  pp512 |      5675.78 ± 94.88 |
| qwen35 27B NVFP4-MXFP6|  15.23 GiB |    27.32 B | CUDA       |  99 |  tg128 |         73.71 ± 1.24 

<B><U>Model Quality</U></B><BR>

Running the ppl/kld test on the standard <b>wiki.test</B>:

<BR>

<B>With</B> NVFP4-Repack

====== Perplexity statistics ======
Mean PPL(Q)                   :   7.081974 ±   0.047190
Mean PPL(base)                :   6.900856 ±   0.045374
Cor(ln(PPL(Q)), ln(PPL(base))):  98.63%
Mean ln(PPL(Q)/PPL(base))     :   0.025907 ±   0.001098
Mean PPL(Q)/PPL(base)         :   1.026246 ±   0.001127
Mean PPL(Q)-PPL(base)         :   0.181118 ±   0.007867

====== KL divergence statistics ======
Mean    KLD:   0.054232 ±   0.000890
Maximum KLD:  25.507811
99.9%   KLD:   4.043683
99.0%   KLD:   0.546427
95.0%   KLD:   0.152158
90.0%   KLD:   0.088751
Median  KLD:   0.018002
10.0%   KLD:   0.000454
 5.0%   KLD:   0.000123
 1.0%   KLD:   0.000016
 0.1%   KLD:   0.000002
Minimum KLD:  -0.000066

====== Token probability statistics ======
Mean    Δp: -0.213 ± 0.017 %
Maximum Δp: 99.434%
99.9%   Δp: 46.017%
99.0%   Δp: 15.815%
95.0%   Δp:  7.191%
90.0%   Δp:  4.217%
75.0%   Δp:  0.866%
Median  Δp: -0.002%
25.0%   Δp: -0.963%
10.0%   Δp: -4.442%
 5.0%   Δp: -7.884%
 1.0%   Δp: -20.542%
 0.1%   Δp: -61.177%
Minimum Δp: -99.506%
RMS Δp    :  6.427 ± 0.059 %
Same top p: 90.779 ± 0.075 %

<B>Without</B> NVFP4 Repack:

====== Perplexity statistics ======
Mean PPL(Q)                   :   7.088015 ±   0.047272
Mean PPL(base)                :   6.900856 ±   0.045374
Cor(ln(PPL(Q)), ln(PPL(base))):  98.68%
Mean ln(PPL(Q)/PPL(base))     :   0.026760 ±   0.001081
Mean PPL(Q)/PPL(base)         :   1.027121 ±   0.001111
Mean PPL(Q)-PPL(base)         :   0.187159 ±   0.007769

====== KL divergence statistics ======
Mean    KLD:   0.054489 ±   0.000910
Maximum KLD:  20.432301
99.9%   KLD:   4.555052
99.0%   KLD:   0.521524
95.0%   KLD:   0.151849
90.0%   KLD:   0.087985
Median  KLD:   0.017979
10.0%   KLD:   0.000450
 5.0%   KLD:   0.000121
 1.0%   KLD:   0.000016
 0.1%   KLD:   0.000002
Minimum KLD:  -0.000051

====== Token probability statistics ======
Mean    Δp: -0.209 ± 0.017 %
Maximum Δp: 99.650%
99.9%   Δp: 46.285%
99.0%   Δp: 15.675%
95.0%   Δp:  7.215%
90.0%   Δp:  4.196%
75.0%   Δp:  0.861%
Median  Δp: -0.002%
25.0%   Δp: -0.963%
10.0%   Δp: -4.445%
 5.0%   Δp: -7.878%
 1.0%   Δp: -20.604%
 0.1%   Δp: -57.880%
Minimum Δp: -99.882%
RMS Δp    :  6.386 ± 0.060 %
Same top p: 90.855 ± 0.075 %