Bucoid/Qwen3.8-27B-Heretic-Ara-16GB-VRAM-IQ4-XS-MTP-GGUF overview

---

license: apache-2.0

base_model:

• Qwen/Qwen3.8-27B

---

---

Qwen 3.8 27B Heretic‑Ara IQ4_XS 量化模型（适配 16GB 显存）

本模型基于 Qwen 3.8 27B Heretic‑Ara BF16 进行 IQ4_XS 量化（4‑bit），文件体积为 12.7-13.3 GiB，专为 16GB 显存的显卡优化

8/22 更新修复思考的问题，模型性能没有变化

大概还需要1天我会更新MTP的版本

8/22 Updated and fixed the issues with thinking; model performance remains unchanged

It will probably take about another day for me to update the MTP version.

8/23 更新，对模型本身的性能进行一定优化，体积略微变大，无MTP版本13Gib，MTP版本13.3Gib

8/23 Update: The model's performance has been slightly optimized, resulting in a slight increase in size

The non-MTP version is 13GiB, while the MTP version is 13.3GiB

与同体积的 Heretic‑Ara‑Q3_K_M（12.4 GiB）量化方案进行了全面对比

本模型使用Heretic Arbitrary-Rank Ablation做到的无审查

📊 量化质量对比

| 评估指标 | Heretic-Ara BF16 (base) | IQ4_XS-3.0 | IQ4_XS-2.0| Heretic-Ara-Q3_K_M|

|----------|-------------------------|-----|---------------------|---------------------------|

| 文件大小 | 50.1 GiB | 13 GiB (带MTP 13.3 GiB) | 12.7 GiB | 12.4 GiB |

| 量化精度 | BF16 | IQ4_XS (4‑bit) | IQ4_XS (4‑bit) | Q3_K_M (约 3‑bit) |

| 模型困惑度 (Mean PPL) | 7.008212 ± 0.045362 | 7.046980 ± 0.045498 | 7.102940 ± 0.046017 | 7.403971 ± 0.048924 |

| 与基座模型 PPL 相关性 | 100% | 99.34% | 99.26% | 98.31% |

| 平均 KL 散度 (Mean KLD) | 0 | 0.027832 ± 0.000324 | 0.033398 ± 0.000308 | 0.076034 ± 0.000554 |

| 最大 KL 散度 (Max KLD) | 0 | 18.317436 | 15.094215 | 17.866985 |

| 99.9% KL 分位数 | 0 | 1.162850 | 1.130034 | 2.448278 |

| Top‑1 一致率 (Same top p) | 100% | 92.867% ± 0.067% | 91.619% ± 0.072% | 88.152% ± 0.084% |

| 平均概率变化 (Mean Δp) | 0% | -0.243% ± 0.012% | -0.306% ± 0.013% | -0.490% ± 0.020% |

| RMS 概率变化 (RMS Δp) | 0% | 4.538% ± 0.045% | 4.952% ± 0.041% | 7.560% ± 0.054% |

> 注：基座（BF16）的 KL 散度、Δp 等指标均为 0（自身对比），一致率为 100%。

在不启用 MTP 的情况下，IQ4_XS 模型在 16 GiB 无显存占用（不作为 Windows 显示显卡）下可支持约 110k 上下文

开启 MTP 后约为 80k

Qwen 3.8 27B Heretic‑Ara IQ4_XS Quantized Model (Optimized for 16 GB VRAM)

This model is quantized from Qwen 3.8 27B Heretic‑Ara BF16 using the IQ4_XS scheme (4‑bit), with a file size of 12.8 GiB, specifically designed for graphics cards with 16 GB of VRAM.

It has been comprehensively compared against the similarly sized Heretic‑Ara‑Q3_K_M (12.4 GiB) quantization variant.

This model achieves uncensored behavior through Heretic's Arbitrary-Rank Ablation.

📊 Quantization Quality Comparison

| Evaluation Metric | Heretic-Ara BF16 (base) | IQ4_XS-3.0 | IQ4_XS-2.0| Heretic-Ara-Q3_K_M (comparison) |

|-------------------|-------------------------|-----|-------------------------|--------------------------------|

| File Size | 50.1 GiB | 13 GiB (with MTP 13.3 GiB) | 12.7 GiB | 12.4 GiB |

| Quantization Precision | BF16 | IQ4_XS (4‑bit) | IQ4_XS (4‑bit) | Q3_K_M (~3‑bit) |

| Model Perplexity (Mean PPL) | 7.008212 ± 0.045362 | 7.046980 ± 0.045498 | 7.102940 ± 0.046017 | 7.403971 ± 0.048924 |

| PPL Correlation with Base Model | 100% | 99.34% | 99.26% | 98.31% |

| Mean KL Divergence (Mean KLD) | 0 | 0.027832 ± 0.000324 | 0.033398 ± 0.000308 | 0.076034 ± 0.000554 |

| Max KL Divergence (Max KLD) | 0 | 18.317436 | 15.094215 | 17.866985 |

| 99.9% KL Quantile | 0 | 1.162850 | 1.130034 | 2.448278 |

| Top‑1 Agreement Rate (Same top p) | 100% | 92.867% ± 0.067% | 91.619% ± 0.072% | 88.152% ± 0.084% |

| Mean Probability Change (Mean Δp) | 0% | -0.243% ± 0.012% | -0.306% ± 0.013% | -0.490% ± 0.020% |

| RMS Probability Change (RMS Δp) | 0% | 4.538% ± 0.045% | 4.952% ± 0.041% | 7.560% ± 0.054% |

> Note: For the base model (BF16), KL divergence, Δp, etc. are all 0 (self‑comparison), and the agreement rate is 100%.

With MTP (Multi‑Token Prediction) disabled, the IQ4_XS model supports approximately 110k context length on a 16 GiB GPU with no VRAM reserved for display (i.e., not used as the primary display adapter).

With MTP enabled, the context length is approximately 80k.