---

license: apache-2.0

base_model: gjtgjt/Qwen3.8-27B-heretic

pipeline_tag: image-text-to-text

library_name: gguf

tags:

- gguf

- llama.cpp

- qwen

- qwen3

- qwen3.8

- heretic

- abliterated

language:

- en

- zh

---

Qwen3.8-27B-heretic-GGUF

GGUF quantizations of gjtgjt/Qwen3.8-27B-heretic, a Heretic 1.4.0 FULL (single-shot) abliteration of official Qwen/Qwen3.8-27B (1d4bf0f2).

gjtgjt/Qwen3.8-27B-heretic 的 GGUF 量化。该全精度仓是对官方 Qwen/Qwen3.8-27B 做 Heretic 1.4.0 单轮 FULL 消融的结果。

This is a separate line from gjtgjt/Qwen3.8-27B-heretic-r1n (iterative true rank-1 / PRE). GGUF for that line: gjtgjt/Qwen3.8-27B-heretic-r1n-GGUF.

这与 PRE 迭代线 gjtgjt/Qwen3.8-27B-heretic-r1n 是两条线。对应量化仓：gjtgjt/Qwen3.8-27B-heretic-r1n-GGUF。

Files / 文件

| File | Quant | Size | Notes |

|---|---|---|---|

| Qwen3.8-27B-heretic-Q8_0.gguf | Q8_0 | 26.63 GiB / 28.60 GB | higher fidelity |

| Qwen3.8-27B-heretic-Q5_K_M.gguf | Q5_K_M | 17.91 GiB / 19.23 GB | smaller |

| mmproj-Qwen3.8-27B-BF16.gguf | BF16 | 0.87 GiB / 0.93 GB | vision encoder; required for images |

Text files are 64 language-model layers. The native MTP draft head was not written into these GGUFs (qwen35.block_count=64, nextn_predict_layers=0). Vision was not ablated; the mmproj is converted from the same heretic tree.

文本 GGUF 是 64 层语言模型。原生 MTP 草稿头没有打进这些文件。视觉塔未被消融；mmproj 从同一套 heretic 权重转出。

Converted with llama.cpp b94041a (convert_hf_to_gguf.py --outtype f16, then llama-quantize).

转换：llama.cpp b94041a，先 F16 再量化。

Parent model / 全精度来源

| | |

|---|---|

| Base | Qwen/Qwen3.8-27B @ 1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0 |

| Method | Heretic 1.4.0, default FULL row-normalization (not PRE) |

| Search | 200 trials / 60 startup, instruct cut \n</think>\n\n |

| Selected | trial 145 |

| Eval | Keyword 27 / 100, KL vs original 0.0446 (Heretic default English eval) |

| Untouched | vision tower, MTP |

| License | Apache-2.0 |

FULL writes a ~rank-3 SVD approximation after row-norm restore. The r1n line is iterative true rank-1 (PRE) and is not this file.

FULL 在行范数还原后再做约 rank-3 的 SVD 近似。r1n 线是迭代真 rank-1（PRE），不是这个文件。

Usage / 用法

Needs a llama.cpp build that loads Qwen3.8 (qwen35). Sampling from the official Qwen3.8 card.

需要能加载 Qwen3.8（qwen35）的 llama.cpp。采样参数来自官方模型卡。

# thinking (default)
llama-cli \
  -m Qwen3.8-27B-heretic-Q8_0.gguf \
  -ngl 99 -n 16384 \
  --temp 1.0 --top-p 0.95 --top-k 20 \
  -p "Explain residual connections in one paragraph."

# instruct (thinking off)
llama-cli \
  -m Qwen3.8-27B-heretic-Q8_0.gguf \
  -ngl 99 --reasoning off \
  --temp 0.7 --top-p 0.8 --top-k 20 --presence-penalty 1.5 \
  -p "Say hello in one short sentence."

Vision / 图像

llama-mtmd-cli \
  -m Qwen3.8-27B-heretic-Q8_0.gguf \
  --mmproj mmproj-Qwen3.8-27B-BF16.gguf \
  --image ./photo.jpg \
  -p "Describe this image." \
  --temp 0.7 --top-p 0.8 --top-k 20

LM Studio / jan / KoboldCpp: load the GGUF and keep the embedded Qwen3.8 chat template.

Sampling

| Mode | temperature | top_p | top_k | presence_penalty |

|---|---:|---:|---:|---:|

| Thinking (default) | 1.0 | 0.95 | 20 | 0.0 |

| Instruct (thinking off) | 0.7 | 0.80 | 20 | 1.5 |

Native context is 262,144 tokens. Set -c to what you need; KV cache, not the weight file, dominates VRAM at long context.

原生上下文 262,144。长上下文时 KV 才是显存大头。

Notes / 说明

• No perplexity or speed numbers are claimed here; none were measured for these files.
• 这里不提供 PPL 或测速数字，这些文件没有做过那些评测。
• Keyword counts are English-only (Heretic default markers).
• Keyword 计数只覆盖 Heretic 默认英文评测。

Provenance

Quantization of gjtgjt/Qwen3.8-27B-heretic. Upstream base Qwen/Qwen3.8-27B. Weights Apache-2.0.

Acknowledgements

• Base: Qwen team
• Abliteration: Heretic
• GGUF: llama.cpp