---

license: apache-2.0

base_model: gjtgjt/Qwen3.8-27B-heretic-r1n

pipeline_tag: image-text-to-text

library_name: gguf

tags:

- gguf

- llama.cpp

- qwen

- qwen3

- qwen3.8

- heretic

- abliterated

language:

- en

- zh

---

Qwen3.8-27B-heretic-r1n-GGUF

GGUF quantizations of gjtgjt/Qwen3.8-27B-heretic-r1n, an iterative true rank-1 (PRE) Heretic 1.4.0 of official Qwen/Qwen3.8-27B (1d4bf0f2).

gjtgjt/Qwen3.8-27B-heretic-r1n 的 GGUF 量化。该全精度仓是对官方 Qwen/Qwen3.8-27B 做 真 rank-1（PRE） 迭代 Heretic 1.4.0 的结果。

This is a separate line from gjtgjt/Qwen3.8-27B-heretic (single-shot FULL). GGUF for that line: gjtgjt/Qwen3.8-27B-heretic-GGUF.

这与单轮 FULL 线 gjtgjt/Qwen3.8-27B-heretic 是两条线。对应量化仓：gjtgjt/Qwen3.8-27B-heretic-GGUF。

Files / 文件

| File | Quant | Size | Notes |

|---|---|---|---|

| Qwen3.8-27B-heretic-r1n-Q8_0.gguf | Q8_0 | 26.63 GiB / 28.60 GB | higher fidelity |

| Qwen3.8-27B-heretic-r1n-Q5_K_M.gguf | Q5_K_M | 17.91 GiB / 19.23 GB | smaller |

| mmproj-Qwen3.8-27B-BF16.gguf | BF16 | 0.87 GiB / 0.93 GB | vision encoder; required for images |

Text files are 64 language-model layers. Converted with --no-nextn, so the native MTP draft head is not in these GGUFs. Vision was not ablated; the mmproj is the same file as in the FULL GGUF repo (identical SHA-256).

文本 GGUF 是 64 层语言模型，转换时加了 --no-nextn，不含 MTP。视觉塔未被消融；mmproj 与 FULL 量化仓是同一份（SHA-256 相同）。

Converted with llama.cpp b94041a (convert_hf_to_gguf.py --outtype f16 --no-nextn, then llama-quantize).

转换：llama.cpp b94041a，F16 + --no-nextn，再量化。

Parent model / 全精度来源

| | |

|---|---|

| Base | Qwen/Qwen3.8-27B @ 1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0 |

| Method | Heretic 1.4.0, --row-normalization PRE (true rank-1), iterated |

| Final | round 2, trial 109 |

| Eval | Keyword 18 / 100, cumulative KL vs original 0.0931 |

| Rejected | round 3, Keyword 8, KL 0.1983 (over the 0.1 cap) |

| Untouched | vision tower, MTP |

| License | Apache-2.0 |

| Round | Keywords / 100 | KL vs original | Decision |

|---|---:|---:|---|

| Base | 98 | 0 | — |

| 1 (trial 162) | 25 | 0.0525 | accepted |

| 2 (trial 109) | 18 | 0.0931 | this release |

| 3 | 8 | 0.1983 | rejected (KL > 0.1) |

Contrast: the FULL single-shot line is Keyword 27 / KL 0.0446. This PRE line refuses less (18) with higher but still capped KL.

对照：FULL 单轮是 Keyword 27、KL 0.0446。本线拒绝更低（18），KL 更高但仍在 0.1 以内。

Usage / 用法

Needs a llama.cpp build that loads Qwen3.8 (qwen35). Sampling from the official Qwen3.8 card.

需要能加载 Qwen3.8（qwen35）的 llama.cpp。采样参数来自官方模型卡。

# thinking (default)
llama-cli \
  -m Qwen3.8-27B-heretic-r1n-Q8_0.gguf \
  -ngl 99 -n 16384 \
  --temp 1.0 --top-p 0.95 --top-k 20 \
  -p "Explain residual connections in one paragraph."

# instruct (thinking off)
llama-cli \
  -m Qwen3.8-27B-heretic-r1n-Q8_0.gguf \
  -ngl 99 --reasoning off \
  --temp 0.7 --top-p 0.8 --top-k 20 --presence-penalty 1.5 \
  -p "Say hello in one short sentence."

Vision / 图像

llama-mtmd-cli \
  -m Qwen3.8-27B-heretic-r1n-Q8_0.gguf \
  --mmproj mmproj-Qwen3.8-27B-BF16.gguf \
  --image ./photo.jpg \
  -p "Describe this image." \
  --temp 0.7 --top-p 0.8 --top-k 20

LM Studio / jan / KoboldCpp: load the GGUF and keep the embedded Qwen3.8 chat template.

Sampling

| Mode | temperature | top_p | top_k | presence_penalty |

|---|---:|---:|---:|---:|

| Thinking (default) | 1.0 | 0.95 | 20 | 0.0 |

| Instruct (thinking off) | 0.7 | 0.80 | 20 | 1.5 |

Native context is 262,144 tokens. Set -c to what you need; KV cache, not the weight file, dominates VRAM at long context.

原生上下文 262,144。长上下文时 KV 才是显存大头。

Notes / 说明

• No perplexity or speed numbers are claimed here; none were measured for these files.
• 这里不提供 PPL 或测速数字，这些文件没有做过那些评测。
• Keyword counts are English-only (Heretic default markers).
• Keyword 计数只覆盖 Heretic 默认英文评测。

Provenance

Quantization of gjtgjt/Qwen3.8-27B-heretic-r1n. Upstream base Qwen/Qwen3.8-27B. Weights Apache-2.0.

Acknowledgements

• Base: Qwen team
• Abliteration: Heretic
• GGUF: llama.cpp