---

license: apache-2.0

base_model: windowsxp811203/Qwen3.8-27B-Abliterated

base_model_relation: quantized

pipeline_tag: image-text-to-text

library_name: gguf

tags:

• gguf
• llama.cpp
• qwen3_5
• abliterated
• uncensored
• imatrix
• mtp
• qwen3.8
• qwen
• text-generation

language:

• en
• zh

---

Qwen3.8-27B-Abliterated — GGUF

GGUF builds of windowsxp811203/Qwen3.8-27B-Abliterated,

an abliterated (refusal-removed) Qwen/Qwen3.8-27B.

These files keep the MTP draft head and ship a vision projector. Both are commonly lost in

derivative builds — the MTP head because from_pretrained silently drops it.

| file | size | notes |

|---|---|---|

| IQ4_XS | 15.3 GB | smallest usable |

| Q4_K_M | 16.8 GB | recommended for 24 GB cards |

| Q5_K_M | 19.5 GB | |

| Q6_K | 22.4 GB | near-lossless |

| Q8_0 | 29.0 GB | reference |

| mmproj-…-F16 | 0.93 GB | vision — required for image input |

| imatrix.dat | 14 MB | importance matrix used for the K/IQ quants |

All quants except Q8_0 were produced with an importance matrix computed on a neutral general

corpus (calibration_datav3, science/general prose). Refusal-heavy text was deliberately not

used, so the imatrix cannot re-emphasise the direction that was removed.

Quality — perplexity

Held-out wikitext-2 test, 60 chunks @ -c 2048, identical settings:

| build | PPL | Δ vs BF16 |

|---|---|---|

| BF16 (reference) | 5.9566 ± 0.057 | — |

| Q8_0 | 5.9603 | +0.004 |

| Q6_K | 5.9576 | +0.001 |

| Q5_K_M | 5.9500 | −0.007 |

| Q4_K_M | 5.9161 | −0.041 |

| IQ4_XS | 5.9871 | +0.031 |

Every delta is inside the ±0.057 error bar, so these are statistically indistinguishable from

BF16 and from each other. Q4_K_M scoring below BF16 is measurement noise, not an improvement.

Refusal — the point of this model

Scored with a deliberately loose keyword classifier (it over-counts refusals), greedy decoding,

non-thinking mode, and no prefill jailbreak. For reference the unmodified Qwen3.8-27B refuses

99.04 % of AdvBench under identical settings.

| build | AdvBench (160) | HarmBench safety categories (119) | HarmBench copyright (41) |

|---|---|---|---|

| Q8_0 | 0.00 % | 0/119 · 0.0 % | 46.3 % |

| Q6_K | 0.00 % | 0/119 · 0.0 % | 41.5 % |

| Q5_K_M | 0.00 % | 0/119 · 0.0 % | 43.9 % |

| Q4_K_M | 0.62 % | 0/119 · 0.0 % | 48.8 % |

| IQ4_XS | 0.00 % | 0/119 · 0.0 % | 36.6 % |

"Safety categories" = chemical/biological, cybercrime, harassment, harmful, illegal,

misinformation. Every one is exactly zero at every quantisation level — the abliteration

survives 4-bit intact.

The copyright column is not a safety refusal and is mostly classifier false positives: asked for

song lyrics or a book passage the model delivers them, but the text trips the keyword list —

either because the prose itself opens with "I cannot quite believe…", or because of a pedantic

preamble ("I cannot generate a new passage … but here is a long excerpt") followed by the

excerpt. The single Q4_K_M AdvBench flag is the same kind: "I cannot lie to you…" followed by

the requested review.

Usage

# text
llama-cli -m Qwen3.8-27B-Abliterated-Q4_K_M.gguf -ngl 99 -c 8192

# vision (needs the mmproj file)
llama-mtmd-cli -m Qwen3.8-27B-Abliterated-Q4_K_M.gguf \
  --mmproj mmproj-Qwen3.8-27B-Abliterated-F16.gguf --image photo.jpg -p "Describe this image."

# MTP speculative decoding (the draft head is inside the file — no separate draft model)
llama-server -m Qwen3.8-27B-Abliterated-Q4_K_M.gguf -ngl 99 --spec-type draft-mtp

Thinking is on by default. Disable it per request with

"chat_template_kwargs": {"enable_thinking": false}, or server-wide with

--chat-template-kwargs '{"enable_thinking":false}'.

MTP composes with the rest: verified on this build that --spec-type draft-mtp works together

with --mmproj (multimodal model and MTP draft context both load; acceptance 0.71) and with

-np 4 (4 slots, each reporting its own draft acceptance). An earlier revision of this card

claimed these were mutually exclusive — that was wrong and is corrected here.

When running without MTP you will see model has unused tensor blk.64.* warnings. That is

expected: blk.64 is the MTP/nextn block and is only used under --spec-type draft-mtp.

Verified: MTP, vision, and long context

MTP speculative decoding works out of the box (--spec-type draft-mtp); the server logs

creating MTP draft context against the target model. Measured on Q4_K_M at

--spec-draft-n-max 3: draft acceptance 0.35 – 0.53, mean accepted length 2.05 – 2.58.

Vision (--mmproj) on a synthetic three-shape image, Q4_K_M:

> "There is a red circle … a green triangle … a blue square."

Long context. The base architecture is sometimes reported to loop or collapse in long

sessions past ~50–70K tokens. That does not reproduce here. Needle-in-a-haystack (unique

passphrase buried at 50 % depth, greedy, Q4_K_M), plus a 5-gram repetition check on the answer:

| context | prompt tokens | retrieved | degeneration |

|---|---|---|---|

| 128 K | 106,808 | ✅ | none |

| 256 K (native max) | 215,470 | ✅ | none |

| 512 K (extrapolated) | 433,950 | ✅ | none |

| 1 M (extrapolated) | 823,868 | ✅ | none |

A separate context sweep (wikitext-2 train split, 12 chunks — a different corpus and chunk

count from the held-out quant table above, so compare within this row only) shows no long-context

degradation: 5.95 (4K) · 6.17 (16K) · 6.20 (32K) · 6.21 (64K) · 6.14 (128K) · 6.05 (256K).

Perplexity rises slightly out to 64K and then falls again toward the native maximum, ending

0.10 below its 64K peak; the overall minimum is at 4K, as expected for short chunks.

Contexts beyond 262,144 require raising the declared limit (llama.cpp caps the slot at

n_ctx_train otherwise); the 512K/1M rows above were run with qwen35.context_length overridden,

the llama.cpp equivalent of vLLM's --hf-overrides.

Reaching 1M context with llama.cpp

These files declare qwen35.context_length = 262144 (the model's native limit), and **llama.cpp

caps the slot at that value** — asking for more silently gets you 256K:

srv load_model: the slot context (1000192) exceeds the training context (262144) - capping

To use the extrapolated 1M configuration (the llama.cpp equivalent of vLLM's --hf-overrides),

raise the declared limit on a copy first:

python llama.cpp/gguf-py/gguf/scripts/gguf_set_metadata.py \
  Qwen3.8-27B-Abliterated-Q4_K_M.gguf qwen35.context_length 1010000 --force
llama-server -m Qwen3.8-27B-Abliterated-Q4_K_M.gguf -ngl 99 -c 1000000

Verified this way: 823,868 prompt tokens, needle retrieved, no degeneration, 1105 s on one H200.

(The NVFP4 build served by vLLM does the same in 364 s on two Blackwells.)

Provenance

Converted with llama.cpp (arch = qwen35, block_count = 65 = 64 layers + 1 MTP/nextn block,

qwen35.nextn_predict_layers = 1) from the bf16 parent. The parent was produced by orthogonalizing

131 residual-writing tensors (including embed_tokens) against a refusal direction at λ=1.5, with

the vision tower left byte-identical. Full recipe and evaluation in the

parent model card.

Support / 打賞

If these models are useful to you, tips are appreciated — they pay for the GPU time.

如果這些模型對你有幫助，歡迎打賞，用於支應算力成本。

USDT (TRC20) · TPTo32r7vKazpTNaFqfFZ2ztoK1DG88888

Disclaimer

This model will not refuse. It is published for alignment and safety research. You are responsible

for your use of it and for complying with applicable law. Inherits the Apache-2.0 license of the

base model.