---

license: apache-2.0

base_model: Qwen/Qwen3.8-27B

library_name: gguf

pipeline_tag: image-text-to-text

tags:

• abliterated
• uncensored
• qwen3.8
• gguf
• imatrix
• mtp
• vision
• mmproj
• not-for-all-audiences

---

Qwen3.8-27B - Abliterated (GGUF, imatrix)

Imatrix-quantized GGUF ladder of

RobinsonLabs/Qwen3.8-27B-abliterated,

which is an abliterated bf16 base of Qwen/Qwen3.8-27B.

The MTP head is abliterated in-band and the vision tower is preserved -- see the base

repo for the method, the verification, and the measured refusal numbers.

fp16 base

The full-precision master these were cut from is

RobinsonLabs/Qwen3.8-27B-abliterated -- bf16 safetensors, carrying the method, the

verification, and the measured refusal numbers. Go there to re-abliterate, LoRA-merge, fine-tune,

or roll your own quants.

Vision

mmproj-Qwen3.8-27B-abliterated-f16.gguf (334 tensors) is published here. Download it

alongside whichever main quant you pick and pass it with --mmproj to get the vision half.

Without it you have a capable text model and no image input.

llama-server -m Qwen3.8-27B-abliterated-Q4_K_M.gguf \
             --mmproj mmproj-Qwen3.8-27B-abliterated-f16.gguf \
             -ngl 99 -c 32768

Quants

| file | bits | size | bpw | fits |

|---|---|---|---|---|

| Q8_0 | 8 | 29.05 GB | 8.51 | 2x24GB, or 32GB+ |

| Q6_K | 6 | 22.43 GB | 6.57 | 24GB card, quality ceiling |

| Q5_K_M | 5 | 19.54 GB | 5.72 | 24GB comfortable |

| Q4_K_M | 4 | 16.84 GB | 4.93 | 24GB / 16GB with offload -- the volume rung |

| IQ4_XS | 4 | 15.37 GB | 4.50 | 16GB card |

| Q3_K_M | 3 | 13.59 GB | 3.98 | 16GB tight |

| IQ3_XS | 3 | 12.26 GB | 3.59 | 12GB card |

| IQ2_M | 2 | 10.30 GB | 3.02 | 10-12GB card -- quality-compromised, read the note |

Sizes are the built artifacts, exact. bpw is bytes x 8 over the model's own 27,320,697,856

parameters, summed from the master's tensor shapes rather than taken off the "27B" in the name.

Every K/IQ rung is imatrix-guided except for the MTP block, which no imatrix covers -- see below

for what we do about it. Q8_0 uses no imatrix by design; it gains essentially nothing from

importance weighting.

What the low rungs actually cost

Measured rather than asserted. Perplexity over a held-out slice of wikitext-2 -- deliberately NOT

the operator corpus the imatrix was calibrated on, so this is an out-of-distribution read and not a

flattering one. What matters is each rung's distance from the Q8_0 reference on identical text.

| rung | PPL | vs Q8_0 |

|---|---|---|

| Q8_0 (reference) | 5.9283 | -- |

| IQ3_XS | 6.2212 | +4.9% |

| IQ2_M | 6.7226 | +13.4% |

IQ2_M is a real quality step down and is labelled as such. It stays coherent -- it holds an

argument, follows a format instruction, and reasons correctly about physics in spot checks -- but

if you have the VRAM for IQ3_XS or above, take it. Ship IQ2_M when 10-12 GB is the constraint,

not because it is close to the top of the ladder.

We also built IQ2_XS and did not publish it. It measured 7.4030, or +24.9% against the

reference, to save 0.91 GB over IQ2_M -- roughly three times worse per gigabyte than the step

above it, and past the point where we are willing to put our name on the output. The file exists;

it is not here on purpose.

The imatrix is not the usual one

Most published imatrix quants calibrate on calibration_datav3.txt or similar generic English.

This ladder is calibrated on an in-domain corpus -- 1.8 MB / 17.5K lines of real technical

operator transcripts (infrastructure work, debugging, model-building dialogue), 120 chunks,

final PPL 10.3431 +/- 0.168.

That is a deliberate trade, not an accident. It biases the quantization error toward preserving

behaviour on long technical dialogue, tool use, and operator-style instruction-following. If your

use case is closer to general chat or non-English, a generic-calibrated ladder may serve you

better and that is fine -- we would rather tell you the calibration than let you assume it.

What the imatrix does not cover

Being straight about a limitation, since the calibration is the selling point.

llama-imatrix collects its statistics during a perplexity-style forward pass, and that pass never

runs the model's MTP (multi-token-prediction) draft head. So the imatrix carries entries for

blk.0 through blk.63 -- the 64 trunk layers -- and nothing for blk.64, the MTP block.

llama.cpp handles that two ways, and only one of them tells you: above its "very low-bit"

threshold it quantizes the block blind at the trunk's depth and says nothing, and at IQ3_XS it

refuses and aborts the build. The loud case is the honest one, and it is how we found the quiet

one.

Rather than ship a block with no importance data at low precision, this ladder holds an invariant:

> blk.64 is never below q5_K, and never an I-quant, in any rung.

Q8_0, Q6_K and Q5_K_M already satisfied that from the stock mixture and are untouched.

Q4_K_M, IQ4_XS, Q3_K_M and IQ3_XS pin it explicitly with

--tensor-type 'blk\.64\.=q5_K'. IQ4_XS is the one worth calling out: its MTP block would

otherwise be a blind iq4_xs, and I-quants are precisely the family that leans on importance

data. The pin costs roughly 0.1 GB per rung.

If you re-quantize this model yourself, check your imatrix's block coverage against n_layers

before you start, rather than discovering it at the bottom rung.

Disclosure

Abliterated: the hard-refusal reflex on adult / creative content is reduced via single-direction

weight orthogonalization. Harm guardrails are retained by design -- self-harm prompts still

redirect to help (e.g. 988) rather than comply. Not a jailbreak-for-anything model, not intended

to assist genuine wrongdoing. Tagged not-for-all-audiences. See the base repo for the full

disclosure and the measured base-vs-abliterated numbers.

Provenance

Built by Robinson Labs with ModelForge.

Base pinned at 1d4bf0f2. Quantized from our own bf16 abliterated master, not from someone

else's quant -- so the ladder is a single lineage, not a requant chain.