---

base_model:

• JonathanColetti/Qwen3.8-27B-Uncensored

base_model_relation: quantized

license: apache-2.0

pipeline_tag: text-generation

library_name: gguf

language:

- en

- zh

tags:

- gguf

- uncensored

- abliterated

- qwen3.8

- imatrix

- dynamic-quant

---

!Qwen 3.8 27B UNLEASHED

Qwen 3.8 27B UNLEASHED — GGUF

<div>

<a href="https://huggingface.co/JonathanColetti/Qwen3.8-27B-Uncensored"><img src="https://img.shields.io/badge/base-Uncensored%20bf16-a78bfa?style=flat-square" alt="base"></a>

<a href="https://unsloth.ai/docs/basics/dynamic-3.0-ggufs"><img src="https://img.shields.io/badge/recipe-Unsloth%20Dynamic%203.0-22d3ee?style=flat-square" alt="recipe"></a>

<img src="https://img.shields.io/badge/context-262144-0ea5e9?style=flat-square" alt="context">

<img src="https://img.shields.io/badge/license-Apache%202.0-64748b?style=flat-square" alt="license">

</div>

> ### ✅ All 9 tiers verified and live (2026-08-21)

>

> UD-IQ1_M and UD-IQ2_S were briefly published broken, then withdrawn, and are now

> rebuilt, load-tested, and back. If you downloaded either before 2026-08-21 22:00 UTC,

> re-download. No other tier was ever affected.

>

> If you are quantizing this model yourself, this is the part worth reading. Upstream

> builds low-bit tiers with a different architecture, not just different bit widths:

>

> | | Q3_K_XL and above | IQ1_M / IQ2_S |

> |---|---|---|

> | tensors | 506 | 498 |

> | highest block | 64 (MTP head at q6_k) | 63 (MTP pruned) |

> | block_count | 65 | 64 |

> | nextn_predict_layers | 1 | 0 |

>

> Their imatrix has 496 entries because it matches that pruned topology. Quantizing a

> 65-block f16 against it fails with

> Missing importance matrix for tensor blk.64.attn_k.weight.

>

> --prune-layers 64 alone is not enough — it removes the tensors but leaves the model

> declaring an MTP head, so loading dies on blk.63.nextn.eh_proj.weight not found: a

> structurally valid file containing an unloadable model. You also have to rewrite the

> metadata:

>

> ```bash

> llama-quantize --imatrix <imatrix> --tensor-type-file IQ1_M.txt \

> --prune-layers 64 \

> --override-kv qwen35.block_count=int:64 \

> --override-kv qwen35.nextn_predict_layers=int:0 \

> model-f16.gguf out.gguf IQ1_M 12

> ```

>

> Every file here is now gated on exit code → magic bytes → an actual load test before

> upload. Header checks alone let two broken builds through; only loading the model catches it.

Uncensored Qwen3.8-27B, quantized with per-tensor dynamic bit allocation.

Most uncensored GGUFs are uniform quants — every layer gets the same treatment. These use a

per-tensor type map, so sensitive tensors keep precision while tolerant ones compress hard.

The result: a Q3 that outperforms a Q4 3.3 GB larger, with full 250k-token retrieval.

> Provenance, stated plainly. The per-tensor quant map was **extracted from

> unsloth/Qwen3.8-27B-GGUF**

> (Unsloth Dynamic 3.0) by reading the

> tensor-type table from their published files, then applied to

> JonathanColetti/Qwen3.8-27B-Uncensored

> weights with that repo's imatrix. The recipe is Unsloth's work, not ours — we did not

> independently derive it. All upstream projects are Apache 2.0. See Credits.

---

Measurements

Every number below is from one machine, one harness, same session — RTX 4090 24 GB,

llama.cpp with DFlash2 speculative decoding, q4_0 KV cache, -c 262144,

wikitext-2 perplexity at 60 chunks.

⚠️ Do not compare these PPL values against numbers from other setups. Chunk count and

harness change the absolute scale; only within-table comparisons are meaningful.

| Quant | Size | PPL ↓ | Median tok/s | Needle retrieval | Uncensored |

|---|---|---|---|---|---|

| Unleashed UD-IQ4_XS | 14.3 GB | 6.3502 | — | (pending) | ✅ |

| Unleashed UD-Q3_K_XL | 13.2 GB | 6.4036 | 112.8 | ✅ 250,806 tok | ✅ |

| Unleashed UD-IQ3_XXS | 11.0 GB | 6.4818 | — | (pending) | ✅ |

| unsloth UD-Q3_K_XL (ref) | 12.24 GB | 6.3993 | 110.7 | ✅ 250,806 tok | ❌ |

| uncensored IQ4_XS (ref) | 15.1 GB | 6.4149 | 107.3 | 32k | ✅ |

| unsloth UD-Q4_K_XL (ref) | 16.7 GB | 6.4181 | 62.6 | 4k | ❌ |

| uniform Q3_K_M + imatrix (ref) | 12.57 GB | 6.5316 | 84.8 | ✅ 258,900 tok | ✅ |

| unsloth UD-Q2_K_XL (ref) | 9.15 GB | 6.6469 | — | — | ❌ |

Tiers marked (pending) are uploaded and usable; their full speed/retrieval sweep is still running.

What the table says:

• The dynamic Q3 beats every Q4 tested, including a Q4 2.2 GB larger.
• Against uniform Q3 on identical weights: −0.13 PPL and +33% throughput for 0.6 GB more.
• Quality matches Unsloth's own Q3 within error (Δ 0.0043, σ ≈ 0.045) — call it a tie,

with the difference being uncensored weights at ~1 GB more.

• Q2 is the floor. The dynamic recipe halves the damage of a naive Q2 but still costs

~3.8%. Q3 is where quality holds.

Speed: what you should actually expect

Measured on real traffic (n=1,696 requests, RTX 4090, Q3_K_XL, dflash2 draft). **Throughput

depends far more on how many tokens you generate than on context depth**, because every

request pays fixed overhead (prefill, sampler init, draft warm-up) that only amortises over

a long generation:

| Tokens generated | n | Median tok/s |

|---|---|---|

| 1-50 (tool calls, short acks) | 1,289 | 24.9 |

| 51-200 | 97 | 32.6 |

| 201-600 | 191 | 43.6 |

| 601-1500 | 61 | 46.9 |

| 1500+ (long prose, code) | 58 | 56.3 |

Peak observed on a single request: ~195 tok/s. Do not plan around peak.

If you are running an agent that mostly emits short tool calls, expect the ~25 tok/s end of

that table, not the top. That is not a defect of this quant - it is the fixed cost of a

27B model per request, and it applies to any GGUF of this model. Speculative decoding needs

a few tokens to build acceptance momentum, so very short replies finish before the draft

warms up.

Lowering context does not fix this. Only 17 of 65 layers are full-attention (the rest are

linear/DeltaNet), so KV work is a small share of per-token cost. We measured 61 tok/s median at

16-64k context versus 30 tok/s at <4k - deeper context was faster, because those requests

generated more tokens. Context depth is not the lever; generation length is.

Retrieval at depth

Exact-string needle recovered from a 250,806-token context (98% window occupancy):

| Context depth | Prompt tokens | Result | Gen tok/s |

|---|---|---|---|

| 32k | 31,265 | ✅ found | 50.4 |

| 120k | 119,779 | ✅ found | 60.6 |

| 250k | 250,806 | ✅ found | 40.4 |

---

Files

All nine tiers, same recipe, same weights, plus a tested vision projector.

Sizes are on-disk (decimal GB).

| File | Size | Fits (weights + 262k ctx) | Notes |

|---|---|---|---|

| UD-IQ1_M | 6.8 GB | 8 GB card | ⚠️ scores at chance (25.8% MMLU) — published for completeness, not for use |

| UD-IQ2_S | 8.5 GB | 12 GB card | smallest usable tier — 70.8% MMLU, passes retrieval |

| UD-Q2_K_XL | 9.9 GB | 12 GB card | best sub-10GB option |

| UD-IQ3_XXS | 11.0 GB | 16 GB card | strong value tier |

| UD-Q3_K_XL | 13.2 GB | 16–24 GB card | recommended — full 262k ctx on a 4090 |

| UD-IQ4_XS | 14.3 GB | 24 GB card | best measured PPL of the ladder |

| UD-Q4_K_M | 16.5 GB | 24 GB card | reduce ctx or use q8_0 KV |

| UD-Q5_K_M | 19.8 GB | 24 GB @ ~131k ctx | quality ceiling territory |

| UD-Q6_K | 22.1 GB | 24 GB @ ~65k ctx | diminishing returns above Q4 |

VRAM math for a 24 GB card: weights + ~5 GB KV (262k @ q4_0) + ~2 GB draft model +

compute buffers. The KV cache is small for a 27B because Qwen3.8 is hybrid DeltaNet —

only 17 of 65 layers are full attention.

Quick start

llama-server -m Qwen3.8-27B-Unleashed-UD-Q3_K_XL.gguf \
  -ngl 999 -c 262144 -fa on --cache-type-k q4_0 --cache-type-v q4_0 --jinja

Charts

!Quality vs size

!Perplexity is not capability

!Throughput by generation length

!Speed vs size

!Context retrieval

Which file do I want?

| VRAM | File | Notes |

|---|---|---|

| 24 GB | UD-Q3_K_XL | recommended — best quality/size, full 256k ctx |

| 24 GB | UD-Q4_K_M / UD-Q5_K_M | if you want headroom over ctx |

| 16 GB | UD-Q3_K_XL | reduce -c, or use q4_0 KV |

| 12 GB | UD-Q2_K_XL | measurable quality cost — see table |

| 8 GB | UD-IQ2_S | the smallest tier that still works — avoid IQ1_M |

---

How the base model ranks

External context for the base model (stock Qwen3.8-27B, bf16 — not our quants):

• Artificial Analysis (Aug 2026): ranks alongside DeepSeek V4 and GPT-5.6-class models on the

Intelligence Index — discussion ·

charts

• "Beats all medium models (40B–150B); same score as DeepSeek V4 Flash" — HN summary of AA data
• Hybrid DeltaNet architecture: only 17 of 65 layers are full attention → KV cache at 262k ctx is

~5 GB (q4_0), which is why full context fits on a 24 GB card at all

Our quants inherit these capabilities minus quantization loss (measured above).

Optimal settings — these matter

| Setting | Value | Why |

|---|---|---|

| temperature | 1.0 | Qwen3.8 published default; pairs with top_p 0.95 / top_k 20 |

| top_p / top_k / min_p | 0.95 / 20 / 0.0 | stock Qwen3.8 recommendation |

| repeat_penalty | 1.0 | K-quants at this size don't loop; leave it off |

| KV cache | q4_0 | required for 262k ctx on 24 GB; ≤131k ctx can use q8_0 |

| reasoning effort | low default, raise per-task | thinking burns output-token budget fast |

| enable_thinking | off for code-heavy tasks | reasoning chains eat the output budget |

Which tier should you actually use?

Every tier was measured on the same box (RTX 4090, llama.cpp-dflash2) on the same day:

wiki.test perplexity, generation speed, MMLU (10 questions x 57 subjects = 570, greedy,

thinking disabled), needle-in-haystack retrieval, and a 20-prompt refusal battery.

| Tier | Size | PPL ↓ | tok/s | MMLU ↑ | Needle retrieval | Refusal |

|---|---|---|---|---|---|---|

| UD-IQ1_M | 6.8 GB | 8.4493 | 77.0 | 25.83% ⚠️ | ❌ failed @32k | 0.0% |

| UD-IQ2_S | 8.5 GB | 7.0195 | 66.6 | 70.83% | ✅ | 0.0% |

| UD-Q2_K_XL | 9.9 GB | 6.6367 | 68.2 | 70.83% | ✅ | 0.0% |

| UD-IQ3_XXS | 11.0 GB | 6.4818 | 62.0 | 75.00% | ✅ | 0.0% |

| UD-Q3_K_XL | 13.2 GB | 6.4036 | 52.2 | 82.50% | ✅ | 0.0% |

| UD-IQ4_XS | 14.3 GB | 6.3502 | 49.5 | 81.67% | ✅ | 0.0% |

| UD-Q4_K_M | 16.5 GB | 6.3857 | 48.0 | 78.33% | ✅ | 0.0% |

| UD-Q5_K_M | 19.8 GB | 6.4395 | 41.1 | 79.17% | ✅ | 0.0% |

| UD-Q6_K | 22.1 GB | 6.4065 | 37.8 | 79.17% | ✅ | 0.0% |

*MMLU here is n=120 per tier (subject-stratified) for ladder comparison; the headline 82.98%

figure below is the full n=570 run on Q3_K_XL. n=120 carries roughly ±4pp, so treat small

gaps as noise.*

Three findings worth stating plainly

0. UD-IQ1_M scores at random chance. Do not use it. MMLU is 4-way multiple choice, so

25% is the floor for guessing. IQ1_M scored 25.83% — statistically indistinguishable from

noise — with perplexity 32% worse than Q3 and a failed needle retrieval at 32k. It is the

fastest file here (77 tok/s) and it retains no usable knowledge. We are shipping it with the

numbers rather than quietly omitting the tier: if you have an 8 GB card, run UD-IQ2_S

instead — it is 1.7 GB larger, holds 70.83% MMLU, and passes retrieval.

The degradation is not gradual. IQ2_S matches Q2_K_XL exactly (70.83%) while being 1.4 GB

smaller; one step further down falls off a cliff. The usable floor of this model is 2-bit.

1. Perplexity is not capability. UD-IQ4_XS has the best perplexity in the entire ladder

(6.3502) and is not the best model — it scores below UD-Q3_K_XL on MMLU while being

1.1 GB larger and 5% slower. If you are choosing a quant, choose on task benchmarks, not PPL.

2. Nothing above Q3 buys anything. UD-Q6_K is 8.9 GB larger and 28% slower than

UD-Q3_K_XL, and does not score better. Quality rises steeply from Q2 to Q3 and then flattens.

The recommended tier is the knee of every curve, not a compromise.

3. Abliteration survives quantization. Refusal rate is 0.0% at every tier, down to

9.9 GB. Whatever the low-bit tiers cost you, it is not censorship creeping back in.

Capability: MMLU

Measured on this exact quant (UD-Q3_K_XL), not the parent bf16 — so it includes

quantization loss and abliteration loss.

| Model | MMLU (0-shot, 10/subject, n=570) |

|---|---|

| Unleashed UD-Q3_K_XL (13.2 GB, uncensored) | 82.98% (473/570) |

| stock Qwen3.8-27B bf16, same protocol (published by OBLITERATUS) | 85.3% |

Protocol: greedy (temperature 0), thinking disabled, single-letter answer extraction,

10 questions per subject across all 57 MMLU subjects, 0 parse failures.

Reading this honestly: ~2.3 points below stock bf16, and that gap covers both the

abliteration and 3-bit quantization. For a 13.2 GB file that also keeps 262k context and

runs at 25-56 tok/s on one consumer GPU (see the speed section - it depends heavily on

how many tokens you generate), that is the trade being offered. If you need every

last point of benchmark accuracy and don't need uncensored, run the stock bf16.

Refusal behaviour

| Model | Refusal rate |

|---|---|

| Unleashed UD-Q3_K_XL | 0 / 20 (0.0%) |

| stock Qwen3.8-27B | refuses most of this set |

20-prompt boundary battery (security mechanics, drug pharmacology, graphic fiction, social

engineering, morally repugnant character writing, suggestive romance). Scored by refusal-marker

detection on the response opening; every prompt produced a substantive answer (806-1004 chars

on the last three).

Does the abliteration method matter?

We tested this directly. Taking a different uncensored base — [OBLITERATUS

V2](https://huggingface.co/OBLITERATUS/Qwen3.8-27B-OBLITERATED), which uses complementary

blending (SVD + LEACE) rather than the Heretic method — and running it through the identical

UD-Q3_K_XL recipe, same hardware, same day, so the only variable was the base weights:

| Base | MMLU (n=570) | Refusal |

|---|---|---|

| Heretic-abliterated (this repo) | 82.98% (473/570) | 0/20 |

| SVD+LEACE-abliterated, same recipe | 82.98% (473/570) | 0/20 |

Identical scores. The two models disagree on 44 individual questions (92.3% prediction

agreement) — they are genuinely different models that happen to land in the same place.

At 3-bit, the abliteration technique appears to be roughly fungible: different methods, same

measured capability, same refusal behaviour. Worth knowing before you switch bases chasing a

benchmark claim.

Caveat, stated plainly: n=20 is a spot check, not a 800-prompt corpus. The parent model's

card describes refusals as substantially reduced, not eliminated - you will likely find edge

cases. This measures our quant of it, at n=20, on the prompts listed in refusal-results.jsonl.

MTP and vision

• MTP head: present. The nextn.* multi-token-prediction tensors survived abliteration

(re-grafted by the parent repo) and are retained in every quant here (eh_proj at Q6_K).

• Vision: bundled and tested. mmproj-Unleashed-f16.gguf (0.93 GB) is in this repo,

built from the same parent weights as these quants. Every other uncensored GGUF of this

model we are aware of is text-only, because convert_hf_to_gguf.py defaults to text and

drops the 333 vision tensors unless you pass --mmproj.

# vision is one extra flag - the weights themselves are unchanged
llama-server \
  -m Qwen3.8-27B-Unleashed-UD-Q3_K_XL.gguf \
  --mmproj mmproj-Unleashed-f16.gguf \
  -ngl 999 -c 262144 -fa on \
  --cache-type-k q4_0 --cache-type-v q4_0 --jinja

Then post images to /v1/chat/completions with an image_url content part (OpenAI format),

or use llama-mtmd-cli.

What we verified (Q3_K_XL + this projector, single 4090):

| Check | Result |

|---|---|

| loads alongside a 3-bit quant | pass - 15.9 GB VRAM at 32k ctx |

| shape + colour recognition | pass - "a solid red circle" |

| spatial reasoning (left/right) | pass - "LEFT is blue, RIGHT is yellow" |

Tested with synthetic images generated at test time, so ground truth is exact and the

model cannot be scored generously. Abliteration did not break cross-modal alignment -

that was the open risk and it came back clean.

Cost: the projector adds ~0.9 GB resident VRAM. Text-only requests never touch the vision

encoder, so there is no generation-speed penalty for having it loaded; you pay a one-time

prefill per image. Omit --mmproj and behaviour is identical to a text-only build.

Not yet measured by us: image-heavy benchmark accuracy, OCR quality, multi-image prompts.

Running it

Recommended llama.cpp server config — this is the setup all measurements were taken with:

llama-server \
  -m Qwen3.8-27B-Unleashed-UD-Q3_K_XL.gguf \
  -ngl 999 -c 262144 -b 2048 -ub 512 -fa on \
  --cache-type-k q4_0 --cache-type-v q4_0 \
  --cache-reuse 256 \
  --jinja --reasoning-effort medium \
  --temperature 1 --top-p 0.95 --top-k 20 --min-p 0.0 \
  --presence-penalty 0.0 --repeat-penalty 1.0

Speculative decoding with DFlash2 (optional, big win)

Every speed number on this card was measured with a DFlash2 draft model attached. It is not

part of these files - it is a separate ~2 GB model that predicts several tokens ahead, which the

main model then verifies in one pass. On our runs it accepted 48.6% of drafted tokens with a

mean accepted run of 3.92 tokens. Note that acceptance on real agent traffic is lower

(median ~0.37-0.51) than on benchmark prompts.

Get the draft model from

spiritbuun/Qwen3.8-27B-DFlash2 (Q8_0, ~2 GB) and add:

  -md Qwen3.8-27B-DFlash2-Q8_0.gguf -ngld 999 \
  --spec-type draft-dflash --spec-draft-n-max 6 --spec-draft-p-min 0.75 \
  --spec-draft-type-k q4_0 --spec-draft-type-v q4_0

Tuning note: we A/B'd the draft parameters. n_max=6, p_min=0.75 beat both n_max=10, p_min=0.55

(41% acceptance) and n_max=10, p_min=0.40 (30% acceptance). Higher p_min wins here - drafting

more aggressively costs more than it gains. Without a draft model everything still works, just

slower; subtract roughly a third of the throughput.

Notes:

• --cache-reuse 256 matters more than it looks. With a stable system prompt, throughput at

120k depth measured higher than at 32k — prefix reuse amortizes the prefill.

• q4_0 KV cache is what makes 262k fit in 24 GB. At f16 KV you will OOM well before that.
• Sampler values are Qwen3.8's published recommendations, not tuned by us.

---

Method

1. Read the tensor-type table from each published Unsloth UD GGUF header (HTTP range request —

metadata only, no full download).

1. Emit a --tensor-type-file map, one tensor=type line per quantized tensor (506 for Q3_K_XL).
2. Run llama-quantize against jonathan's uncensored f16 with his published imatrix and that map.

The map is non-obvious and worth looking at. Across every tier from IQ1_S to Q8_K_XL, the

~96 ssm_alpha / ssm_beta tensors are held at Q8_0 — even in the 1-bit builds. Those gate

Qwen3.8's DeltaNet path. Our own hand-rolled attempt (uniform Q3_K on those tensors, heavier

attention) scored 6.5865 — the worst result we measured. The specific per-layer allocation

is the contribution, and it is Unsloth's.

Limitations

• Uncensored ≠ unaligned. Refusal behaviour is substantially reduced, not eliminated.

See jonathan's card for measured refusal rates.

• PPL is a weak proxy for task quality. A 16-task BenchLoop coding

suite (local-first LLM benchmark harness) scored identically (14/16) across every quant tested

here — it did not discriminate at this tier. Treat PPL as a relative signal.

• Retrieval was tested with exact-string needles, which is the easy case. Multi-hop reasoning at

250k was not measured.

• Single-machine numbers. No cross-hardware validation.
• Vision: use the mmproj from the upstream repo; not included here.

Credits

• JonathanColetti — the uncensored base weights

(abliteration via Heretic, MTP head re-grafted and verified)

and the imatrix used for these quants. None of this exists without that work.

• Unsloth — the Dynamic 3.0 per-tensor

quantization recipe. We extracted and reapplied their map; the method and the analysis behind

it are entirely theirs. If you can use the censored base, use their files directly.

• Qwen — the base model.
• llama.cpp — quantization and inference.

Apache 2.0, inherited from all upstream components. Unsloth and Qwen names are used descriptively

for attribution only; no affiliation or endorsement implied.