---

license: apache-2.0

base_model: philbert440/Qwen3.8-27B-Uncensored-Aggressive

base_model_relation: quantized

library_name: llama.cpp

pipeline_tag: text-generation

tags:

- gguf

- qwen

- qwen3.8

- qwen35

- llama.cpp

- quantized

- imatrix

- speculative-decoding

- mtp

- uncensored

- abliterated

widget:

- text: "Explain speculative decoding in simple terms."

- text: "Write a Python function that merges two sorted linked lists."

model-index:

- name: Qwen3.8-27B-Uncensored-Aggressive i1 IQ4_XS Smaller GGUF

results:

- task:

type: text-generation

dataset:

name: WikiText-2

type: wikitext

metrics:

- name: Perplexity

type: perplexity

value: 6.5848

---

Qwen3.8-27B-Uncensored-Aggressive · i1 IQ4_XS Smaller GGUF

中文版本：README.zh-CN.md

Model overview

This is a purpose-built, mixed-quantization llama.cpp release of philbert440/Qwen3.8-27B-Uncensored-Aggressive. Its goal is to make this 27B qwen35 model practical on 16 GB-class consumer GPUs, while retaining a separate MTP draft model for users who want speculative decoding.

The design is not a generic low-bit conversion. It spends bits unevenly: the dense FFN projections are compressed more aggressively, while selected attention tensors and the output head keep higher precision. The result is a 13.34 GB main GGUF rather than a uniformly quantized model at the same settings.

| Artifact | Size | Role |

| --- | ---: | --- |

| Qwen3.8-27B-Uncensored-Aggressive-i1-IQ4_XS-Smaller.gguf | 13.34 GB | Required 27B target model |

| mtp-Qwen3.8-27B-Uncensored-Aggressive-Q4_0.gguf | 1.68 GB | Optional MTP draft model |

| Both files | 15.02 GB | Weight files before runtime and KV-cache allocations |

The model has 64 transformer blocks and a native 262,144-token context limit. This repository is for text generation; the source family is multimodal, but a compatible vision mmproj is not included.

> [!IMPORTANT]

> “16 GB-class” describes the hardware target and the quantization trade-off, not a guarantee that the main model, the MTP draft, and a large KV cache can all be fully offloaded to a 16 GB GPU. The two weight files alone total about 15.02 GB. For long contexts on a 16 GB card such as an RTX 4080 or RTX 5080, use partial GPU offload, skip the MTP sidecar, reduce context, or use more VRAM. Actual headroom also depends on runtime buffers and backend overhead.

Run it

Use a recent llama.cpp build and the chat template stored in the GGUF.

# Main model only
llama-server \
  -m Qwen3.8-27B-Uncensored-Aggressive-i1-IQ4_XS-Smaller.gguf \
  -ngl 999 \
  --jinja

# Main model with the optional MTP draft model
llama-server \
  -m Qwen3.8-27B-Uncensored-Aggressive-i1-IQ4_XS-Smaller.gguf \
  -md mtp-Qwen3.8-27B-Uncensored-Aggressive-Q4_0.gguf \
  -ngl 999 \
  --spec-type draft-mtp \
  --spec-draft-n-max 3 \
  --jinja

-ngl 999 attempts maximum GPU offload. Lower it when VRAM is insufficient; this is especially important when a long context or the MTP file is enabled.

How it works

1. A hybrid main-model quantization

The target starts from IQ4_XS, then changes the three FFN projection families—ffn_gate, ffn_up, and ffn_down—to IQ3_S. Across 64 blocks, that affects 192 FFN tensors. The space saved there is used to avoid uniformly lowering all other tensors.

The verified output distribution is:

| Tensor type | Count | Purpose in the layout |

| --- | ---: | --- |

| F32 | 353 | Small and non-weight tensors |

| IQ3_S | 192 | All FFN gate/up/down projections |

| IQ4_XS | 241 | Main low-bit weight tier |

| Q5_K | 64 | Selected attention tensors |

| Q6_K | 1 | output.weight |

| Total | 851 | — |

No BF16/F16 weight tensors remain in the finished main GGUF. This allocation is the central idea of the release: compress the large FFNs further while preserving more precision where it is most valuable to the final distribution.

2. An optional, separate MTP draft model

MTP is not embedded in the target GGUF. It is released as a separate Q4_0 draft model, so the main model remains usable by itself and users can decide whether faster speculative decoding is worth the additional memory.

During speculative decoding, the draft model proposes tokens and the 27B target verifies them. Draft quantization therefore influences acceptance rate and speed, but an unverified draft token is not emitted as output. Speedup depends on the hardware, context length, offload plan, and acceptance rate; this release does not claim a universal speedup.

How it was made

Source and calibration

The main model was quantized directly from a BF16 GGUF, not re-quantized from an existing low-bit file. It uses the model-specific importance matrix from mradermacher/Qwen3.8-27B-Uncensored-Aggressive-i1-GGUF:

Qwen3.8-27B-Uncensored-Aggressive.imatrix.gguf

Main-model quantization

llama-quantize \
  --imatrix Qwen3.8-27B-Uncensored-Aggressive.imatrix.gguf \
  --tensor-type ffn_down=iq3_s \
  --tensor-type ffn_up=iq3_s \
  --tensor-type ffn_gate=iq3_s \
  Qwen3.8-27B-Uncensored-Aggressive-BF16-noMTP.gguf \
  Qwen3.8-27B-Uncensored-Aggressive-i1-IQ4_XS-Smaller.gguf \
  IQ4_XS

MTP draft quantization

The MTP head was separated from the same BF16 source and quantized independently. --pure prevents automatic promotion of large vocabulary-related tensors; the output head and token embeddings are explicitly kept at Q4_0 to control the sidecar's size.

llama-quantize \
  --pure \
  --output-tensor-type q4_0 \
  --token-embedding-type q4_0 \
  Qwen3.8-27B-Uncensored-Aggressive-MTP-BF16.gguf \
  mtp-Qwen3.8-27B-Uncensored-Aggressive-Q4_0.gguf \
  Q4_0

Verify downloads

c9fdab970822cb72bc2d585b73bec24a5fd68fda1dfa976af3f6e9dd47f8bd1f  Qwen3.8-27B-Uncensored-Aggressive-i1-IQ4_XS-Smaller.gguf
e10857664938dfc50240ec16313b8766e9432c5975a9db386bb1a278720242c4  mtp-Qwen3.8-27B-Uncensored-Aggressive-Q4_0.gguf

sha256sum Qwen3.8-27B-Uncensored-Aggressive-i1-IQ4_XS-Smaller.gguf
sha256sum mtp-Qwen3.8-27B-Uncensored-Aggressive-Q4_0.gguf

Results

The main GGUF was compared with the BF16 source using the same llama.cpp runtime and decoding configuration. MTP was disabled for these measurements.

| Evaluation | BF16 | This release | Difference |

| --- | ---: | ---: | ---: |

| WikiText-2 perplexity | 6.3640 | 6.5848 | +0.2208 (+3.47%) |

| IFEval instruction strict | 88.73% | 88.61% | -0.12 pp |

| IFEval prompt strict | 83.92% | 83.36% | -0.55 pp |

| Same top-token winner, 32 chunks | — | ~90.68% | — |

Probability divergence: mean KLD 0.059965, median KLD 0.024371, and 95th-percentile KLD 0.173735.

The deterministic GSM8K and GPQA runs included individual score increases. That should not be read as quantization improving reasoning ability: low-bit conversion changes greedy generation trajectories, so individual benchmark outcomes can move in either direction. Treat these measurements as release-specific evidence, then evaluate the model on your own workload.

Credits, license, and use

• Source model: philbert440/Qwen3.8-27B-Uncensored-Aggressive
• Importance matrix: mradermacher/Qwen3.8-27B-Uncensored-Aggressive-i1-GGUF
• Quantization and runtime: llama.cpp

This quantized derivative and its documentation are available under the Apache License 2.0; attribution and modification details are in NOTICE. Apache-2.0 does not grant trademark rights.

The source model is uncensored / de-refused. It can produce unsafe, inaccurate, biased, or unsuitable content. Users, redistributors, and deployers are responsible for their own evaluations, safeguards, and compliance with applicable law and the upstream license.