---

license: apache-2.0

base_model: Qwen/Qwen3.6-27B

tags:

- abliteration

- uncensored

- qwen3

- qwen3.6

- hybrid-attention

- deltanet

- vlm

- multimodal

- gguf

- llama-cpp

- ollama

- mtp

model_type: qwen3_5

pipeline_tag: image-text-to-text

library_name: gguf

---

Qwen3.6-27B-abliterated-GGUF

GGUF quantizations of Qwen/Qwen3.6-27B with refusal behavior removed via abliteration. For use with llama.cpp, Ollama, LM Studio, and other GGUF-compatible runtimes.

> Blog post: Abliteration: Uncensoring LLMs via Weight Surgery

Standard Quantizations (No MTP)

Maximum compatibility — works with Ollama, LM Studio, KoboldCPP, llama.cpp out of the box.

| File | Size | RAM Required | Notes |

|------|------|-------------|-------|

| ...-F16.gguf | 53.8 GB | 64+ GB | Full precision |

| ...-Q4_K_M.gguf | 26.9 GB | 32+ GB | Recommended (F16 vision embed) |

| ...-Q4_K_M-Q8.gguf | 19.7 GB | 24+ GB | Q4 language + Q8 vision embed |

MTP Quantizations (Multi-Token Prediction)

Include MTP draft head for speculative decoding. Larger files but faster inference with compatible runtimes.

| File | Size | RAM Required | Notes |

|------|------|-------------|-------|

| ...-MTP-F16.gguf | 54.7 GB | 64+ GB | Full precision + MTP |

| ...-MTP-Q8_0.gguf | 29 GB | 36+ GB | Near-lossless + MTP |

| ...-MTP-Q6_K.gguf | 22.4 GB | 32+ GB | Very high quality + MTP |

| ...-MTP-Q5_K.gguf | 19.5 GB | 24+ GB | Recommended for 48GB + MTP |

| ...-MTP-Q4_K.gguf | 16.8 GB | 20+ GB | Good quality + MTP |

| ...-MTP-Q3_K.gguf | 13.5 GB | 16+ GB | 16GB VRAM + MTP |

| ...-MTP-Q2_K.gguf | 10.9 GB | 12+ GB | 2-bit + MTP |

Vision Encoder

| File | Size | Notes |

|------|------|-------|

| ...-mmproj-f16.gguf | 928 MB | Required for multimodal (image understanding) |

MTP (Multi-Token Prediction)

MTP files include a draft head for speculative decoding — faster token generation:

./llama-server -m Qwen3.6-27B-abliterated-MTP-Q5_K.gguf \
  --jinja --spec-type draft-mtp --spec-draft-n-max 6 -ngl 99

> Runtime compatibility: MTP requires llama-server b9180+. Ollama does not support MTP yet. Use the standard (non-MTP) files for Ollama/LM Studio.

Multimodal (Vision)

The mmproj-f16.gguf file is the vision encoder for image understanding:

./llama-mtmd-cli -m Qwen3.6-27B-abliterated-Q4_K_M.gguf \
  --mmproj Qwen3.6-27B-abliterated-mmproj-f16.gguf \
  -p "Describe this image" --image photo.jpg

Quickstart — Ollama

# Recommended for Apple Silicon 48GB+ (M4 Pro, M4 Max)
ollama run hf.co/Bahushruth/Qwen3.6-27B-abliterated-GGUF:Q4_K_M

# For 24GB systems
ollama run hf.co/Bahushruth/Qwen3.6-27B-abliterated-GGUF:Q4_K_M-Q8

Usage — llama.cpp

huggingface-cli download Bahushruth/Qwen3.6-27B-abliterated-GGUF \
  Qwen3.6-27B-abliterated-Q4_K_M.gguf --local-dir .

./llama-cli -m Qwen3.6-27B-abliterated-Q4_K_M.gguf \
  -p "You are a helpful assistant." \
  --chat-template chatml -cnv -c 262144

Architecture Notes

Qwen3.6-27B is a dense hybrid-attention multimodal model:

• Hybrid attention: 16 blocks of (3x Gated DeltaNet + 1x Gated Attention) = 64 layers
• Parameters: 27B dense (all active per token)
• Hidden size: 5120
• Context: 262K native, extensible to 1M+ via YaRN
• Multimodal: Vision encoder for image understanding
• MTP: Multi-token prediction draft head for speculative decoding

Disclaimer

This model has had safety guardrails removed. Released for research purposes. The creator assumes no responsibility for downstream use.