---

language:

- en

- ru

license: apache-2.0

library_name: gguf

base_model:

- WaveCut/Qwythos-9B-v2-Heretic

base_model_relation: quantized

tags:

- heretic

- uncensored

- abliteration

- gguf

- llama.cpp

- qwen3.5

pipeline_tag: text-generation

---

Qwythos-9B-v2-Heretic-GGUF

GGUF quantizations of WaveCut/Qwythos-9B-v2-Heretic — the Heretic-decensored version of empero-ai/Qwythos-9B-v2. Use these with llama.cpp, Ollama, LM Studio, KoboldCpp, or any other GGUF runtime.

Files

| File | Format | ~Bits/weight | Size | Recommended use |

|---|---|---|---|---|

| Qwythos-9B-v2-Heretic.Q4_K_M.gguf | K-quant (mixed) | 4.5 | ~5.6 GB | Best 4-bit default — fits in ~6.5 GB VRAM, balanced quality/size |

| Qwythos-9B-v2-Heretic.Q5_K_M.gguf | K-quant (mixed) | 5.5 | ~6.5 GB | Higher quality 5-bit, ~7.5 GB VRAM |

| Qwythos-9B-v2-Heretic.Q6_K.gguf | K-quant (mixed) | 6.6 | ~7.4 GB | Very close to FP16, ~8.5 GB VRAM |

| Qwythos-9B-v2-Heretic.Q8_0.gguf | 8-bit symmetric | 8.5 | ~9.5 GB | Effectively lossless, ~10.5 GB VRAM |

Quantization recipe

| Step | Tool | Version |

|---|---|---|

| HF → F16 GGUF | convert_hf_to_gguf.py | llama.cpp b9986 (commit 91c631b), run with --no-mtp |

| Quantize | llama-quantize | llama.cpp b9986 (prebuilt linux-x64) |

Note on --no-mtp: the base Qwen3.5 model bundles a multi-token-prediction (MTP) head as block #32. Excluding it via --no-mtp produces a clean 32-block text GGUF that loads correctly in current llama.cpp builds. If you want speculative decoding with the MTP head, run convert_hf_to_gguf.py --mtp separately.

No imatrix (importance matrix) calibration was used — K-quants hold up well without one for this model size, and Q8_0 is format-defined and never benefits from it.

Architecture

Qwen3.5 hybrid — 32 blocks mixing attention and SSM (Mamba-style) layers. Supported in llama.cpp b9986 and later under the qwen35 architecture key.

Usage

# llama.cpp
llama-cli -m Qwythos-9B-v2-Heretic.Q4_K_M.gguf -p "Hello" --chat-template chat_template.jinja

# Ollama (create a Modelfile pointing to the .gguf)
ollama create qwythos-heretic -f Modelfile
ollama run qwythos-heretic

Disclaimer

Uncensored (safety alignment removed via Heretic). The original empero-ai/Qwythos-9B-v2 maintainers are not affiliated with this derivative. Use responsibly.