---

base_model: mrcuddle/Mistral-Heretica-12B

base_model_relation: quantized

quantized_by: AnkitAI

library_name: gguf

pipeline_tag: text-generation

license: other

language:

• en

tags:

• gguf
• llama.cpp
• quantized
• mistral
• merge
• roleplay
• creative-writing

---

Mistral-Heretica-12B-GGUF

!Mistral Heretica 12B GGUF

GGUF quantizations of mrcuddle/Mistral-Heretica-12B.

Original model: a Task Arithmetic merge of Mistral-Nemo-Instruct-2407, Lumimaid-v0.2-12B, and absolute-heresy — tuned for uncensored roleplay and creative writing. 12B params, Mistral architecture.

Quantized with llama.cpp (release b9890). Run these in LM Studio, Ollama, KoboldCpp, SillyTavern, or llama.cpp directly.

Download

| File | Quant | Size | Description |

|---|---|---|---|

| Q8_0 | Q8_0 | 12 GB | Maximum quality, near-lossless. Overkill for most. |

| Q6_K | Q6_K | 9.4 GB | Very high quality, effectively lossless. |

| Q5_K_M | Q5_K_M | 8.1 GB | High quality. Balanced pick. |

| Q4_K_M | Q4_K_M | 7.0 GB | Good quality, best size/speed tradeoff. Recommended default. |

F16 (23 GB) is the unquantized conversion — only needed if you want to re-quantize yourself.

Which file should I choose?

Pick the largest quant that fits in your RAM/VRAM with room to spare (leave ~1–2 GB for context).

• 8 GB RAM/VRAM → Q4_K_M
• 12 GB → Q5_K_M or Q6_K
• 16 GB+ → Q6_K or Q8_0

For roleplay/creative use, Q4_K_M is plenty — quality loss over Q8 is negligible in practice.

Prompt format

Mistral instruct format (primary):

<s>[INST] {prompt} [/INST]

Most front-ends (SillyTavern, LM Studio) apply this automatically when you select a Mistral / Mistral-Nemo template. The Lumimaid component was also trained with ChatML, so ChatML works too — but Mistral format is the safe default.

Recommended settings

Mistral-Nemo (this model's base) is temperature-sensitive — high temperature makes it incoherent. Keep it low:

| Setting | Value | Note |

|---|---|---|

| Temperature | 0.5 – 0.7 | Do not exceed ~1.0. Lower = more coherent, higher = more creative. |

| Min-P | 0.05 | Primary truncation. Leave Top-P/Top-K off if using Min-P. |

| Repetition penalty | 1.05 – 1.1 | Or use DRY (0.8 / 1.75 / 2) if your front-end supports it. |

| Context | up to 16k reliable | Nemo's trained context is 128k but quality degrades well before that. |

Start at temp 0.6 / min-p 0.05 and adjust from there.

Run it

llama.cpp:

llama-cli -m Mistral-Heretica-12B-GGUF-Q4_K_M.gguf --jinja -p "Write a short noir opening."

Download one file with the HF CLI (skip the rest):

hf download AnkitAI/Mistral-Heretica-12B-GGUF \
  Mistral-Heretica-12B-GGUF-Q4_K_M.gguf --local-dir .

Ollama:

ollama run hf.co/AnkitAI/Mistral-Heretica-12B-GGUF:Q4_K_M

Verified

Q4_K_M smoke-tested: loads correctly, follows instructions (valid JSON output), and produces coherent creative prose. ~5–6 tok/s generation on Apple Silicon (M-series).

License

The base model declares no explicit license. This merge inherits terms from its source models — notably Lumimaid-v0.2-12B, which is CC-BY-NC-4.0 (non-commercial). Treat this model as non-commercial unless you verify otherwise with the original authors. These are quantizations only; all model rights belong to the original creators.

Credits

• Original model: mrcuddle
• Merge components: NeverSleep, MuXodious, Mistral AI
• Quantization tooling: llama.cpp

More models: ankitaglawe.com