Carlosian/Gemma-4-12b-it-Abliterated-GGUF overview
Gemma 4 12b it Abliterated — GGUF quant ladder Quantized GGUF builds of Carlosian/Gemma 4 12b it Abliterated https://huggingface.co/Carlosian/Gemma 4 12b it Ab…
Runs locally from ~4.50 GB disk (8 GB VRAM class GPUs with llama.cpp / guIDE).
Repository Files & Downloads
| File | Type | Quantization | Size | Link |
|---|---|---|---|---|
| Gemma-4-12b-it-Abliterated.Q2_K.gguf | GGUF | GGUF | 4.50 GB | Download |
| Gemma-4-12b-it-Abliterated.Q3_K_M.gguf | GGUF | GGUF | 5.67 GB | Download |
| Gemma-4-12b-it-Abliterated.Q3_K_S.gguf | GGUF | GGUF | 5.15 GB | Download |
| Gemma-4-12b-it-Abliterated.Q4_0.gguf | GGUF | GGUF | 6.50 GB | Download |
| Gemma-4-12b-it-Abliterated.Q4_K_M.gguf | GGUF | GGUF | 6.87 GB | Download |
| Gemma-4-12b-it-Abliterated.Q4_K_S.gguf | GGUF | GGUF | 6.54 GB | Download |
| Gemma-4-12b-it-Abliterated.Q5_K_M.gguf | GGUF | GGUF | 7.96 GB | Download |
| Gemma-4-12b-it-Abliterated.Q5_K_S.gguf | GGUF | GGUF | 7.77 GB | Download |
| Gemma-4-12b-it-Abliterated.Q6_K.gguf | GGUF | GGUF | 9.11 GB | Download |
| Gemma-4-12b-it-Abliterated.Q8_0.gguf | GGUF | GGUF | 11.80 GB | Download |
| Gemma-4-12b-it-Abliterated.f16.gguf | GGUF | GGUF | 22.20 GB | Download |
Model Details
| Model ID | Carlosian/Gemma-4-12b-it-Abliterated-GGUF |
|---|---|
| Author | Carlosian |
| Pipeline | text-generation |
| License | other |
| Base model | Carlosian/Gemma-4-12b-it-Abliterated |
| Last modified | 2026-07-20T05:57:32.000Z |
Model README
---
base_model: Carlosian/Gemma-4-12b-it-Abliterated
base_model_relation: quantized
library_name: gguf
pipeline_tag: text-generation
tags:
- gguf
- quantized
- abliterated
- uncensored
- llama.cpp
license: other
---
Gemma-4-12b-it-Abliterated — GGUF quant ladder
Quantized GGUF builds of Carlosian/Gemma-4-12b-it-Abliterated for llama.cpp / Ollama / LM Studio.
| Quant | Typical use |
|-------|-------------|
| Q2_K | smallest / extreme low VRAM |
| Q3_K_S / Q3_K_M | low VRAM |
| Q4_0 / Q4_K_S / Q4_K_M | recommended default (Q4_K_M) |
| Q5_K_S / Q5_K_M | higher quality |
| Q6_K | near-lossless |
| Q8_0 | highest quality quant |
| F16 | full intermediate (large) |
Built with llama.cpp on Thunder Compute (A100). Generated 2026-07-20.
Acknowledgments & method references
The refusal-removal method used to build the base checkpoint draws on:
- Herring, S., Naviasky, J., Malhotra, K. (2026). Targeted Neuron Modulation via Contrastive Pair Search. Nous Research. https://huggingface.co/papers/2605.12290
- Nous Research (2026). llm-abliteration — toolkit for abliteration (norm-preserving & biprojected variants, multi-architecture). github.com/NousResearch/llm-abliteration.
Run Carlosian/Gemma-4-12b-it-Abliterated-GGUF with guIDE
Download guIDE — the AI-native code editor with local LLM inference and 69 built-in tools.
Source: Hugging Face · Compare models