---

license: llama3.2

base_model: DavidAU/Llama-3.2-3B-Instruct-heretic-ablitered-uncensored

tags:

- llama

- llama-3

- gguf

- quantized

- abliterated

- uncensored

language:

- en

pipeline_tag: text-generation

library_name: gguf

---

Llama-3.2-3B-Instruct-heretic-ablitered-uncensored — GGUF Quantizations

GGUF quantizations of DavidAU/Llama-3.2-3B-Instruct-heretic-ablitered-uncensored, an abliterated and uncensored version of Meta's Llama 3.2 3B Instruct model. Refusal mechanisms have been removed while preserving the model's general instruction-following capabilities.

---

Available Quantizations

| File | Bits | Approx Size | Use Case |

|------|------|-------------|----------|

| model_f16.gguf | 16 | ~6.4 GB | Maximum quality, reference |

| model_q8_0.gguf | 8 | ~3.4 GB | Near-lossless, recommended if RAM allows |

| model_q6_k.gguf | 6 | ~2.6 GB | High quality, good balance |

| model_q5_k_m.gguf | 5 | ~2.3 GB | Great quality, recommended |

| model_q5_k_s.gguf | 5 | ~2.2 GB | Slightly smaller Q5 variant |

| model_q4_k_m.gguf | 4 | ~2.0 GB | Best quality/size ratio ⭐ |

| model_q4_k_s.gguf | 4 | ~1.9 GB | Smaller Q4 variant |

| model_q3_k_l.gguf | 3 | ~1.7 GB | Low RAM, acceptable quality |

| model_q3_k_m.gguf | 3 | ~1.6 GB | Lower RAM |

| model_q3_k_s.gguf | 3 | ~1.5 GB | Minimal RAM |

| model_q2_k.gguf | 2 | ~1.3 GB | Extreme compression, lowest quality |

---

VRAM / RAM Requirements

| Quant | RAM Required | Fits On |

|-------|-------------|---------|

| F16 | ~7 GB | 8 GB GPU / 16 GB RAM |

| Q8_0 | ~4 GB | 6 GB GPU / 8 GB RAM |

| Q6_K | ~3 GB | 4 GB GPU / 8 GB RAM |

| Q5_K_M | ~2.5 GB | 4 GB GPU / 8 GB RAM |

| Q4_K_M | ~2.2 GB | 4 GB GPU / 6 GB RAM |

| Q3_K_M | ~1.8 GB | CPU / 4 GB RAM |

| Q2_K | ~1.5 GB | CPU / 4 GB RAM |

---

Usage

llama.cpp (CLI)

./llama-cli \
  -m model_q4_k_m.gguf \
  -p "You are a helpful assistant." \
  -cnv \
  --temp 0.7 \
  -c 4096

llama.cpp (Server)

./llama-server \
  -m model_q4_k_m.gguf \
  --port 8080 \
  -c 4096

llama-cpp-python

from llama_cpp import Llama

llm = Llama(
    model_path="model_q4_k_m.gguf",
    n_ctx=4096,
    n_threads=8,
)

response = llm.create_chat_completion(
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Tell me about yourself."}
    ],
    temperature=0.7,
    max_tokens=512,
)
print(response["choices"][0]["message"]["content"])

Ollama

ollama run hf.co/tinyopsec/Llama-3.2-3B-Instruct-heretic-ablitered-uncensored-GGUF:Q4_K_M

LM Studio

Search for tinyopsec/Llama-3.2-3B-Instruct-heretic-ablitered-uncensored-GGUF in the LM Studio model browser and download your preferred quant.

---

About the Original Model

DavidAU/Llama-3.2-3B-Instruct-heretic-ablitered-uncensored is based on Meta's Llama 3.2 3B Instruct with abliteration applied — a technique that surgically removes refusal behavior by identifying and neutralizing the model's "refusal direction" in the residual stream. The result is a model that follows instructions without built-in content restrictions, while maintaining the original's language understanding and reasoning quality.

Key characteristics:

• Architecture: LlamaForCausalLM
• Parameters: ~3 billion
• Context length: 128K tokens
• Instruction-tuned, chat-optimized
• Abliterated — no refusal behavior
• Language: English

---

Recommended Quant

Q4_K_M — best balance of quality and size for most users. Use Q5_K_M or Q8_0 if you have extra RAM/VRAM.

---

Credits

• Original model: DavidAU
• Base model: meta-llama/Llama-3.2-3B-Instruct
• Quantization: tinyopsec using llama.cpp