---

base_model: richardyoung/Agents-A1-4B-heretic

base_model_relation: quantized

license: apache-2.0

pipeline_tag: image-text-to-text

tags: [gguf, llama.cpp, ollama, heretic, abliterated, uncensored, imatrix, vision]

---

Agents-A1-4B-Heretic GGUF

GGUF quantizations of richardyoung/Agents-A1-4B-heretic, an abliterated build of InternScience/Agents-A1-4B (InternScience's 4B agent model built on Qwen3.5, tuned for tool use and multi-step tasks) made with Heretic.

Abliteration Results

| Metric | Value |

|---|---|

| Base Refusals | 99/100 |

| Abliterated Refusals | 11/100 |

| Refusal Reduction | 89% |

| KL Divergence | 0.060 |

The very low KL divergence (0.060, far below the 0.5 “damage” threshold) means the model retains essentially all of its original capabilities and coherence.

Quick Start

With Ollama

ollama run richardyoung/agents-a1-4b-heretic
# or straight from Hugging Face
ollama run hf.co/richardyoung/Agents-A1-4B-heretic-GGUF

With llama.cpp

huggingface-cli download richardyoung/Agents-A1-4B-heretic-GGUF \
    --include "*Q4_K_M*" --local-dir ./models

./llama-cli -m ./models/*Q4_K_M*.gguf -ngl 99

With Python (llama-cpp-python)

from llama_cpp import Llama

llm = Llama.from_pretrained(
    repo_id="richardyoung/Agents-A1-4B-heretic-GGUF",
    filename="*Q4_K_M*",
    n_gpu_layers=-1,
)

output = llm.create_chat_completion(
    messages=[{"role": "user", "content": "Explain abliteration in simple terms."}]
)
print(output["choices"][0]["message"]["content"])

Image input (llama.cpp)

huggingface-cli download richardyoung/Agents-A1-4B-heretic-GGUF --include "*Q4_K_M*" "mmproj*" --local-dir ./models
./llama-mtmd-cli -m ./models/Agents-A1-4B-heretic-Q4_K_M.gguf --mmproj ./models/mmproj-Agents-A1-4B-heretic-F16.gguf \
    --image page.png -p "Describe this image."

Available Quantizations

| Quantization | Size | Use Case |

|---|---|---|

| IQ3_M | 2.2 GB | Smallest |

| IQ4_XS | 2.5 GB | Great quality-size balance |

| Q4_K_M | 2.7 GB | Recommended — good balance |

| Q5_K_M | 3.1 GB | Higher quality |

| Q6_K | 3.5 GB | Very high quality |

| Q8_0 | 4.5 GB | Near-lossless |

| mmproj-Agents-A1-4B-heretic-F16 | 0.67 GB | Vision projector (needed for image input) |

| Agents-A1-4B-heretic-imatrix.gguf | – | Importance matrix used for the IQ and Q4_K_M/Q5_K_M quants (calibration_datav3) |

What is Abliteration?

Abliteration removes the "refusal direction" from a model's residual stream — a surgical modification that disables safety refusals without retraining. See Refusal in Language Models Is Mediated by a Single Direction. Heretic automates it: it searches ablation parameters with an optimizer (200 trials here) and keeps the trials that best trade off fewer refusals against KL divergence from the original model.

Intended Use

Research, creative writing, education on alignment techniques, and unrestricted local inference. This model will comply with requests the original refuses; use responsibly and in accordance with applicable laws. License: Apache-2.0, inherited from the base model.

Other Models by richardyoung

• New Heretic models (reproducible, listed in the Heretic Grimoire): LFM2.5-1.2B | Granite-4.2-3B | Granite-4.2-8B | Olmo-3-7B-Think | Phi-4-mini | DeepSeek-R1-0528-Qwen3-8B | Agents-A1-4B | FrogNano-4B
• Abliterated/Uncensored models: Qwen2.5-7B | Qwen3-14B | DeepSeek-R1-32B | Qwen3-8B
• OCR & Vision: MiniCPM-V-4.6 GGUF | dots.mocr GGUF | Unlimited-OCR GGUF | olmOCR GGUF | LightOnOCR-3-4B GGUF | WeVisDoc-4B GGUF | Xiaomi-OCR-0 GGUF
• MLX quantizations (Apple Silicon): Kimi-K2 series | olmOCR MLX
• Healthcare/Medical: Synthea 575K patients dataset | CardioEmbed
• Research: LLM Instruction-Following Evaluation (arxiv:2510.18892)

---

Built & maintained by Richard Young · DeepNeuro