---

base_model: coder3101/Ministral-3-8B-Reasoning-2512-heretic

base_model_relation: quantized

library_name: gguf

license: apache-2.0

language:

• en
• fr
• es
• de
• it
• pt
• nl
• zh
• ja
• ko
• ar

pipeline_tag: image-text-to-text

tags:

• gguf
• text-generation
• multimodal
• vision
• reasoning
• math
• stem
• agentic
• tool-use
• abliterated
• uncensored
• mistral
• q4_k_m
• iq4_nl
• imatrix
• conversational

---

Ministral-3-8B-Reasoning-2512-heretic — Importance Matrix GGUF

This repository hosts importance-matrix (imatrix) optimized GGUF weights, available in multiple quantization formats, and the associated vision projection matrix for Ministral-3-8B-Reasoning-2512-heretic, quantized from the source floating-point tensors provided by coder3101/Ministral-3-8B-Reasoning-2512-heretic.

About the Model

Ministral-3-8B-Reasoning-2512 is a vision-language model by Mistral AI from the Ministral 3 family, built for edge deployment. It combines an 8.4B language model with a 0.4B vision encoder for multimodal understanding, and is the reasoning post-trained variant — specifically optimised for math, coding, STEM, and complex multi-step reasoning. The model supports multilingual input, native function calling, JSON output, and best-in-class agentic capabilities at its scale.

The heretic suffix denotes post-processing via the Heretic v1.1.0 abliteration framework performed by coder3101, using the direction_index method to suppress refusal vectors while preserving reasoning chains and multimodal capabilities.

---

Model & Architecture Specifications

| Property | Value |

|---|---|

| Base Architecture | Ministral-3 (8.4B LM + 0.4B Vision Encoder) |

| Developed by | Mistral AI |

| Primary Use | Reasoning, math, STEM, vision, agentic tasks |

| Context Window | 262,144 tokens |

| Vision Encoder | 0.4B (integrated, early-fusion) |

| Languages | English, French, Spanish, German, Italian, Portuguese, Dutch, Chinese, Japanese, Korean, Arabic |

| Abliteration Tool | Heretic v1.1.0 |

| Abliteration Method | direction_index (single-direction refusal suppression) |

| Prompt Format | ChatML |

---

Abliteration Parameters

| Parameter | Value |

|---|---|

| direction_index | 15.03 |

| attn.o_proj.max_weight | 1.17 |

| attn.o_proj.max_weight_position | 20.88 |

| attn.o_proj.min_weight | 0.55 |

| attn.o_proj.min_weight_distance | 2.72 |

| mlp.down_proj.max_weight | 1.49 |

| mlp.down_proj.max_weight_position | 23.45 |

| mlp.down_proj.min_weight | 1.40 |

| mlp.down_proj.min_weight_distance | 18.55 |

Abliteration Performance

> [!NOTE]

> The metrics below are self-reported by the original model author (coder3101) and have not been independently reproduced.

| Metric | This model | Original (mistralai/Ministral-3-8B-Reasoning-2512) |

|---|---|---|

| KL divergence | 0.0712 | 0 (by definition) |

| Refusals | 4/100 | 96/100 |

---

Quantization Details

| Property | Value |

|---|---|

| Text Tensor Types | Q4_K_M, IQ4_NL (both with imatrix calibration) |

| Vision Tensors | BF16 (unquantized, preserves visual feature quality) |

| Importance Matrix | bartowski's calibration_datav5.txt |

Quantization Commands

# Q4_K_M
./llama-quantize --imatrix /content/imatrix.dat \
  /content/model-bf16.gguf \
  /content/Ministral-3-8B-Reasoning-2512-heretic-Q4_K_M-imatrix.gguf q4_k_m

# IQ4_NL
./llama-quantize --imatrix /content/imatrix.dat \
  /content/model-bf16.gguf \
  /content/Ministral-3-8B-Reasoning-2512-heretic-IQ4_NL-imatrix.gguf iq4_nl

---

Repository Files

| Filename | Format | Size | llama.cpp Build | Description |

|---|---|---|---|---|

| Ministral-3-8B-Reasoning-2512-heretic-Q4_K_M-imatrix.gguf | Q4_K_M | 5.20 GB | b9803 | Main model weights, imatrix-calibrated |

| Ministral-3-8B-Reasoning-2512-heretic-IQ4_NL-imatrix.gguf | IQ4_NL | 4.94 GB | b9843 | Main model weights, imatrix-calibrated, smaller non-linear 4-bit quant |

| mmproj-Ministral-3-8B-Reasoning-2512-heretic-BF16.gguf | BF16 | 867 MB | b9803 | Vision projector — required for all image inputs |

---

Inference

> [!IMPORTANT]

> The vision projector (--mmproj) must be supplied at runtime whenever image inputs are used. Omitting it disables multimodal capability entirely.

> [!NOTE]

> Mistral AI's recommended sampling temperature for this model is 0.7.

llama.cpp CLI (with image)

./llama-cli \
  -m Ministral-3-8B-Reasoning-2512-heretic-Q4_K_M-imatrix.gguf \
  --mmproj mmproj-Ministral-3-8B-Reasoning-2512-heretic-BF16.gguf \
  -c 8192 \
  -ngl 99 \
  --temp 0.7 \
  --image "path/to/image.jpg" \
  -p "<|im_start|>user\nSolve the problem shown in the image step by step.<|im_end|>\n<|im_start|>assistant\n"

OpenAI-Compatible API Server

./llama-server \
  --host 0.0.0.0 \
  --port 8080 \
  -m Ministral-3-8B-Reasoning-2512-heretic-Q4_K_M-imatrix.gguf \
  --mmproj mmproj-Ministral-3-8B-Reasoning-2512-heretic-BF16.gguf \
  -c 16384 \
  -ngl 99 \
  --flash-attn

---

Prompt Format (ChatML)

<|im_start|>system
You are an expert reasoning assistant. Think step by step.<|im_end|>
<|im_start|>user
Your question or image payload here.<|im_end|>
<|im_start|>assistant

---

Notes & Limitations

• This model is abliterated and will generate content that standard aligned models refuse. Use responsibly and in compliance with applicable laws.
• Designed for edge deployment; fits in 24 GB VRAM at BF16 and under 12 GB when quantized.
• Vision tensors are kept at BF16 to prevent degradation of spatial and symbolic features in diagrams and formulas.
• Imatrix calibration improves perplexity recovery compared to non-imatrix quantization, particularly on low-frequency tokens.
• IQ4_NL produces a smaller file than Q4_K_M and tends to run faster on CPU and ARM devices; imatrix calibration narrows the quality gap between the two formats considerably.