---

license: apache-2.0

base_model: huihui-ai/Huihui-MiniCPM5-2B-abliterated

tags:

- llama-cpp

- gguf

- minicpm

- quantized

- abliterated

language:

- en

pipeline_tag: text-generation

library_name: gguf

---

Huihui-MiniCPM5-2B-abliterated — GGUF Quantizations

This repository contains GGUF quantizations of huihui-ai/Huihui-MiniCPM5-2B-abliterated.

---

📦 Available Quantizations

| File | Bits | Size (approx.) | Use case |

|------|------|----------------|----------|

| model_f16.gguf | 16-bit | ~5.0 GB | Maximum quality, reference |

| model_q8_0.gguf | 8-bit | ~2.7 GB | Near-lossless, high VRAM |

| model_q6_k.gguf | 6-bit | ~2.1 GB | Excellent quality |

| model_q5_k_m.gguf | 5-bit | ~1.8 GB | Great quality/size balance |

| model_q5_k_s.gguf | 5-bit | ~1.7 GB | Slightly smaller than K_M |

| model_q4_k_m.gguf | 4-bit | ~1.5 GB | Recommended default ✅ |

| model_q4_k_s.gguf | 4-bit | ~1.4 GB | Smaller 4-bit variant |

| model_q3_k_l.gguf | 3-bit | ~1.2 GB | Low VRAM, decent quality |

| model_q3_k_m.gguf | 3-bit | ~1.1 GB | Balanced 3-bit |

| model_q3_k_s.gguf | 3-bit | ~1.0 GB | Minimum 3-bit |

| model_q2_k.gguf | 2-bit | ~0.8 GB | Extreme compression |

---

🚀 Usage

llama.cpp

./llama-cli -m model_q4_k_m.gguf -p "Your prompt here" -n 512

LM Studio

Download any .gguf file and load it directly in LM Studio.

Ollama

ollama run hf.co/tinyopsec/Huihui-MiniCPM5-2B-abliterated-GGUF:Q4_K_M

Python (llama-cpp-python)

from llama_cpp import Llama

llm = Llama.from_pretrained(
    repo_id="tinyopsec/Huihui-MiniCPM5-2B-abliterated-GGUF",
    filename="model_q4_k_m.gguf",
)
output = llm("Your prompt here", max_tokens=512)
print(output["choices"][0]["text"])

---

🔧 Quantization Details

• Source model: huihui-ai/Huihui-MiniCPM5-2B-abliterated
• Tool: llama.cpp
• Base format: F16 GGUF

---

💡 Which quant should I use?

| VRAM | Recommended |

|------|-------------|

| 1 GB | Q2_K |

| 2 GB | Q3_K_M |

| 3 GB | Q4_K_M ✅ |

| 4 GB | Q5_K_M |

| 6 GB | Q6_K |

| 8 GB+ | Q8_0 / F16 |

---

📄 License

Refer to the original model license.