Abiray/MiniCPM5-2B-GGUF overview

---

license: apache-2.0

base_model: openbmb/MiniCPM5-2B

language:

- en

- zh

library_name: gguf

pipeline_tag: text-generation

tags:

- minicpm

- minicpm5

- llama

- gguf

- llama.cpp

- text-generation

- long-context

- tool-calling

- on-device

- edge-ai

datasets:

- openbmb/Ultra-FineWeb

- openbmb/UltraX-Preview

- openbmb/Ultra-FineWeb-L3

- openbmb/UltraData-Math

- openbmb/UltraData-Code

- openbmb/UltraData-SFT-2605

- openbmb/UltraData-SFT-Agent-2609

- openbmb/UltraData-RL-2609

---

<div align="center">

<img src="https://raw.githubusercontent.com/OpenBMB/MiniCPM/main/assets/minicpm_logo.png" width="500em" />

</div>

<p align="center">

<a href="https://arxiv.org/pdf/2506.07900" target="_blank">MiniCPM Tech Report</a> |

<a href="https://modelbest.feishu.cn/wiki/UtWxwcERfiRIpIkBOjuc3h9tn1D" target="_blank">MiniCPM Wiki (Chinese)</a> |

<a href="https://github.com/OpenBMB/MiniCPM" target="_blank">GitHub Repo</a> |

<a href="https://ultradata.openbmb.cn/" target="_blank">UltraData</a> |

<a href="https://huggingface.co/spaces/openbmb/MiniCPM5-2B-Demo" target="_blank">Online Demo</a>

</p>

<p align="center">

English |

<a href="https://huggingface.co/openbmb/MiniCPM5-2B/blob/main/README-cn.md" target="_blank">中文</a>

</p>

Highlights

This repository provides quantized GGUF weights for MiniCPM5-2B, the second model in OpenBMB's MiniCPM5 series. It is a dense 2B Transformer scaling up the proven recipe for on-device deployment, edge AI, and local inference via llama.cpp, Ollama, and LM Studio.

🏆 2B-class open-source SOTA: MiniCPM5-2B achieves state-of-the-art performance against models of similar size and remains highly competitive with 4B-class architectures across code generation, mathematics, 128k long-context comprehension, tool use, and multi-step agentic workflows.

---

Available GGUF Files

| Quantization | File Name | Size | Recommendation / Use Case |

| :--- | :--- | :--- | :--- |

| Q3_K_M | MiniCPM5-2B-Q3_K_M.gguf | 1.29 GB | Ultra-compact; suitable for tight VRAM or RAM constraints. |

| Q4_K_S | MiniCPM5-2B-Q4_K_S.gguf | 1.50 GB | Fast 4-bit quantization with minimal memory overhead. |

| Q4_K_M | MiniCPM5-2B-Q4_K_M.gguf | 1.56 GB | Recommended: Best balance of speed, perplexity, and footprint. |

| Q5_K_M | MiniCPM5-2B-Q5_K_M.gguf | 1.81 GB | High accuracy; preserves subtle reasoning and code logic. |

| Q6_K | MiniCPM5-2B-Q6_K.gguf | 2.07 GB | High-fidelity 6-bit quantization; near-identical output to BF16. |

| Q8_0 | MiniCPM5-2B-Q8_0.gguf | 2.68 GB | Near-lossless 8-bit quantization for maximal benchmark fidelity. |

---

Quickstart Guide

llama.cpp

Run inference using llama-cli:

llama-cli \
  -m MiniCPM5-2B-Q4_K_M.gguf \
  -p "Who are you? Please briefly introduce yourself." \
  -n 256 \
  -c 4096 \
  --temp 1.0 \
  --top-p 0.95