---

license: apache-2.0

base_model: openbmb/MiniCPM5-2B

pipeline_tag: text-generation

language:

- en

- zh

tags:

- minicpm

- minicpm5

- llama

- gguf

- text-generation

- long-context

- tool-calling

- on-device

- edge-ai

---

MiniCPM5-2B — GGUF quants

Full llama.cpp quant sweep of openbmb/MiniCPM5-2B, OpenBMB's dense 2B model for on-device and local deployment. 30 files — the BF16 source plus 29 quants from Q8_0 down to Q1_0 — all cut from the official BF16 release, with the importance matrix computed on the BF16 weights.

The source model uses the standard LlamaForCausalLM architecture — no custom kernels and no model-code fork are needed; these files load directly in llama.cpp, Ollama and LM Studio. (OpenBMB also publishes their own MiniCPM5-2B-GGUF; this repo complements it with the complete quant sweep including the IQ and extreme quants. For vLLM, SGLang or transformers, use the source repo directly.)

Which file should I get?

| File | Size | Use |

| --- | --- | --- |

| MiniCPM5-2B-BF16.gguf | 4.69 GB | Full precision source. Every quant below is cut from this file. |

| MiniCPM5-2B-Q8_0.gguf | 2.50 GB | Effectively lossless. Use when disk and RAM are not the constraint. |

| MiniCPM5-2B-Q6_K.gguf | 1.93 GB | Near-lossless; the last stop before quality becomes measurable. |

| MiniCPM5-2B-Q5_1.gguf | 1.78 GB | Legacy. Prefer Q5_K_M. |

| MiniCPM5-2B-Q5_K_M.gguf | 1.68 GB | Very good quality, noticeably smaller than Q6_K. |

| MiniCPM5-2B-Q5_K_S.gguf | 1.65 GB | Slightly smaller than Q5_K_M for a slight quality cost. |

| MiniCPM5-2B-Q5_0.gguf | 1.65 GB | Legacy. Prefer Q5_K_M. |

| MiniCPM5-2B-Q4_1.gguf | 1.52 GB | Legacy. Prefer Q4_K_M. |

| MiniCPM5-2B-Q4_K_M.gguf | 1.45 GB | The usual default. Best quality-per-byte for most people. |

| MiniCPM5-2B-Q4_K_S.gguf | 1.40 GB | A little smaller than Q4_K_M, a little worse. |

| MiniCPM5-2B-IQ4_NL.gguf | 1.39 GB | Non-linear 4-bit; good on hardware without fast K-quant kernels. |

| MiniCPM5-2B-Q4_0.gguf | 1.39 GB | Legacy round-to-nearest. Prefer Q4_K_M unless a runtime needs this. |

| MiniCPM5-2B-IQ4_XS.gguf | 1.33 GB | Best value at this size; usually beats Q4_K_S while smaller. |

| MiniCPM5-2B-Q3_K_L.gguf | 1.28 GB | Small, with real quality loss. Usable when RAM is tight. |

| MiniCPM5-2B-Q3_K_M.gguf | 1.20 GB | Smaller again; noticeable degradation. |

| MiniCPM5-2B-IQ3_M.gguf | 1.14 GB | Strong 3-bit choice, clearly better than Q3_K_M. |

| MiniCPM5-2B-IQ3_S.gguf | 1.11 GB | Slightly smaller than IQ3_M. |

| MiniCPM5-2B-Q3_K_S.gguf | 1.11 GB | Aggressive. Prefer IQ3_M at a similar size. |

| MiniCPM5-2B-IQ3_XS.gguf | 1.07 GB | Aggressive but coherent. |

| MiniCPM5-2B-IQ3_XXS.gguf | 0.99 GB | Very aggressive; the last coherent step down. |

| MiniCPM5-2B-Q2_K.gguf | 0.97 GB | Very small, heavily degraded. For experimentation. |

| MiniCPM5-2B-Q2_K_S.gguf | 0.92 GB | Smaller than Q2_K, at a further quality cost. |

| MiniCPM5-2B-Q2_0.gguf | 0.87 GB | Extreme, group-64. Included for completeness. |

| MiniCPM5-2B-IQ2_M.gguf | 0.90 GB | The smallest size most people find usable. |

| MiniCPM5-2B-IQ2_S.gguf | 0.85 GB | Below the usual usability line. |

| MiniCPM5-2B-IQ2_XS.gguf | 0.80 GB | Experimental. |

| MiniCPM5-2B-IQ2_XXS.gguf | 0.74 GB | Experimental. |

| MiniCPM5-2B-IQ1_M.gguf | 0.68 GB | Extreme. Expect substantial degradation. |

| MiniCPM5-2B-IQ1_S.gguf | 0.64 GB | Extreme. Expect substantial degradation. |

| MiniCPM5-2B-Q1_0.gguf | 0.50 GB | Extreme. Included for completeness. |

All 30 files combined: 40.15 GB.

Shortcuts: default Q4_K_M (1.45 GB) · lossless Q8_0 (2.50 GB) · smallest usable IQ2_M (0.90 GB) · 3-bit sweet spot IQ3_M (1.14 GB).

Usage

Run straight from the Hub:

llama-cli -hf NANI-Nithin/MiniCPM5-2B-GGUF:Q4_K_M -p "Hello"

Serve an OpenAI-compatible endpoint:

llama-server -hf NANI-Nithin/MiniCPM5-2B-GGUF:Q4_K_M

Or download a file first:

huggingface-cli download NANI-Nithin/MiniCPM5-2B-GGUF MiniCPM5-2B-Q4_K_M.gguf --local-dir .
llama-cli -m MiniCPM5-2B-Q4_K_M.gguf -p "Hello"

These are llama.cpp GGUF files. They are not loaded by vLLM, SGLang or transformers — for those backends use the source repo.

About MiniCPM5-2B

MiniCPM5-2B is the second model in the MiniCPM5 series from OpenBMB, following MiniCPM5-1B. It is a dense 2B Transformer built for local assistants, coding agents, tool-use workflows and reasoning scenarios where a compact footprint matters. Per the source model card:

• 2,516,756,480 parameters (1,981,982,720 non-embedding), 42 layers, GQA with 16 query heads and 2 KV heads
• 131,072-token context with native long-context support
• English and Chinese
• Trained with the UltraData tiered data management recipe (arXiv:2602.09003): base + mid-training on the released web datasets, then 400B tokens of deep-thinking SFT, RL, and On-Policy Distillation (OPD) of 16 RL teacher models into the release checkpoint
• OpenBMB's own evaluation reports 2B-class open-source SOTA with an average score of 53.9 in their comparison set, competitive with 4B-class models, with particular strengths in code reasoning, math, long-context understanding, tool use and agentic tasks

Quantization notes

• Quantized from the official BF16 release on 2026-09-07
• Importance matrix computed on the BF16 weights, so the IQ and sub-Q4 quants benefit from calibration
• All 29 quant types of the standard sweep are present; nothing was skipped for size

Citation

If you use this model, please cite the source work:

@article{minicpm4,
  title={Minicpm4: Ultra-efficient llms on end devices},
  author={MiniCPM, Team},
  journal={arXiv preprint arXiv:2506.07900},
  year={2025}
}

Disclaimer

Per the source model card, outputs are generated text and may be inaccurate, biased, or offensive; responses on sensitive topics are not professional advice. The model is provided AS IS, without warranty of any kind; users must use it only for lawful, compliant and ethical purposes.

License

Apache-2.0, matching the source model.