---

license: apache-2.0

base_model: openbmb/MiniCPM5-2B

base_model_relation: quantized

library_name: llama.cpp

pipeline_tag: text-generation

language:

• en
• zh

tags:

• gguf
• llama.cpp
• minicpm5
• quantized
• text-generation
• long-context
• on-device
• tool-calling

---

MiniCPM5-2B GGUF

Community GGUF quantizations of

openbmb/MiniCPM5-2B.

<div align="center" style="background-color:#f59e0b;color:#ffffff;padding:16px 20px;border-radius:10px;line-height:1.7;">

☕ If this GGUF made your day easier, a coffee would make mine.<br>

<a href="https://ko-fi.com/ngquocvinh" style="color:#ffffff;"><strong style="color:#ffffff;">Send a coffee ☕</strong></a><br>

I build and test these releases myself. Your coffee helps keep me going.<br>

Thank you for supporting this work.

</div>

About MiniCPM5-2B

MiniCPM5-2B is OpenBMB's dense 2B-class model for local assistants, coding

agents, tool-use workflows, reasoning, and resource-constrained deployment. It

uses the standard LlamaForCausalLM architecture, has 2,516,756,480 parameters,

42 layers, GQA with 16 query / 2 KV heads, and a native 131,072-token context

length. See the official model card

for the upstream capabilities, evaluation results, datasets, and supported

ecosystem.

This is a quantization-only release. No training, fine-tuning, merging, or

weight modification other than BF16-to-GGUF conversion and quantization was

performed. The source BF16 checkpoint was converted directly from the locked

upstream revision; no quantized file was used as input to another quantization

rung.

The upstream chat template is included as chat_template.jinja

and is also stored in the GGUF metadata. The official template supports

thinking and XML-style tool calls, but tool-call behavior is not certified by

this release's smoke test.

Fidelity measurements

The table below compares each published GGUF with the BF16 reference on a

held-out WikiText pilot: 16 chunks from wiki.test.raw and 16 chunks from

wiki.valid.raw, with a 4,096-token context and the same llama.cpp runtime.

Values are averaged across the two splits. Lower Mean KLD, ΔPPL, and RMS Δp,

and higher Top-1 agreement, indicate closer next-token behavior to BF16. The

BF16 reference mean PPL was 11.7502 in this pilot.

| File | Mean KLD ↓ | Top-1 vs BF16 ↑ | ΔPPL | RMS Δp |

|---|---:|---:|---:|---:|

| MiniCPM5-2B-Q8_0.gguf | 0.001514 | 97.792% | +0.133% | 0.999% |

| MiniCPM5-2B-Q6_K.gguf | 0.006022 | 95.828% | +0.140% | 1.933% |

| MiniCPM5-2B-Q5_K_M.gguf | 0.018555 | 93.028% | +1.247% | 3.430% |

| MiniCPM5-2B-Q5_1.gguf | 0.018927 | 92.770% | +1.003% | 3.460% |

| MiniCPM5-2B-Q5_K_S.gguf | 0.021076 | 92.497% | +1.301% | 3.640% |

| MiniCPM5-2B-Q4_K_M.gguf | 0.063265 | 87.537% | +3.384% | 6.272% |

| MiniCPM5-2B-IQ4_NL.gguf | 0.072267 | 86.721% | +4.668% | 6.721% |

| MiniCPM5-2B-IQ4_XS.gguf | 0.074946 | 86.399% | +5.338% | 6.892% |

| MiniCPM5-2B-Q3_K_L.gguf | 0.160423 | 80.606% | +16.239% | 10.157% |

| MiniCPM5-2B-Q3_K_M.gguf | 0.182391 | 79.684% | +17.441% | 10.824% |

| MiniCPM5-2B-IQ3_M.gguf | 0.197723 | 78.853% | +15.540% | 11.770% |

| MiniCPM5-2B-IQ3_S.gguf | 0.229139 | 77.404% | +18.694% | 12.638% |

| MiniCPM5-2B-Q2_K.gguf | 0.741842 | 62.160% | +88.959% | 22.302% |

| MiniCPM5-2B-Q2_K_S.gguf | 1.054076 | 55.437% | +156.195% | 26.530% |

| MiniCPM5-2B-IQ2_XS.gguf | 2.119478 | 41.038% | +632.593% | 37.621% |

| MiniCPM5-2B-Q1_0.gguf | 13.416317 | 0.000% | +47,867,030% | 55.561% |

Q5_K_S is the compact modern Q5 option; Q5_1 is included as a

legacy-format comparison. Q6_K remains the Q6 control in this release.

The compact machine-readable results are available in

reproducibility/quality-summary.tsv,

with the corpus hashes, evaluation profile, and runtime commit recorded in the

reproducibility/manifest.md.

Quick start

With a recent llama.cpp build:

./llama-cli \
  -m MiniCPM5-2B-Q4_K_M.gguf \
  --chat-template-file chat_template.jinja \
  --jinja \
  --reasoning off \
  -p 'Answer briefly in English: What is GGUF and why is it useful for running language models locally?' \
  -n 128 -c 4096 -ngl 99

The upstream model card documents vLLM, SGLang, Transformers, llama.cpp,

Ollama, and LM Studio deployment paths. Use a runtime version that supports

MiniCPM5 and the model's chat template.

Reproducibility and validation

The reproducibility/manifest.md locks the

upstream revision, source hashes, converter/runtime commit, calibration data,

imatrix hash, quantization commands, validation profile, and benchmark profile.

The package also includes the model-specific calibration corpus, combined

imatrix, quantization script, smoke-test script, and compact quality summary.

Published files passed load/generate smoke tests with tensor checking and the

official MiniCPM5 chat template. Raw conversion, quantization, imatrix,

smoke-test, quality-evaluation, and benchmark logs are retained locally and

are not included here.

License and attribution

The upstream model is released under the Apache License 2.0. Preserve the

upstream attribution and LICENSE when redistributing these

derivative files. These are community GGUF quantizations, not an OpenBMB

release or endorsement.

Verify downloaded artifacts with SHA256SUMS.txt.