---

license: apache-2.0

language:

- en

- zh

base_model:

- openbmb/MiniCPM5-2B

base_model_quantized: openbmb/MiniCPM5-2B-GGUF

library_name: gguf

pipeline_tag: text-generation

tags:

- gguf

- minicpm

- minicpm5

- llama

- text-generation

- long-context

- tool-calling

- on-device

- edge-ai

- imatrix

- conversational

---

MiniCPM5-2B GGUF

GGUF quantizations of openbmb/MiniCPM5-2B.

Model Description

MiniCPM5-2B is a dense ~2.5B parameter Transformer from the MiniCPM5 series, built for on-device and resource-constrained deployment. LlamaForCausalLM architecture, 131K context, strong at coding, math, tool use, and agentic tasks in its class.

Architecture: LlamaForCausalLM | 42 layers | 2048 hidden | 16 attn heads | 2 KV heads | 130,560 vocab

Quantization

Converted from the official F16 GGUF master using llama.cpp b10842 (CPU-only, no imatrix).

| File | Size | Type | bpw |

|------|------|------|-----|

| MiniCPM5-2B-Q4_K.gguf | 1.56 GB | Q4_K - Medium | ~4.95 |

Q4_K (Mixed): Q4_K base with Q6_K on attention and FFN output tensors (llama.cpp's built-in Q4_K_M large-precision treatment).

Usage

Ollama

ollama run hf.co/peasantsmith/MiniCPM5-2B-GGUF:Q4_K

llama.cpp

llama-cli -m MiniCPM5-2B-Q4_K.gguf -p "Your prompt here" -ngl <layers that fit your VRAM>

Python (llama-cpp-python)

from llama_cpp import Llama
llm = Llama(model_path="MiniCPM5-2B-Q4_K.gguf", n_ctx=8192, n_gpu_layers=-1)
output = llm("Your prompt here", max_tokens=256)
print(output["choices"][0]["text"])

Composition

| Type | Count | Size |

|------|-------|------|

| Q4_K | 253 tensors | 1110 MiB |

| Q6_K | 43 tensors | 445 MiB |

| F32 | 85 tensors | <1 MiB |

Original Model

• Model: openbmb/MiniCPM5-2B
• GGUF source: openbmb/MiniCPM5-2B-GGUF
• License: Apache 2.0