---

language:

• en

license: apache-2.0

library_name: llama.cpp

tags:

• gguf
• llama.cpp
• minicpm
• minicpm5
• quantized
• text-generation
• chat
• reasoning
• tool-calling
• instruct

base_model:

• openbmb/MiniCPM5-1B

pipeline_tag: text-generation

---

MiniCPM5-1B-GGUF

GGUF quantizations of openbmb/MiniCPM5-1B for use with llama.cpp, Ollama, LM Studio, Jan, OpenWebUI, llama-cpp-python, and other GGUF-compatible runtimes.

Base Model

Base Model: openbmb/MiniCPM5-1B

MiniCPM5-1B is a compact 1B-parameter language model from OpenBMB focused on:

• General chat
• Reasoning
• Coding assistance
• Tool calling
• Long-context workloads
• Edge and local deployment

This repository contains community-generated GGUF conversions and quantizations derived from the original BF16 checkpoint.

---

Available Files

Full Precision

| File | Description |

|--------|--------|

| MiniCPM5-1B-BF16.gguf | BF16 GGUF conversion of the original checkpoint |

Standard Quantizations

| Quant | Recommended Usage |

|---------|---------|

| Q2_K | Smallest footprint |

| Q3_K_S | Extreme memory savings |

| Q3_K_M | Low memory systems |

| Q3_K_L | Better quality than Q3_K_M |

| Q4_0 | Legacy quant |

| Q4_1 | Legacy quant |

| Q4_K_S | Balanced |

| Q4_K_M | Recommended general-purpose quant |

| Q5_K_S | Higher quality |

| Q5_K_M | Near-lossless experience |

| Q6_K | Very high quality |

| Q8_0 | Maximum quality quantized version |

Importance Matrix (IQ) Quantizations

Generated using llama.cpp importance matrix quantization.

| Quant | Recommended Usage |

|---------|---------|

| IQ2_M | Smallest IQ variant |

| IQ3_XXS | Ultra compact |

| IQ3_XS | Compact |

| IQ3_M | Strong quality-to-size ratio |

| IQ4_XS | High quality compact quant |

| IQ4_NL | Best IQ variant in this repository |

---

Recommended Quant

If you are unsure which file to choose:

Most Users

MiniCPM5-1B-Q4_K_M.gguf

Best balance between:

• Quality
• Speed
• Memory usage
• Compatibility

Higher Quality

MiniCPM5-1B-Q5_K_M.gguf

or

MiniCPM5-1B-Q6_K.gguf

Maximum Quality

MiniCPM5-1B-Q8_0.gguf

Best IQ Quant

MiniCPM5-1B-IQ4_NL.gguf

---

Example Usage

llama.cpp

llama-cli \
    -m MiniCPM5-1B-Q4_K_M.gguf \
    -ngl 99 \
    -c 8192

llama-server

llama-server \
    -m MiniCPM5-1B-Q4_K_M.gguf \
    -c 8192 \
    -ngl 99

Ollama

Create a Modelfile:

FROM MiniCPM5-1B-Q4_K_M.gguf

Then:

ollama create minicpm5-1b -f Modelfile
ollama run minicpm5-1b

Python

from llama_cpp import Llama

llm = Llama(
    model_path="MiniCPM5-1B-Q4_K_M.gguf",
    n_ctx=8192,
    n_gpu_layers=-1,
)

print(
    llm(
        "Explain quantum computing in simple terms.",
        max_tokens=256
    )
)

---

Conversion Details

Conversion pipeline:

1. Download original Hugging Face model.
2. Convert to BF16 GGUF using llama.cpp.
3. Generate importance matrix for IQ quantization.
4. Generate standard GGUF quantizations.
5. Generate IQ quantizations.
6. Upload all artifacts to Hugging Face.

Tools used:

• llama.cpp
• convert_hf_to_gguf.py
• llama-quantize
• llama-imatrix

---

Hardware Guidance

| Quant | Approximate Use Case |

|---------|---------|

| Q2_K | Very constrained devices |

| Q3_* | Raspberry Pi / older CPUs |

| Q4_K_M | Most laptops and desktops |

| Q5_K_M | Modern systems |

| Q6_K | High-end CPUs and GPUs |

| Q8_0 | Largest RAM footprint |

---

Disclaimer

This repository is a community conversion of the original model and is not an official OpenBMB release.

All model weights, architecture, training methodology, and evaluation results belong to the original authors of MiniCPM5-1B.

If you use this model in research or production environments, please cite and credit the original MiniCPM team.

---

Credits

• OpenBMB for creating MiniCPM5-1B
• ggml-org for llama.cpp
• Hugging Face for model hosting
• Community contributors supporting GGUF deployment

Original model:

https://huggingface.co/openbmb/MiniCPM5-1B