---

tags:

• gguf
• minicpm5
• reasoning
• distillation
• llama.cpp
• ollama
• 2b

license: apache-2.0

base_model: jigs97022/minicpm5-2b-distilled-reasoning

pipeline_tag: text-generation

library_name: llama.cpp

---

MiniCPM5-2B Distilled Reasoning (GGUF)

Quantized GGUF version of jigs97022/minicpm5-2b-distilled-reasoning. Optimized for local CPU/GPU inference with llama.cpp, Ollama, LM Studio, and vLLM.

Model Origin

This model was distilled from three frontier models (Qwen3.8-Max, GLM-5.2, Kimi K3) onto MiniCPM5-2B using 10K high-quality reasoning traces. It excels at structured chain-of-thought problem solving in math, code, logic puzzles, and tool-use scenarios.

Quick Start

Ollama (Recommended)

ollama run hf.co/jigs97022/minicpm5-2b-distilled-reasoning-gguf

LM Studio

1. Search "jigs97022/minicpm5-2b-distilled-reasoning-gguf" in the search bar
2. Download the Q4_K_M quantization
3. Load and chat

llama.cpp / Python

from llama_cpp import Llama

llm = Llama.from_pretrained(

repo_id="jigs97022/minicpm5-2b-distilled-reasoning-gguf",

filename="*q4_k_m.gguf",

n_ctx=4096,

n_gpu_layers=-1 # Set to 0 for CPU-only

)

output = llm.create_chat_completion(

messages=[{"role": "user", "content": "If 3x + 7 = 22, what is x? Show step-by-step reasoning."}],

max_tokens=512,

temperature=0.7

)

print(output["choices"][0]["message"]["content"])

vLLM

vllm serve jigs97022/minicpm5-2b-distilled-reasoning-gguf \

--max-model-len 4096 \

--gpu-memory-utilization 0.8

Quantization Details

| File | Size | Quality | Best For |

|------|------|---------|----------|

| Q4_K_M | ~1.5 GB | Recommended default | Laptop CPU/GPU, balanced speed/quality |

Hardware Requirements

| Quant | RAM/VRAM Min | Inference Speed (T4) |

|-------|-------------|---------------------|

| Q4_K_M | 2 GB | ~40 tok/s |

Chat Template

Uses native MiniCPM5 formatting. Do NOT override the chat template in llama.cpp/Ollama; the model expects its built-in template.

Limitations

• Context window limited to 4096 tokens
• Trained on 10K samples; may underperform full-corpus variant on rare edge cases
• Knowledge cutoff inherited from base MiniCPM5 (~2024)
• May produce verbose <think> sections; adjust temperature down to 0.5 if you prefer concise outputs

License

Apache-2.0 (inherited from base MiniCPM5-2B). Verify upstream distillation dataset license before commercial deployment.

Acknowledgments

• OpenBMB for MiniCPM5-2B
• r0b0tlab for the multi-teacher distillation dataset
• Unsloth for accelerated training and GGUF export
• llama.cpp team for GGUF format specification