---

license: apache-2.0

language:

• en

base_model:

• openbmb/MiniCPM-V-4.6

pipeline_tag: image-text-to-text

library_name: transformers

tags:

• llama-cpp
• text-generation-inference
• minicpm-v
• multimodal
• On-Device Model
• lightweight

---

MiniCPM-V-4.6-GGUF

> MiniCPM-V-4.6 is OpenBMB's most edge-deployment-friendly multimodal model to date, a pocket-sized MLLM built on SigLIP2-400M and the Qwen3.5-0.8B LLM, designed for ultra-efficient image and video understanding directly on phones. It inherits strong single-image, multi-image, and video understanding from the MiniCPM-V family while introducing mixed 4x/16x visual token compression and a LLaVA-UHD v4-based architecture that cuts visual encoding FLOPs by over 50%, delivering ~1.5x token throughput versus Qwen3.5-0.8B. Despite its compact size, it scores 13 on the Artificial Analysis Intelligence Index — beating Qwen3.5-0.8B (10) with 19x fewer tokens, Qwen3.5-0.8B-Thinking (11) with 43x fewer tokens, and even the larger Ministral 3 3B (11) — while reaching Qwen3.5-2B-level performance on benchmarks like OpenCompass, RefCOCO, HallusionBench, MUIRBench, and OCRBench. The model is deployable across iOS, Android, and HarmonyOS with fully open-sourced edge adaptation code, supports flexible slicing/frame-sampling parameters for balancing detail versus speed, integrates with inference frameworks including vLLM, SGLang, llama.cpp, and Ollama (with GGUF/BNB/AWQ/GPTQ quantized variants), supports fine-tuning via SWIFT and LLaMA-Factory, and is released under the Apache-2.0 license.

Model Files

File Name | Quant Type | File Size | File Link |

|-----------|------------|-----------|-----------|

| MiniCPM-V-4.6.BF16.gguf | BF16 | 1.52 GB | Download |

| MiniCPM-V-4.6.F16.gguf | F16 | 1.52 GB | Download |

| MiniCPM-V-4.6.F32.gguf | F32 | 3.02 GB | Download |

| MiniCPM-V-4.6.Q3_K_L.gguf | Q3_K_L | 491 MB | Download |

| MiniCPM-V-4.6.Q3_K_M.gguf | Q3_K_M | 466 MB | Download |

| MiniCPM-V-4.6.Q3_K_S.gguf | Q3_K_S | 435 MB | Download |

| MiniCPM-V-4.6.Q4_K_M.gguf | Q4_K_M | 529 MB | Download |

| MiniCPM-V-4.6.Q4_K_S.gguf | Q4_K_S | 505 MB | Download |

| MiniCPM-V-4.6.Q5_K_M.gguf | Q5_K_M | 578 MB | Download |

| MiniCPM-V-4.6.Q5_K_S.gguf | Q5_K_S | 563 MB | Download |

| MiniCPM-V-4.6.Q8_0.gguf | Q8_0 | 812 MB | Download |

| MiniCPM-V-4.6.mmproj-bf16.gguf | mmproj-bf16 | 1.11 GB | Download |

| MiniCPM-V-4.6.mmproj-f16.gguf | mmproj-f16 | 1.11 GB | Download |

| MiniCPM-V-4.6.mmproj-q8_0.gguf | mmproj-q8_0 | 728 MB | Download |

llama.cpp

LLM inference in C/C++ — https://github.com/ggml-org/llama.cpp