avar6/MiMo-V2.5-Pro-Base-gguf overview
Model This repo contains specialized MoE quants for MiMo V2.5 Pro Base. The idea being that given the huge size of the FFN tensors compared to the rest of the …
Runs locally from ~5.7 MB disk (4 GB VRAM class GPUs with llama.cpp / guIDE).
Repository Files & Downloads
| File | Type | Quantization | Size | Link |
|---|---|---|---|---|
| IQ2_S/MiMo-V2.5-Pro-Base-IQ2_S-00001-of-00008.gguf | GGUF | IQ2_S | 5.7 MB | Download |
| IQ2_S/MiMo-V2.5-Pro-Base-IQ2_S-00002-of-00008.gguf | GGUF | IQ2_S | 46.07 GB | Download |
| IQ2_S/MiMo-V2.5-Pro-Base-IQ2_S-00003-of-00008.gguf | GGUF | IQ2_S | 45.38 GB | Download |
| IQ2_S/MiMo-V2.5-Pro-Base-IQ2_S-00004-of-00008.gguf | GGUF | IQ2_S | 45.31 GB | Download |
| IQ2_S/MiMo-V2.5-Pro-Base-IQ2_S-00005-of-00008.gguf | GGUF | IQ2_S | 45.51 GB | Download |
| IQ2_S/MiMo-V2.5-Pro-Base-IQ2_S-00006-of-00008.gguf | GGUF | IQ2_S | 45.38 GB | Download |
| IQ2_S/MiMo-V2.5-Pro-Base-IQ2_S-00007-of-00008.gguf | GGUF | IQ2_S | 45.31 GB | Download |
| IQ2_S/MiMo-V2.5-Pro-Base-IQ2_S-00008-of-00008.gguf | GGUF | IQ2_S | 26.75 GB | Download |
| IQ3_XS/MiMo-V2.5-Pro-Base-IQ3_XS-00001-of-00008.gguf | GGUF | IQ3_XS | 5.7 MB | Download |
| IQ3_XS/MiMo-V2.5-Pro-Base-IQ3_XS-00002-of-00008.gguf | GGUF | IQ3_XS | 45.94 GB | Download |
| IQ3_XS/MiMo-V2.5-Pro-Base-IQ3_XS-00003-of-00008.gguf | GGUF | IQ3_XS | 45.37 GB | Download |
| IQ3_XS/MiMo-V2.5-Pro-Base-IQ3_XS-00004-of-00008.gguf | GGUF | IQ3_XS | 45.37 GB | Download |
| IQ3_XS/MiMo-V2.5-Pro-Base-IQ3_XS-00005-of-00008.gguf | GGUF | IQ3_XS | 45.37 GB | Download |
| IQ3_XS/MiMo-V2.5-Pro-Base-IQ3_XS-00006-of-00008.gguf | GGUF | IQ3_XS | 45.37 GB | Download |
| IQ3_XS/MiMo-V2.5-Pro-Base-IQ3_XS-00007-of-00008.gguf | GGUF | IQ3_XS | 45.37 GB | Download |
| IQ3_XS/MiMo-V2.5-Pro-Base-IQ3_XS-00008-of-00008.gguf | GGUF | IQ3_XS | 44.05 GB | Download |
| IQ4_XS/MiMo-V2.5-Pro-Base-IQ4_XS-00001-of-00011.gguf | GGUF | IQ4_XS | 5.7 MB | Download |
| IQ4_XS/MiMo-V2.5-Pro-Base-IQ4_XS-00002-of-00011.gguf | GGUF | IQ4_XS | 46.20 GB | Download |
| IQ4_XS/MiMo-V2.5-Pro-Base-IQ4_XS-00003-of-00011.gguf | GGUF | IQ4_XS | 45.72 GB | Download |
| IQ4_XS/MiMo-V2.5-Pro-Base-IQ4_XS-00004-of-00011.gguf | GGUF | IQ4_XS | 45.72 GB | Download |
| IQ4_XS/MiMo-V2.5-Pro-Base-IQ4_XS-00005-of-00011.gguf | GGUF | IQ4_XS | 45.72 GB | Download |
| IQ4_XS/MiMo-V2.5-Pro-Base-IQ4_XS-00006-of-00011.gguf | GGUF | IQ4_XS | 45.72 GB | Download |
| IQ4_XS/MiMo-V2.5-Pro-Base-IQ4_XS-00007-of-00011.gguf | GGUF | IQ4_XS | 45.72 GB | Download |
| IQ4_XS/MiMo-V2.5-Pro-Base-IQ4_XS-00008-of-00011.gguf | GGUF | IQ4_XS | 45.72 GB | Download |
| IQ4_XS/MiMo-V2.5-Pro-Base-IQ4_XS-00009-of-00011.gguf | GGUF | IQ4_XS | 45.72 GB | Download |
| IQ4_XS/MiMo-V2.5-Pro-Base-IQ4_XS-00010-of-00011.gguf | GGUF | IQ4_XS | 45.72 GB | Download |
| IQ4_XS/MiMo-V2.5-Pro-Base-IQ4_XS-00011-of-00011.gguf | GGUF | IQ4_XS | 43.04 GB | Download |
| imatrix.gguf | GGUF | GGUF | 1.42 GB | Download |
Model Details
Model README
---
base_model:
- XiaomiMiMo/MiMo-V2.5-Pro-Base
tags:
- gguf
- xiaomi
- MiMo
---
Model
This repo contains specialized MoE-quants for MiMo-V2.5-Pro-Base. The idea being that given the huge size of the FFN tensors compared to the rest of the tensors in the model, it should be possible to achieve a better quality while keeping the overall size of the entire model smaller compared to a similar naive quantization. To that end, the quantization type default is kept in high quality and the FFN UP + FFN GATE tensors are quanted down along with the FFN DOWN tensors.
| Quant | Size | Mixture | PPL | 1-(Mean PPL(Q)/PPL(base)) | KLD |
| :----- | :--------------------- | :------------------------------- | :------------------ | :------------------------ | :------------------ |
| Q8_0 | 1012.92 GiB (8.50 BPW) | Q8_0 | 2.344588 ± 0.010019 | +0% | 0 |
| Q6_K | 787.32 GiB (6.61 BPW) | Q8_0 / Q6_K / Q6_K / Q6_K | 2.345036 ± 0.010024 | +0.0335% | 0.007237 ± 0.000078 |
| Q5_K_M | 704.84 GiB (5.92 BPW) | Q8_0 / Q5_K / Q5_K / Q6_K | 2.348139 ± 0.010046 | +0.1658% | 0.009220 ± 0.000084 |
| Q4_K_M | 586.58 GiB (4.92 BPW) | Q8_0 / Q4_K / Q4_K / Q5_K | 2.366152 ± 0.010162 | +0.9342% | 0.016214 ± 0.000138 |
| IQ4_XS | 454.99 GiB (3.82 BPW) | Q8_0 / IQ3_S / IQ3_S / IQ4_XS | 2.417036 ± 0.010382 | +3.1048% | 0.042863 ± 0.000354 |
| IQ3_S | 350.82 GiB (2.95 BPW) | Q6_K / IQ2_S / IQ2_S / IQ3_S | 2.638261 ± 0.011570 | +12.5417% | 0.126933 ± 0.000917 |
| IQ3_XS | 316.86 GiB (2.66 BPW) | Q6_K / IQ2_XS / IQ2_XS / IQ3_XXS | 2.819200 ± 0.012818 | +20.2601% | 0.188844 ± 0.001289 |
| IQ2_S | 299.70 GiB (2.52 BPW) | Q6_K / IQ2_XS / IQ2_XS / IQ2_S | 2.922672 ± 0.013398 | +24.6740% | 0.223651 ± 0.001460 |
Run avar6/MiMo-V2.5-Pro-Base-gguf with guIDE
Download guIDE — the AI-native code editor with local LLM inference and 69 built-in tools.
Source: Hugging Face · Compare models