---

license: other

license_name: upstream-license-not-specified

base_model: XiaomiMiMo/MiMo-V2.6-Distill-Qwen-9B

base_model_relation: quantized

library_name: llama.cpp

pipeline_tag: image-text-to-text

tags:

• gguf
• llama.cpp
• mimo_v2
• quantized
• image-text-to-text
• agentic
• code
• tool-use
• multimodal
• long-context

---

MiMo-V2.6-Distill-Qwen-9B GGUF

Community GGUF quantizations of XiaomiMiMo/MiMo-V2.6-Distill-Qwen-9B.

<div align="center" style="background-color:#f59e0b;color:#ffffff;padding:16px 20px;border-radius:10px;line-height:1.7;">

☕ If this GGUF made your day easier, a coffee would make mine.<br>

<a href="https://ko-fi.com/ngquocvinh" style="color:#ffffff;"><strong style="color:#ffffff;">Send a coffee ☕</strong></a><br>

I build and test these releases myself. Your coffee helps keep me going.<br>

Thank you for supporting this work.

</div>

About MiMo-V2.6-Distill-Qwen-9B

MiMo-V2.6-Distill-Qwen-9B is a 9B agentic supervised-fine-tuning checkpoint from Xiaomi MiMo, based on Qwen3.5-9B. The upstream card describes coding, general-purpose agent tasks, visual coding, and cybersecurity. This GGUF package contains quantized language-model files and a matching BF16 multimodal projector for image input.

The converted language model preserves the upstream hybrid Qwen3.5 architecture and 262,144-token native context metadata. Practical context length depends on the selected quantization, KV-cache settings, runtime, and available memory. The image path was validated with llama.cpp; video behavior is not claimed by this release.

Fidelity measurements

These are next-token fidelity measurements against the converted BF16 language-model reference. They use four chunks from wiki.valid.raw at context length 2048, with the same qwen4exp llama.cpp llama-perplexity runtime for every row. Values are averaged across the evaluated chunks. Lower Mean KLD, ΔPPL, and RMS Δp, and higher Top-1 agreement, indicate closer next-token behavior to BF16. These measurements are not task, coding, tool-use, or visual-understanding benchmarks.

| File | Size (GB) | Mean KLD ↓ | Top-1 vs BF16 ↑ | ΔPPL | RMS Δp |

|---|---:|---:|---:|---:|---:|

| MiMo-V2.6-Distill-Qwen-9B-Q8_0.gguf | 9.527498 | 0.006916 | 97.654% | +1.367% | 2.718% |

| MiMo-V2.6-Distill-Qwen-9B-Q6_K.gguf | 7.359256 | 0.022759 | 95.723% | +1.103% | 4.903% |

| Sweet spot — MiMo-V2.6-Distill-Qwen-9B-Q5_K_S.gguf | 6.305306 | 0.107555 | 91.618% | -5.794% | 8.105% |

| MiMo-V2.6-Distill-Qwen-9B-Q5_K_M.gguf | 6.467966 | 0.113086 | 91.178% | -5.654% | 8.250% |

| Sweet spot — MiMo-V2.6-Distill-Qwen-9B-IQ4_XS.gguf | 5.196437 | 0.087589 | 91.300% | -0.402% | 6.968% |

| MiMo-V2.6-Distill-Qwen-9B-IQ4_NL.gguf | 5.418211 | 0.083207 | 91.349% | -0.617% | 6.968% |

| MiMo-V2.6-Distill-Qwen-9B-Q4_K_M.gguf | 5.629106 | 0.107240 | 90.420% | -1.406% | 7.952% |

| MiMo-V2.6-Distill-Qwen-9B-Q4_K_S.gguf | 5.351626 | 0.123219 | 89.980% | -0.033% | 8.623% |

| MiMo-V2.6-Distill-Qwen-9B-Q3_K_M.gguf | 4.623521 | 0.209500 | 84.262% | +17.576% | 11.276% |

| MiMo-V2.6-Distill-Qwen-9B-Q2_K.gguf | 3.827259 | 0.427047 | 78.470% | -0.887% | 16.041% |

| MiMo-V2.6-Distill-Qwen-9B-IQ2_XS.gguf | 3.285342 | 0.493313 | 73.045% | +19.596% | 18.616% |

| MiMo-V2.6-Distill-Qwen-9B-IQ1_M.gguf | 2.877265 | 1.030725 | 59.971% | +82.724% | 29.851% |

| MiMo-V2.6-Distill-Qwen-9B-Q1_0.gguf | 1.965512 | 13.998099 | 0.464% | +96273458.8% | 61.158% |

The bold rows are the current sweet spots: they offer a practical memory/fidelity

balance while retaining high Top-1 agreement with BF16. Sizes use decimal GB

(bytes / 1e9) and are also recorded in reproducibility/quantization-manifest.tsv.

Exact VRAM fit depends

on context length, KV-cache settings, offload, and runtime, so this is not a

fixed VRAM guarantee. The remaining low-bit files are lower-memory or

experimental profiles and are not preferred when fidelity matters.

The negative ΔPPL values for Q5_K_M, Q4_K_M, and Q2_K do not mean that quantization improves the model; they are measurements of this fixed small hold-out and should be interpreted together with KLD, Top-1 agreement, and RMS Δp.

The compact machine-readable table is reproducibility/quality-summary.tsv. Commands, hashes, source split information, and validation details are in reproducibility/manifest.md.

Quick start

Text generation with Q4_K_M:

./llama-cli \
  -m MiMo-V2.6-Distill-Qwen-9B-Q4_K_M.gguf \
  --chat-template-file chat_template.jinja \
  --jinja \
  --reasoning off \
  -p 'Answer briefly in English: What is GGUF and why is it useful for running language models locally?' \
  -n 128 -c 4096 -ngl 99

Image input with the BF16 projector:

./llama-mtmd-cli \
  -m MiMo-V2.6-Distill-Qwen-9B-Q4_K_M.gguf \
  --mmproj mmproj-MiMo-V2.6-Distill-Qwen-9B-BF16.gguf \
  --image image.jpg \
  --jinja \
  -p 'Describe the visible objects and their spatial relationship in this image in one concise English paragraph.' \
  -n 128 -c 4096 -ngl 99

Adjust GPU offload, context, batch size, and KV-cache types for the target machine. The Q2/IQ2/IQ1/Q1 files are experimental and should not be used when capability retention is important.

Reproducibility and validation

• Upstream repository: XiaomiMiMo/MiMo-V2.6-Distill-Qwen-9B
• Locked upstream revision: f2773fb482ac3dd047a4af4003b86e56b7225d0d
• Source format: upstream BF16 safetensors; no quantized model was used as input
• Converted BF16 language-model GGUF and BF16 projector were produced directly from that source
• All nine language-model files were quantized directly from the same BF16 language-model GGUF
• The imatrix is model-specific and combines WikiText-2 train text with EN/ZH/code/tool/agent calibration text
• wiki.valid.raw and wiki.test.raw were not used to create the imatrix
• All nine language files passed deterministic text load/generate smoke tests
• Q4_K_M passed an image smoke test with the matching BF16 projector
• Artifact checksums are in SHA256SUMS.txt

Raw conversion, imatrix, quantization, smoke-test, and fidelity logs remain local under reports/ and are not part of this package.

License and attribution

The upstream MiMo model repository did not provide a license declaration at the time of this build. Public redistribution is therefore intentionally blocked until Xiaomi MiMo's applicable license terms are confirmed. The temporary license: other metadata must not be treated as a grant of rights.

These are community GGUF quantizations, not an official Xiaomi MiMo release or endorsement.