---

license: mit

base_model: XiaomiMiMo/MiMo-V2.6-Flash-RL

tags:

- gguf

- llama.cpp

- moe

- imatrix

- iq2_m

quantized_by: YanissAmz

---

MiMo-V2.6-Flash-RL — IQ2_M-class GGUF (100 GiB), fits 128 GB machines

A single-quant GGUF of XiaomiMiMo/MiMo-V2.6-Flash-RL

(309B total / 15B active, 256 experts, top-8), 2.76 bits per weight on average, sized for machines with ~110 GB of

free RAM/VRAM: 128 GB unified-memory systems (AMD Ryzen AI Max+ 395 / Strix Halo, Apple Silicon 128 GB, DGX Spark),

multi-GPU rigs, or CPU + GPU offload. It is a standard GGUF: any llama.cpp build and backend (CUDA, ROCm/HIP,

Vulkan, Metal, CPU) runs it.

| | |

|---|---|

| Size | 100.4 GiB in 3 shards (99.5 GiB model + 1 GiB MTP layers), 2.76 bpw average |

| Quant type | custom IQ2_M-class mix (see Recipe); the header reports IQ2_M |

| KLD vs native MXFP4 | 0.164 mean · 0.021 median · 2.31 p99 |

| Same top-1 token as MXFP4 | 88.2 % |

| PPL ratio vs MXFP4 | 1.128 |

| Decode, example: AMD Strix Halo (Vulkan) | 26.1 tok/s (tg128) |

| Prefill, example: AMD Strix Halo (Vulkan) | ~310 tok/s at -ub 2048 (pp4096) |

Loads and generates on stock llama.cpp (tested: master of 2026-09-23, CPU build). The speeds above are one example

machine (AMD Ryzen AI Max+ 395, Radeon 8060S, llama.cpp Vulkan build); other hardware will differ.

Files

MiMo-V2.6-Flash-RL-IQ2_M-00001-of-00003.gguf
MiMo-V2.6-Flash-RL-IQ2_M-00002-of-00003.gguf
MiMo-V2.6-Flash-RL-IQ2_M-00003-of-00003.gguf

Point llama.cpp at the first shard.

Usage

llama-server -m MiMo-V2.6-Flash-RL-IQ2_M-00001-of-00003.gguf \
  -ngl 99 -fa on -c 32768 -b 2048 -ub 2048 \
  --cache-type-k q8_0 --cache-type-v q8_0 --jinja

-ub 2048 matters for MoE prefill (Strix Halo example: 193 tok/s at the default -ub 512, 310 tok/s at 2048).

Recipe

The experts of MiMo-V2.6-Flash are stored natively in MXFP4, so the ggml-org MXFP4 GGUF is a lossless reference

for them. Every number above is measured against it.

• Per-tensor bit allocation under a 99.5 GiB budget, minimizing the imatrix-weighted weight error of each

expert tensor multiplied by the residual-stream share of its layer. The per-tensor types are chosen for quality per byte, not for one hardware target. MiMo has massive-activation layers

(layer 7 carries ~96 % of the residual norm, layers 5, 2, 6, 47 and 1 also high). Quantizing those to 2 bits

breaks the model (KLD 3.4, PPL ×27 in a uniform-allocation control), so they keep MXFP4 on most of their experts.

• Experts: IQ2_XXS / IQ2_XS / IQ2_S / IQ3_XXS / IQ3_S / Q3_K by tensor, MXFP4 on the high-share layers.
• Attention QKV, attention output and the output head: Q6_K (Q8_0 → Q6_K on attention output and head costs

+0.4 % KLD, within noise, and gives +6.7 % decode). Embeddings and the dense first layer: Q8_0.

• Importance matrix: AesSedai's imatrix for this model.

Evaluation

• Reference: native MXFP4 GGUF logits (CPU run).
• Corpus: 115 KB of mixed text (Wikipedia, code, calibration-style text), 13 chunks × 2048 tokens, llama-perplexity --kl-divergence.
• Speed (example machine): llama-bench, Vulkan (RADV), Radeon 8060S, idle machine, -fa 1.

MTP layers

The file also contains the model's 3 multi-token-prediction layers (Q8_0, ~1 GiB). Stock llama.cpp ignores them

("unused tensor" warnings are expected). In our measurements, MTP self-speculation does not speed up decode yet on

our test hardware (draft acceptance ~50 %); keep speculation off for now.

License

MIT, same as the base model.