---

license: mit

base_model:

- XiaomiMiMo/MiMo-V2.6-Flash-RL

- patrickbdevaney/MiMo-V2.6-Flash-REAP50

tags:

- gguf

- llama.cpp

- reap

- hope

- moe

- pruned

- multimodal

- vision

- audio

- mtp

---

MiMo-V2.6-Flash REAP-50 — GGUF (community)

Community GGUF quantisations of MiMo-V2.6-Flash-REAP50, a 50% routed-expert pruned checkpoint of XiaomiMiMo/MiMo-V2.6-Flash-RL. Experts were selected with the HOPE objective over REAP-style saliency. Not affiliated with or endorsed by Xiaomi.

> Evaluation status. No end-to-end quality benchmark (perplexity, generative or task accuracy) has been run on

> these files or on the pruned checkpoint. The only published measure is saliency retention on the calibration

> corpus (worst domain 0.9935, mean 0.9964; see the base card),

> which is a proxy, not a quality result.

• Base HF Checkpoint: patrickbdevaney/MiMo-V2.6-Flash-REAP50
• Experts Retained: 128 of 256 routed experts per layer across 47 MoE layers (1 dense layer, 47 MoE layers).
• Base Architecture: Native packed MXFP4 (U8, block size 32) experts with unquantized pure BF16 attention and embeddings.
• Towers Included: Vision & Audio multimodal projectors (mmproj) and Multi-Token Prediction speculative draft heads (mtp).

---

Quantization Ladder

| Filename | Quant Type | Size | Description | Recommended VRAM / RAM |

| :--- | :--- | :--- | :--- | :--- |

| MiMo-V2.6-Flash-REAP50-MXFP4_MOE.gguf | MXFP4_MOE | 86.06 GiB | Native packed MXFP4 experts repacked without dequantising (expert weights not re-quantised) + BF16 attention/trunk. End-to-end equivalence to the safetensors checkpoint not separately tested. | 96 GiB+ / 1x 128GB Thor or 2x 48GB |

| MiMo-V2.6-Flash-REAP50-Q2_K.gguf | Q2_K | 61.64 GiB | Hybrid MoE: down-projections kept in native MXFP4, gate/up in Q2_K, trunk in Q8_0 (~3.36 BPW). | 64 GiB+ / 3x 24GB GPUs (72GB) or Mac 64-96GB |

Supporting Towers (Vision, Audio & MTP)

| Filename | Size | Description |

| :--- | :--- | :--- |

| mmproj-MiMo-V2.6-Flash-REAP50-BF16.gguf | 2.56 GiB | Multimodal projector (Vision + Audio) in BF16 |

| mmproj-MiMo-V2.6-Flash-REAP50-Q8_0.gguf | 1.46 GiB | Multimodal projector (Vision + Audio) quantized to Q8_0 |

| mtp-MiMo-V2.6-Flash-REAP50-BF16.gguf | 4.17 GiB | Multi-Token Prediction (MTP) draft head (3 next-n layers) in BF16 |

| mtp-MiMo-V2.6-Flash-REAP50-Q8_0.gguf | 2.22 GiB | Multi-Token Prediction (MTP) draft head (3 next-n layers) in Q8_0 |

---

Key Features

1. Native MXFP4 MoE Preservation:

In the source checkpoint the routed experts ship as native packed mxfp4 (block size 32), 92.9% of the checkpoint by bytes. The GGUF converter repacks these blocks directly into GGMLQuantizationType.MXFP4 instead of dequantising and re-quantising them, so expert weights keep their original FP4 values.

1. Multimodal Projectors (mmproj):

Xiaomi MiMo-V2.6-Flash incorporates both visual and audio processing towers:

- Vision encoder (28-layer ViT, 560px patch representation)

- Audio tokenizer / RVQ speech representations

Both are packed into standard GGUF multimodal projectors (mmproj--BF16.gguf and mmproj--Q8_0.gguf) intended for llama.cpp's multimodal pipeline (--mmproj).

1. Multi-Token Prediction (mtp):

MiMo-V2.6-Flash includes 3 trained MTP layers for speculative decoding. We ship standalone MTP draft models (mtp--BF16.gguf and mtp--Q8_0.gguf) that can be loaded alongside the trunk model with --draft-model. Note: these heads were trained against the unpruned expert set, and no speedup or acceptance rate has been measured on this pruned checkpoint.

---

Running with llama.cpp

1. Standard Text Inference (Optimal Hybrid Q2_K)

./llama-cli \
    -m MiMo-V2.6-Flash-REAP50-Q2_K.gguf \
    -p "You are MiMo, an AI assistant developed by Xiaomi. Explain how MoE expert pruning works:" \
    -n 512 --temp 0.6

Or run the native MXFP4 file:

./llama-cli \
    -m MiMo-V2.6-Flash-REAP50-MXFP4_MOE.gguf \
    -p "You are MiMo, an AI assistant developed by Xiaomi. Explain how MoE expert pruning works:" \
    -n 512 --temp 0.6

2. Speculative Decoding with MTP Draft Head

./llama-cli \
    -m MiMo-V2.6-Flash-REAP50-Q2_K.gguf \
    --draft-model mtp-MiMo-V2.6-Flash-REAP50-Q8_0.gguf \
    -p "Explain quantum teleportation in detail:" \
    -n 512

3. Multimodal Inference (Vision & Audio)

./llama-cli \
    -m MiMo-V2.6-Flash-REAP50-Q2_K.gguf \
    --mmproj mmproj-MiMo-V2.6-Flash-REAP50-Q8_0.gguf \
    --image input.jpg \
    -p "Describe the contents of this image in detail."

4. OpenAI-Compatible API Server

./llama-server \
    -m MiMo-V2.6-Flash-REAP50-Q2_K.gguf \
    --mmproj mmproj-MiMo-V2.6-Flash-REAP50-Q8_0.gguf \
    --port 8080 \
    -ngl 99

---

Background & Pruning Method

Experts were chosen by saliency accumulated over a calibration corpus (English/Chinese text, code, math, science,

finance, agentic traces, and real image/audio/video captions), and the prune set was solved with HOPE

(Higher-Order Pruning of Experts, arXiv 2609.18916). HOPE minimises pᵀFp, the

output error a prune set causes, including the interaction terms between experts; REAP is the same objective with

those off-diagonal terms set to zero. Selection was ranked by the worst calibration domain, with the top 8% of each

domain protected, so audio and video, the domains most at risk, kept the highest-priority experts (worst-domain

retention 0.9935, audio). Routers were refitted and then kept at the teacher

weights because the refit did not beat the baseline.

What this does not show: that the pruned model performs as well as the original. That needs end-to-end

benchmarks, which have not been run. Full settings and limitations are on the

base card and in the

pipeline repo.

---

Created by patrickbdevaney. MIT, inherited from the base model; attribution to Xiaomi MiMo.