---

base_model: XiaomiMiMo/MiMo-V2.5

base_model_relation: quantized

pipeline_tag: text-generation

library_name: gguf

tags:

- gguf

- rocm

- rocmfp4

- rocmfpx

- strix-halo

- gfx1151

- moe

---

MiMo-V2.5 — ROCmFP4 / ROCmFPX GGUF (Strix Halo / gfx1151)

ROCmFP4 and ROCmFPX quantizations of MiMo-V2.5

(310B total / 15B active MoE), built for **AMD Strix Halo (Ryzen AI Max+ 395,

gfx1151)** with the ROCmFPX fork of

llama.cpp.

There are two ways to run this model on Strix Halo, and they want different files:

| you have | use | why |

| --- | --- | --- |

| two 128 GB boxes + USB4/Thunderbolt | ROCmFP4-FAST/ | best quality; 153 GiB does not fit one node |

| one 128 GB box | Q2_0_ROCMFPX-COHERENT/ | 90.9 GiB fits a single node — and is +37% faster at decode than the two-node setup |

The single-node option being faster is not a mistake. llama.cpp's RPC layer split

runs the nodes sequentially, so a second box buys capacity, not speed. If the

model fits on one node, one node wins.

> Note on quality. The ROCmFP4 and ROCmFP4_FAST variants are not

> imatrix-calibrated; Unsloth's UD-Q4_K_XL is, and is tensor-aware on top of that.

> They win on speed and size — benchmark both if marginal output quality matters

> more to you than tok/s.

>

> Q2_0_ROCMFPX-COHERENT is imatrix-calibrated (it has to be — see below). But

> it is 2.5 bpw against FAST's 4.26, and that reduction is real. It passes coherence

> checks on code, factual recall, arithmetic and JSON, but **no pass@1 evaluation has

> been run*, so treat the quality gap as unquantified rather than absent*.

⚠️ Requirements — stock llama.cpp will not load these

Q4_0_ROCMFP4 and Q4_0_ROCMFP4_FAST are quantization types defined by the

ROCmFPX fork. Upstream llama.cpp,

Ollama, LM Studio and every downstream that vendors mainline ggml **cannot read

these files** — you will get an unknown-ggml-type error, not a slow model. You

must build the fork.

They also target gfx1151 specifically (Ryzen AI Max+ 395 / Strix Halo). The

formats are built around that hardware's dequant path; on other GPUs, expect

either a build failure or no benefit.

Build

git clone https://github.com/charlie12345/ROCmFPX.git
cd ROCmFPX
git checkout 3edc3d31ee5ebcea47fd7e0f42c89767bb4245db   # the commit these were built and tested with

cmake -S . -B build \
    -DCMAKE_BUILD_TYPE=Release \
    -DGGML_HIP=ON \
    -DGGML_RPC=ON \
    -DGGML_HIP_FORCE_MMQ=ON \
    -DGGML_HIP_ROCWMMA_FATTN=OFF \
    -DGGML_VULKAN=OFF -DGGML_CUDA=OFF \
    -DCMAKE_HIP_ARCHITECTURES=gfx1151 \
    -DGPU_TARGETS=gfx1151 \
    -DLLAMA_BUILD_SERVER=ON \
    -DLLAMA_BUILD_WEBUI=OFF -DLLAMA_USE_PREBUILT_WEBUI=OFF \
    -DLLAMA_BUILD_TESTS=OFF -DGGML_BUILD_TESTS=OFF

cmake --build build -j "$(nproc)" --target \
    llama-cli llama-server llama-bench llama-quantize rpc-server

-DGGML_RPC=ON is what gives you rpc-server and the RPC0 device — required

for the two-node split. Built against ROCm 6.4.

Use the ROCm backend, not Vulkan. RADV imposes a per-buffer allocation ceiling

that a model this size runs straight into; -dev Vulkan0 will OOM where ROCm0

works fine.

Making the memory available

A single Strix Halo box defaults to a GPU carve-out far below what these need.

Either set GGML_CUDA_ENABLE_UNIFIED_MEMORY=1 in the environment, or raise the

GTT limit on the kernel command line (what we run):

amdgpu.gttsize=126976 ttm.pages_limit=32505856 ttm.page_pool_size=32505856

With the cmdline set, UNIFIED_MEMORY is no longer needed. We measured no

throughput difference between the two approaches — the cmdline is just less

fragile.

⚡ Force the GPU power level — worth 14% decode

This is the single highest-value host setting, and it is easy to miss.

echo high | sudo tee /sys/class/drm/card*/device/power_dpm_force_performance_level

| power_dpm_force_performance_level | pp512 | tg128 |

| --- | ---: | ---: |

| auto (default) | 322.87 | 20.94 |

| high | 329.43 | 23.87 |

+14% decode, +2% prefill, from one sysfs write. The DPM governor never ramps

to peak on MoE decode — the workload is bursty and low-occupancy, so auto reads

it as near-idle and leaves the GPU and fabric below spec. Effective memory

bandwidth goes from 167 to 190.7 GB/s (65% -> 75% of the 256 GB/s theoretical).

DRAM is at its rated 8000 MT/s either way; this is purely a clock-governor effect.

The gain is specific to single-stream decode. Under concurrency the GPU is

already loaded enough that auto ramps by itself — aggregate throughput at C8 is

unchanged.

Transparent hugepages were tested alongside this and make no difference

(21.24 with THP alone vs 20.94 baseline), despite the 153 GiB working set. Not

worth the system-wide side effects.

It resets on reboot. To persist:

# /etc/systemd/system/amdgpu-perf-high.service
[Unit]
Description=Force amdgpu DPM to high
After=multi-user.target

[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=/bin/sh -c 'for c in /sys/class/drm/card*/device/power_dpm_force_performance_level; do echo high > "$c"; done'
ExecStop=/bin/sh -c 'for c in /sys/class/drm/card*/device/power_dpm_force_performance_level; do echo auto > "$c"; done'

[Install]
WantedBy=multi-user.target

Apply it on every node — the RPC peer's clocks matter just as much as the

head node's.

Variants

| variant | effective bpw | size | shards | fits 1 node? | folder |

| --- | ---: | ---: | ---: | :---: | --- |

| Q4_0_ROCMFP4_FAST ← best quality | 4.26 | 153.47 GiB (164,795,562,720 B) | 4 | no | ROCmFP4-FAST/ |

| Q2_0_ROCMFPX ← fastest, single-node | 2.52 | 90.92 GiB (97,626,242,048 B) | 3 | yes | Q2_0_ROCMFPX-COHERENT/ |

| Q4_0_ROCMFP4 | 5.24 | 189.02 GiB (202,969,561,824 B) | 5 | no | ROCmFP4/ |

For reference, unsloth/MiMo-V2.5-GGUF UD-Q4_K_XL is 178.44 GiB at 4.95 bpw —

ROCmFP4_FAST is 14% smaller.

> ROCmFP4_FAST dominates ROCmFP4 on every axis we measured — smaller,

> faster prefill, faster decode. Unless you specifically want the higher-precision

> tensors, take FAST. ROCmFP4 is published for completeness.

A note on the bpw labels. llama-quantize advertises Q4_0_ROCMFP4 as

"4.50 bpw", but measured against the 309.77 B parameter count the real figure is

5.24 bpw — the recipe promotes several tensor classes (e.g. ffn_gate → q5_K)

rather than quantizing everything to ROCmFP4. Q4_0_ROCMFP4_FAST measures 4.26 bpw

against its 4.25 label, so that one is honest. This is why the "non-fast" build

ends up larger than UD-Q4_K_XL rather than smaller.

All were quantized from the BF16 GGUF (unsloth/MiMo-V2.5-GGUF, 14 shards,

619,638,702,336 bytes, verified byte-exact).

The 4-bit variants need no special handling:

llama-quantize MiMo-V2.5-BF16-00001-of-00014.gguf \
    MiMo-V2.5-ROCmFP4-FAST.gguf Q4_0_ROCMFP4_FAST 16

Q2_0_ROCMFPX needs two extra flags or it produces garbage

This is the important part of this section. A plain Q2_0_ROCMFPX run — no imatrix,

default tensor types — yields a model that loads, benchmarks at a healthy 34.97 t/s,

and emits this:

，问题，1.问题，1.问题，10000000000000000000000000000000000000...

Fluent-looking throughput, zero usable output. Both of the following are required:

llama-quantize \
    --imatrix MiMo-V2.5-imatrix.gguf \
    --token-embedding-type q6_k \
    --output-tensor-type q6_k \
    MiMo-V2.5-BF16-00001-of-00014.gguf \
    MiMo-V2.5-Q2_0_ROCMFPX-COHERENT.gguf Q2_0_ROCMFPX 16

• --imatrix — sub-3-bit quantization needs importance-weighted rounding. The

proof is that mainline IQ2_XXS stays coherent at 2.06 bpw, lower than our

2.5, because it is imatrix-aware. Uniform rounding survives 4 bpw; it does not

survive 2.5. The imatrix used here is

bartowski's (MiMo-V2.5-imatrix.gguf,

287 entries over 818 chunks).

• --token-embedding-type q6_k --output-tensor-type q6_k — the embedding table

(152,576 tokens) and LM head are the most precision-sensitive tensors in the model,

and the imatrix does not cover them (did not find weights for token_embd.weight).

Crush them to 2.5 bpw and nearby token vectors collapse into one another, which is

exactly the wrong-language-token + repetition signature above. Cost: **+0.3 GB on a

97 GB file.**

ROCmFPX's own naming corroborates this: Q4_0_ROCMFP4_COHERENT is defined as

"ROCmFP4 + Q6_K token embeddings". Nobody names a variant coherent unless the

alternative was incoherent. Q2_0_ROCMFPX ships with no such provision, hence the

-COHERENT suffix on this folder.

Measured performance

Test setup

Two Beelink GTR 9 Pro (Ryzen AI Max+ 395, gfx1151, 128 GB unified) linked by a

single USB4 cable. The 4-bit variants are layer-split across both with llama.cpp RPC;

Q2_0_ROCMFPX runs on one box with no RPC at all (-ngl 999, no -dev), which is

the whole reason it decodes faster. All numbers are single-stream unless the

concurrency section says otherwise.

| | |

| --- | --- |

| engine | ROCmFPX (llama.cpp fork), ROCm 6.4, HIP backend |

| OS / kernel | Ubuntu 24.04, mainline 6.18.6 |

| topology | node2 = head, node1 = rpc-server over Thunderbolt (192.168.2.1:50052) |

| GPU carve-out | 126976 MiB via amdgpu.gttsize / ttm.pages_limit kernel cmdline |

| transport | TCP over thunderbolt0 (RDMA measured, no difference) |

Benchmark command:

llama-bench -m <model.gguf> \
    -rpc 192.168.2.1:50052 -dev ROCm0/RPC0 \
    -ngl 999 -fa 1 -mmp 0 -r 1 -p 512 -n 128

-mmp 0 (no mmap) is required — with mmap the working set thrashes against

the 128 GB of RAM and never converges. Note llama-bench wants -dev entries

separated by /, while llama-cli wants ,.

Single-stream results

All rows below are measured with power_dpm_force_performance_level=high on both

nodes (see Requirements) — without it every number drops 9–14%.

| variant | nodes | size | prefill (pp512) | decode (tg128) |

| --- | :---: | ---: | ---: | ---: |

| Q2_0_ROCMFPX | 1 | 90.92 GiB | 241.05 t/s | 32.53 t/s |

| Q4_0_ROCMFP4_FAST | 2 | 153.47 GiB | 328.73 t/s | 23.80 t/s |

| _(reference)_ UD-Q4_K_XL | 2 | 178.44 GiB | 339.31 t/s | 17.16 t/s |

| Q4_0_ROCMFP4 _(at dpm=auto)_ | 2 | 189.02 GiB | 248.10 t/s | 16.80 t/s |

**Q2_0_ROCMFPX on one node decodes 36.7% faster than ROCmFP4_FAST on two, for

26.7% less prefill.** Two effects compound: fewer bytes to stream per token (2.52 vs

4.26 bpw), and no RPC layer split — which runs the nodes sequentially, so it costs

decode latency rather than saving it. Prefill is the opposite case: it is

compute-bound and genuinely benefits from a second GPU, which is why the two-node

rows win there.

Q2 error bars over -r 2: pp512 ±0.75, tg128 ±0.08.

**ROCmFP4_FAST is +38.7% decode over UD-Q4_K_XL for −3.1% prefill, while being

14% smaller.** The decode gain far exceeds what the size reduction alone predicts

(4.26 vs 4.95 bpw) — the single-scale layout also dequantizes more cheaply on

gfx1151.

Both were re-measured at dpm=high so the comparison is like-for-like. Worth

noting UD-Q4_K_XL gains only ~9% from that tuning where ROCmFP4_FAST gains 14%,

consistent with the FAST dequant path being more clock-sensitive. At the old

dpm=auto default the gap read as +33%.

ROCmFP4 (non-fast) has not been re-measured at dpm=high; its row is from

dpm=auto and is not comparable to the two above. It was the weakest of the three

on throughput at equal settings and is published only for completeness.

Reproducibility: ROCmFP4_FAST at dpm=high was measured three times —

329.43/23.87, 329.14/23.89, 328.73/23.80 — a 0.2% spread on pp512 and 0.4% on

tg128.

Decode vs. context depth

Q4_0_ROCMFP4_FAST, with power_dpm_force_performance_level=high:

| depth | pp512 | tg128 |

| ---: | ---: | ---: |

| 0 | 328.73 | 23.80 |

| 8192 | 286.70 | 23.00 |

| 32768 | 214.53 | 21.72 |

Decode is remarkably flat with context — a consequence of the 9-full/39-SWA

attention split, where only 9 layers grow with depth. Prefill decays normally.

For reference, the same curve at the default dpm=auto was 315.5/21.07,

267.1/20.68 and 197.7/19.53 — the tuning is worth ~11–13% of decode at every

depth.

Q2_0_ROCMFPX on a single node, same benchmark, dpm=high, -r 2:

| depth | pp512 | tg128 | vs FAST 2-node decode |

| ---: | ---: | ---: | ---: |

| 0 | 241.05 ± 0.75 | 32.53 ± 0.08 | +36.7% |

| 8192 | 220.14 ± 1.73 | 29.19 ± 0.67 | +26.9% |

| 32768 | 177.29 ± 1.66 | 28.64 ± 0.14 | +31.9% |

Q2 on one node beats FAST on two at decode at every depth, and loses at prefill at

every depth. Its decode curve is also flatter past 8k (29.19 → 28.64, −1.9% over a 4×

context increase) — same 9-full/39-SWA attention structure as the 4-bit variants, with

less weight traffic per token on top.

Pick by workload: long prompts / short answers → two-node FAST (prefill wins).

Short prompts / long answers, or interactive chat → single-node Q2 (decode wins,

and on one machine).

Aggregate throughput under concurrency

Total tokens/s across all streams, 16 distinct prompts so that slots cannot share

a prefix-cache hit and inflate the result (--parallel 8, -c 32768).

| variant | C4 | C6 | C8 |

| --- | ---: | ---: | ---: |

| Q4_0_ROCMFP4_FAST (dpm=high) | 33.20 t/s | 34.51 t/s | 33.57 t/s |

| Q4_0_ROCMFP4_FAST (dpm=auto) | 31.93 t/s | 32.61 t/s | 33.92 t/s |

| Q4_0_ROCMFP4 (dpm=auto) | 24.96 t/s | 26.37 t/s | 28.32 t/s |

| _(reference)_ UD-Q4_K_XL (dpm=auto) | 25.81 t/s | 29.19 t/s | 31.72 t/s |

Aggregate throughput plateaus around 33–35 t/s and the dpm=high tuning barely

helps here — under concurrency the GPU is already busy enough that the governor

ramps on its own. That is the mirror image of the single-stream case, where forcing

high is worth 14%.

ROCmFP4_FAST is fastest at every concurrency level, but **its margin over

UD-Q4_K_XL shrinks as concurrency rises** — the FAST layout's advantage is in

memory bandwidth and dequant cost, which dominate single-stream decode; as batch

size grows the workload shifts toward expert scatter and compute, where the quants

converge.

MiMo-V2.5 is a 256-expert top-8 MoE, the least favourable case for batch

amortization: the number of distinct experts touched at batch B grows as

256·(1−(1−8/256)^B), so more expert weights must be read as concurrency rises

instead of being amortized across the batch. This is why aggregate throughput

scales so weakly — ROCmFP4_FAST gains only 6% going from 4 streams to 8, and

per-stream latency roughly halves over that range (7.98 → 4.24 t/s).

Usage

Single node — Q2_0_ROCMFPX (fastest decode)

GGML_CUDA_ENABLE_UNIFIED_MEMORY=1 \
llama-server -m Q2_0_ROCMFPX-COHERENT/MiMo-V2.5-Q2_0_ROCMFPX-COHERENT-00001-of-00003.gguf \
    -ngl 999 -fa 1 --no-mmap -ctk f16 -ctv f16 \
    -c 32768 --parallel 8 --jinja --host 0.0.0.0 \
    --temp 1.0 --top-p 0.95 --min-p 0.0 --repeat-penalty 1.05

No --rpc, no -dev. GGML_CUDA_ENABLE_UNIFIED_MEMORY=1 is required — 90.92 GiB

exceeds the hard VRAM cap even though it fits the 128 GB unified pool.

Two nodes — Q4_0_ROCMFP4_FAST (best quality)

# node1
rpc-server -H 0.0.0.0 -p 50052

# node2
llama-server -m ROCmFP4-FAST/MiMo-V2.5-ROCmFP4-FAST-00001-of-00004.gguf \
    --rpc 192.168.2.1:50052 -dev ROCm0,RPC0 \
    -ngl 999 -fa 1 --no-mmap -ctk f16 -ctv f16 \
    -c 32768 --parallel 8 --jinja --host 0.0.0.0 \
    --temp 1.0 --top-p 0.95 --min-p 0.0 --repeat-penalty 1.05

Point llama.cpp at the first shard; it pulls in the rest automatically.

Sampling — use the model's own recommendation

Xiaomi's model card specifies temperature=1.0, top_p=0.95 for local

deployment. --repeat-penalty 1.05 and --min-p 0.0 come from the community

configuration in llama.cpp #23074.

Use --jinja with the model's built-in chat template. No corrected template exists

or is needed — neither Unsloth nor bartowski overrides it. You may see

Expected iterable or object type in for loop: got None in the log during

reasoning-format detection; it is harmless and also appears on runs that work.

Note a MiMo-level quirk, not a quantization artifact: MiMo-V2.5 can enter runaway

chain-of-thought — "extremely long CoT (sometimes for hundreds of thousands of tokens)

with no progress or tool calls" (#23074), reproduced on Q8_0, so it is not caused

by low-bit quantization. Cap it with --reasoning-budget, and pass

"chat_template_kwargs": {"enable_thinking": false} per-request to disable thinking

entirely. Thinking costs ~0.5% of decode rate — its real cost is the extra tokens.

Mind the flag spelling — llama-server takes --rpc and comma-separated -dev,

while llama-bench takes -rpc and slash-separated -dev (ROCm0/RPC0).

Passing the wrong one makes the server exit immediately with

error: invalid argument.

Things worth knowing

• Keep KV cache at f16 for speed. Measured on this exact model and split:

| -ctk / -ctv | tg128 @d0 | tg128 @32k |

| --- | ---: | ---: |

| f16 / f16 | 20.71 | 19.44 |

| q8_0 / q8_0 | 20.26 | 17.13 |

| q8_0 / q4_0 | 20.17 | 17.24 |

| q4_0 / q4_0 | 20.04 | 17.22 |

The penalty is ~2% at zero depth but 13% at 32k — ROCm dequant costs more

than the bandwidth it saves, and the gap widens as the cache fills. Note the

three quantized configs are indistinguishable: the cost comes from quantizing

at all, not from how aggressively.

But quantizing buys context. f16 KV is 22.5 KiB/token here; q4_0/q4_0 is

~6.6 KiB — about 3.4× the KV pool for that 13%. If maximum context matters

more to you than decode speed, -ctk q4_0 -ctv q4_0 is the trade.

• Prefer the USB4/Thunderbolt link over ethernet for the RPC hop. Against a

switched 1 GbE path we measured −5% decode and −9.5% prefill. The causes differ:

decode is latency-bound (llama.cpp RPC does ~3–4 round trips per token, so hop

latency multiplies), while prefill is bandwidth-bound (4 MB of activations per

512-token chunk). A switched 10 GbE link fixes the prefill half but not the

decode half; a direct point-to-point cable fixes both.

• Speculative decoding does not pay off on this model. Measured on this exact

build and split, with --temp 0 --repeat-penalty 1.0:

| workload | no speculation | DFlash n=1 | DFlash n=2 | DFlash n=4 |

| --- | ---: | ---: | ---: | ---: |

| structured JSON | 21.00 | 14.87 | 13.80 | 10.02 |

| Rust code | 21.18 | 13.00 | 11.96 | 8.94 |

| narrative prose | 21.21 | 12.19 | 10.72 | 8.16 |

DFlash costs 30–60%, and gets worse the deeper you draft. **The cause is

acceptance.** Mean accept length is 1.52 / 1.32 / 1.23 (JSON / code / prose),

and per-position acceptance on JSON runs 0.500, 0.150, 0.011, 0.000 — the

drafter is right about half the time on token 1 and essentially never by token 3.

A DFlash step costs 2.15× a normal decode step here, so break-even needs

accept length ≥ 2.15. Reference implementations on other engines report 3.78 on

this same model and drafter, which would be ~1.76× — so the headroom is real,

it is just not reachable from this engine. An independent DFlash draft GGUF for

MiMo-V2.5-Pro on ik_llama.cpp reports the same shape (54.6–60.4% acceptance,

55.6–59.4 t/s drafted vs 59.9–60.8 undrafted — also a net loss).

Measured with thinking off; enabling it changes JSON and prose by ~0 and

costs code about 16%.

MTP is unavailable (llama.cpp issue #23924 closed not_planned, though these

GGUFs do carry the blk.48-50.nextn.* tensors), and no EAGLE3 drafter has been

published for MiMo-V2.5.

• If you do experiment with DFlash over an RPC split, the target's LM head and

token embeddings must be pinned to the local device or it aborts at load in

ggml_backend_sched_backend_id_from_cur:

-ot "output\.weight=ROCm0" -ot "token_embd\.weight=ROCm0" -devd ROCm0

• KV geometry: MiMo-V2.5 has 9 full-attention + 39 sliding-window layers, so

only the 9 full layers scale with context — about 22.5 KiB/token.

Speculative decoding (MTP) — present, but do not expect a win

MiMo-V2.5 ships its own MTP / NextN head (model_mtp.safetensors upstream →

blk.48/49/50.nextn.* in GGUF, 3 NextN layers on top of 48 trunk layers). All files

here preserve those tensors, so they are available if your build can use them.

Using them requires engine support that is not yet in any release.

llama.cpp PR #26228 adds mimo2

MTP draft support and is still open; the ROCmFPX fork does not carry it either. Without

it the nextn tensors are simply ignored (harmless — the files still load and run

normally on a stock build; verified).

Two warnings, both measured on this hardware, because the intuitive expectations are

wrong in both directions:

*1. Across a two-node RPC split, MTP is a 33% loss — even at 100% acceptance.*

| 2-node ROCmFP4_FAST, -dev RPC0,ROCm0 | decode | acceptance |

| --- | ---: | --- |

| no MTP (control) | 23.55 t/s | — |

| MTP, n-max 1 | 15.87 t/s | 1.000 (27/27) |

Acceptance was perfect and it still lost a third of throughput. The deficit is

per-draft-step cross-device synchronisation, not draft quality — which means **no

better drafter can fix it.** EAGLE3 would be worse still: it needs three hidden-state

taps from layers spread across the trunk, so on a layer split some taps are always

remote, whereas MTP needs only the final pre-norm hidden state.

If you do try it, note the load-time crash and its fix: with -dev ROCm0,RPC0 the

trailing nextn blocks land on the remote node, and reading the hidden state back

aborts inside ggml_backend_rpc_buffer_get_tensor during startup. **Flip the device

order to -dev RPC0,ROCm0** — layers are assigned to -dev entries in order, so the

last device named gets the trailing blocks. -ot moves weights but not the KV

cache; pinning weights alone does not fix it.

2. On a single node, whether MTP helps depends on the quant.

| single node | base | best MTP | verdict |

| --- | ---: | ---: | --- |

| IQ2_XXS (third-party, 2.06 bpw) | 20.75 t/s | 25.81 t/s (+24%) | worth it |

| Q2_0_ROCMFPX (this repo) | 35.57 t/s | 31.50 t/s (−11%) | not worth it |

Two compounding reasons MTP loses on Q2_0_ROCMFPX: the base is already fast, so

there is less fixed overhead to hide, and **the nextn head is itself quantized to

2.5 bpw** — mean acceptance length falls to 1.39–1.75 here versus 1.95 on IQ2_XXS.

Lifting token_embd/output to Q6_K does not help the drafter; the nextn tensors

stay at 2.5 bpw.

Two counter-intuitive tuning notes if you experiment: **higher --spec-draft-p-min

is better** (0.75 → 31.50 vs 0.10 → 30.26; fewer, more confident drafts waste less

verify work), and --spec-draft-n-max 1 is optimal — depth ≥2 loses badly

(22.19 t/s at n=2). Deeper drafts widen the verify batch, and on a 256-expert top-8

MoE a wider batch activates more experts, so weight traffic grows faster than

accepted tokens. Mean acceptance length keeps rising with depth while throughput

falls, so acceptance alone is a misleading metric here.

License

Inherits the license of the base model,

XiaomiMiMo/MiMo-V2.5. Quantization

adds no additional restrictions.

Credits

• XiaomiMiMo — MiMo-V2.5
• unsloth — BF16 GGUF conversion used as the source
• ROCmFPX — the ROCmFP4 quant formats