---

license: other

license_name: minimax-community

license_link: https://huggingface.co/MiniMaxAI/MiniMax-M3/blob/main/LICENSE

base_model: MiniMaxAI/MiniMax-M3

tags:

- gguf

- llama.cpp

- minimax-m3

- msa

- blackwell

- sm_120

---

MiniMax-M3 GGUF — sm_120-safe MSA quants

Built with MiniMax M3.

GGUF quantizations of MiniMaxAI/MiniMax-M3

(428B-A23B, MiniMax Sparse Attention) that **contain no iq1_s, iq2_s or

iq3_s tensors**, so they actually run on NVIDIA Blackwell / compute capability

12.0 (RTX PRO 6000, RTX 5090, RTX 5070 Ti …).

Why these exist

iq1_s, iq2_s and iq3_s have broken CUDA matmul kernels on sm_120.

Verified with test-backend-ops test -o MUL_MAT -b CUDA0 on llama.cpp master

b10148 (2026-07-27): 11/11 FAIL for each of those three types, while iq2_xxs,

iq2_xs, iq3_xxs, iq4_xs, iq4_nl, q2_K, q3_K, q4_K, q5_K and

q6_K all pass.

Quant names hide this. llama-quantize's own IQ4_XS preset assigns iq3_s

to the MoE expert gate/up tensors, so a file labelled IQ4_XS can be mostly a

broken type. At the time of writing, every published sub-Q4 M3 GGUF we surveyed

was affected — one popular 189 GiB IQ4_XS build carries 275 of its 426 G

params in iq3_s.

These builds pin every tensor type explicitly and are verified after

quantization, not assumed.

Variants

Measured on a single host: RTX PRO 6000 96 GB + RTX 5070 Ti 16 GB, 125 GiB

DDR5, Ryzen 9 9950X3D. 32K context, f16 KV, -ngl 99 -ts 95,5 --threads 12,

experts split across both GPUs and CPU.

| variant | expert gate/up | expert down | size | bpw | decode tok/s | prefill tok/s | needle @95% depth |

|---|---|---|---|---|---|---|---|

| Q3_K | q3_K | q4_K | 196 GiB | 3.95 | 11.5–12.6 | ~96 | 5/5 |

| IQ3_XXS| iq3_xxs | iq4_xs | 180 GiB | 3.63 | 13.5–14.4 | ~116 | 5/5 |

| Q2_K | q2_K | q4_K | 170 GiB | 3.39 | 15.1–16.3 | ~130 | 4/5 |

| IQ2_XS | iq2_xs | q4_K | 160 GiB | 3.22 | 16.4–17.6 | ~150 | 7/11 |

Everything else is q8_0, and all 228 indexer.* tensors are kept at F32 —

MSA's block selection depends on them and they cost under 1 GB.

"needle @95% depth" = exact passphrase retrieval with the needle at 95% depth in

a ~29,700-token context (near the 32K ceiling) — the hardest rung we tested,

repeated 5–11×. At moderate depths (25/50/75%) every variant scores at or near

5/5; IQ2_XS is 14/15 there. All four degrade somewhat at the 95% extreme;

the ordering tracks bpw but only the Q3_K/IQ3_XXS-vs-IQ2_XS gap is even close to

significant (p = 0.09), so treat the ranking as suggestive.

Decode is depth-flat in all four — e.g. Q3_K holds 12.55 → 12.76 tok/s from

677 to 20,890 tokens — which is the MSA sparse path working as intended rather

than falling back to dense.

Which to pick

• Q3_K (196 GiB) — highest bpw, clean deep retrieval. Slowest.
• IQ3_XXS (180 GiB) — matches Q3_K on retrieval while being faster. Best

quality/speed balance if it fits.

• Q2_K (170 GiB) — faster still, one deep-needle miss in five. Not

statistically separable from the two above (p = 0.33). **Best all-round

choice** for most setups, and the cheapest to reproduce (pure K-quant, no

imatrix required).

• IQ2_XS (160 GiB) — fastest and smallest. Clean at moderate needle depths

(14/15 at 25/50/75%), weaker at the 95% extreme (7/11, vs 10/10 for

Q3_K and IQ3_XXS). That gap does not clear significance at this sample

size (Fisher exact, p = 0.09) — it is suggestive, not established. Reasonable

under RAM pressure; prefer Q2_K if you have the headroom and your workload

puts critical detail at the very end of a near-full context.

The i-quants were not worth their cost here. They took 241 minutes of

quantize time versus 60 for both K-quants, and neither beat the K-quant nearest

it — q2_K at 3.39 bpw matched iq3_xxs at 3.63 on every probe. If you are

rebuilding these yourself, start with the K-quant recipes.

Required llama.cpp settings

Needs llama.cpp with MSA support — PR #24908, merged 2026-07-26 (b10148 or

later). Earlier builds cannot load these files.

-fa on -np 1
--cache-type-k f16 --cache-type-v f16      # MANDATORY, see below
--fit off                                  # MANDATORY, see below
--no-context-shift

KV cache must be f16. A quantized KV cache enables Hadamard attention

rotation (attn_rot_k is gated on `ggml_is_quantized(type_k) && n_embd_head_k %

64 == 0`, and M3's head dim is 128). MSA does not support it and aborts:

GGML_ASSERT(!inp_attn->self_k_rot && !inp_attn->self_v_rot
            && "MSA: attn-rot not supported")  -- src/models/minimax-m3.cpp:337

So -ctk q8_0 crashes on load. f16 KV costs 120 KiB/token. If KV size ever

forces your hand, LLAMA_ATTN_ROT_DISABLE=1 bypasses it at the cost of KV

accuracy.

--fit off is separately required — the automatic memory-fit probe builds a

rope-shift graph and trips the same assert before the model loads.

Two offload gotchas: -ncmoe counts blocks, and M3's first 3 are dense, so

N is offset by 3. -ts splits by layer count, not bytes — on a 96/16 GiB

pair, -ts 90,10 hands the small card six ~3 GiB layers and fails to allocate;

use 95,5.

M3 reasons by default. A tight max_tokens returns empty content, not a

short answer — reasoning consumes the budget and nothing is left for the reply.

Give it room or set --reasoning-budget.

Set thinking_mode explicitly, or reasoning tags leak into content:

--chat-template-kwargs '{"thinking_mode":"enabled"}'

The chat template accepts enabled / disabled / adaptive, and **defaults to

adaptive when the field is unset** — which llama.cpp does not set. In adaptive

mode the model decides per-response whether to think, and when it declines it

emits a bare </mm:think> closing tag with no opener. Nothing matches it, so it

leaks into message.content. Forcing enabled makes the stream well-formed and

reasoning lands in message.reasoning_content as intended.

Tool calling does not work (upstream gap)

Any request containing tools fails:

{"error":{"code":400,"message":"Failed to initialize samplers: failed to parse grammar"}}

MiniMax-M3 uses a bespoke XML tool-call syntax with a namespace token —

]<]minimax[>[<tool_call> wrapping <invoke name="..."> elements — and

llama.cpp has no handler for it: common/chat.cpp contains no MiniMax chat

format, and its registered formats (CONTENT_ONLY, PEG_SIMPLE, PEG_NATIVE,

PEG_GEMMA4) don't match. With tools present it cannot build a grammar and

sampler init fails.

This is an upstream llama.cpp gap, not a defect in these quants, and it affects

every MiniMax-M3 GGUF equally.

Workaround for clients that always send tool definitions (Hermes, many agent

harnesses): send "tool_choice": "none". The grammar is then not built and the

model works normally for chat — you just don't get tool calls.

Verify before you trust

Type census is the whole point of these files, so check rather than take our

word:

python3 census.py MiniMax-M3-Q2_K-00001-of-00018.gguf

Expect 1176 tensors, 228 indexer, and no iq1_s/iq2_s/iq3_s. The script is

included.

Limitations

• Retrieval degrades near the 32K ceiling for all variants at 95% needle

depth. IQ2_XS trends worst (7/11 vs 10/10 for Q3_K and IQ3_XXS) but the gap is

not statistically established — see the sample-size note below.

• The needle probe uses homogeneous repeated filler, which makes the needle more

salient than a realistic long-context task — treat these pass rates as an

upper bound.

• n = 5–11 per cell. **No pairwise difference between variants reaches

statistical significance** (closest is Q3_K+IQ3_XXS vs IQ2_XS at the 95% rung,

p = 0.09). An earlier revision of this card claimed p = 0.036 for that

comparison; that was computed on 6 samples and did not survive 11. The

ordering below tracks bpw and is plausible, but it is not established.

• Tool calling is broken — see the section above. Upstream llama.cpp gap,

affects all MiniMax-M3 GGUFs.

• Untested: vision (an mmproj is provided but unexercised), multi-turn

behaviour, contexts beyond ~29.7K, and any standard benchmark. No perplexity

numbers.

• Measured on one host. Expect different throughput with a different GPU/RAM

split.

Provenance

Converted from the original BF16 safetensors (795.5 GiB, 59 shards) with

convert_hf_to_gguf.py at b10148, then quantized with explicit per-tensor type

pinning. imatrix reused from

Serpen/Minimax-M3-MSA-GGUF

(1524 entries, all 60 layers, no gaps) for the i-quant builds; the K-quant

builds do not require it.

Recipes, census script and needle harness are included so the whole thing is

reproducible.

License

MiniMax Community License, inherited from the base model. Commercial use

requires "Built with MiniMax M3" attribution and a notice email; over $20M

annual revenue requires written authorization. See the base model repository.