---

license: apache-2.0

base_model: JetBrains/Mellum2-12B-A2.5B-Instruct

base_model_relation: quantized

tags: [gguf, llama.cpp, rocm, gfx1151, strix-halo, amd, ryzen-ai-max-395, rocmfpx, mellum, moe, code, code-completion]

language: [en]

pipeline_tag: text-generation

---

> ### ⚠️ STOCK llama.cpp WILL NOT LOAD THIS MODEL

>

> The Mellum architecture is not merged upstream (PR

> #23966) and these are ROCmFPX quant types.

>

> 11.88 GiB · 74.93 tok/s on a Ryzen AI MAX+ 395.

Mellum2-12B-A2.5B-Instruct — ROCmFPX 8-bit AGENT GGUF

An 8-bit ROCmFPX quantization for AMD gfx1151 (Ryzen AI MAX+ 395 / Strix Halo), quantized

from BF16 GGUF (24,311,968,416 B) — a lossless source, byte-verified against the upstream blob.

| | |

|---|---|

| File | Mellum2-12B-A2.5B-Instruct-Q8_0_ROCMFPX_AGENT.gguf |

| Size | 11.88 GiB (12,754,956,960 B) |

| BPW | 8.39 |

| ftype | Q8_0_ROCMFPX_AGENT (115) |

⛔ Requires a llama.cpp with the ROCmFPX quant types

Q8_0_ROCMFPX (111) / Q8_0_ROCMFPX_AGENT (115) exist only in

charlie12345/ROCmFPX. Stock llama.cpp reports

invalid ggml type 103. The Mellum arch itself is also not merged upstream

(PR #23966). Ignore the auto-generated

"Use this model" commands above.

---

All quant variants

All measured on one box, one binary (Ryzen AI MAX+ 395, gfx1151, ROCm 7.2.4), median of 3,

warm-up discarded — so these rows are directly comparable.

| variant | ftype | size | bpw | decode (median) | range |

|---|---|---|---|---|---|

| 4-bit COHERENT | 102 | 6.49 GiB | 4.59 | 104.99 | 104.96 – 105.73 |

| 8-bit AGENT | 115 | 11.88 GiB | 8.39 | 74.93 | 74.93 – 74.97 |

| 8-bit plain | 111 | 11.70 GiB | 8.27 | 72.76 | 72.61 – 72.79 |

Repos: 4-bit ·

8-bit AGENT ·

8-bit plain

AGENT is faster here — 74.93 vs 72.76, ranges disjoint (+3.0%). Both 8-bit builds are well below the 4-bit build's 104.99 tok/s; they exist for accuracy headroom, not speed.

> On AGENT generally: it keeps more tensors at true Q8_0 instead of the packed 8-bit type.

> That raises MTP draft acceptance on models which have an MTP head (measured +6.2% on

> Qwen3.8-27B). Mellum2 has no MTP head, so there is nothing for the extra precision to feed

> and the two 8-bit builds differ only marginally — in either direction.

Correctness: 17×23 ⇒ ✅ 391 · capital of Japan ⇒ ✅ Tokyo · days in 2024 ⇒ ✅ 366

(checked with a 1024-token budget — a small budget makes these models look broken by spending it

all inside <think> and returning empty content.)

---

What was NOT measured

• No perplexity run, and no quality A/B against the source. The checks above are

memorized-fact prompts — necessary but not sufficient.

• No long-context testing. · No tool-calling evaluation.

Base model licence inherited; credit for the model goes to its authors.