---

license: apache-2.0

base_model: JetBrains/Mellum2.1-12B-A2.5B-Thinking

language:

• en

tags:

• quantization
• gguf
• llama-cpp
• imatrix
• mernik
• humaneval
• evalplus
• code-generation
• moe

---

Mellum2.1-12B-A2.5B-Thinking — MERNIK quants

> STATUS: all builds live — 7500, 10000, 6500 pair. Pick your trade.

Base: JetBrains/Mellum2.1-12B-A2.5B-Thinking (12B MoE, 2.5B active, 28 layers, 64 experts × 8 active).

Method: MERNIK measure-first allocation (smse utility) on the vendor imatrix. MoE routers (ffn_gate_inp) pinned out-of-budget on all 28 layers — the allocator's own rule holds them outside the queue, and the binary lays F32 (measured 28/28 F32 in every file, including stock — same class of behavior as the 1D law).

Honest standings (our protocol, temp 1.0)

| Build | Weight | PPL (wiki) | HumanEval | HumanEval+ | Note |

|:------|-------:|-----------:|----------:|-----------:|:-----|

| MERNIK-7500-SMSE | 7.44 GB | 10.5375 | 76.22% (125/164) | 74.39% (122/164) | ties stock Q4_K_M 125/125 at equal weight (duel 0/3, p=1.0) |

| Stock Q4_K_M | 8.07 GB | 10.1810 | 76.22% (125/164) | 75.00% (123/164) | vendor reference, same harness — the perfect tie |

| Stock Q6_K | 10.88 GB | 10.1562 | 72.56% (119/164) | 70.73% (116/164) | vendor reference, same harness |

| MERNIK-10000-SMSE 👑 | 10.16 GB | 9.5558 | 81.71% (134/164) | 79.27% (130/164) | beats stock Q6 by +15 tasks at −0.7 GB, duel 3/3 SIGNIFICANT |

| MERNIK-6500-SMSE | 7.01 GB | 11.18 | 76.83% (126/164) | 75.61% (124/164) | Q3 territory holds — ties stock Q4_K_M at −1 GB (duel 0/3, triple p=1.0) |

| MERNIK-6500-RECOVERY | 7.01 GB | 10.70 | 75.00% (123/164) | 73.17% (120/164) | twin skeletons, twin verdicts — duel 0/3 NOISE |

Duel 10000-SMSE vs Q6_K: 3/3 SIGNIFICANT (HE 23/8 p=0.0107, HE+ 22/8 p=0.0161,

empties 5/20 p=0.0041) — the first statistically real quant-vs-stock win of the

campaign. The +15 clears the 13-task instrument floor.

Duel 7500-SMSE vs Q6_K: 0/3, NOISE (HE 18/12 p=0.3616, HE+ 18/12 p=0.3616,

empties 13/17 p=0.5847 — all three columns sub-floor).

The +6 lead is a lead, not a crown — stated as lead. The weight saving (−3.4 GB) is arithmetic, not statistics.

Duel 6500-SMSE vs 6500-RECOVERY: 0/3, NOISE (HE 10/7 p=0.6291, HE+ 10/6 p=0.4545,

empties 6/11 p=0.3323). Note the canary/verdict split: recovery owns PPL by −0.48

(10.70 vs 11.18) yet ties on all three verdict columns — the canary demoted, once more.

Empties: 6500-SMSE 33/164, 6500-recovery 38/164 — same silence ladder

(10000: 24, 7500: 35, Q4_K_M: 33, Q6: 39). Note the oddity: stock Q4 goes silent

less than stock Q6 (33 vs 39) while also scoring higher (125 vs 119) — but the

duel is 0/3 NOISE, so the ladder wobbles within sampling. Silence is signal,

just a quiet one.

Note: the Q4 floor sits at 7428 MiB — 7500 is the minimum viable build without

sub-Q4 tiers; 5500 proved impossible (floor 5688 MiB even with Q3 allowed).

Notes

• 10000 build vs the 1070. Two HE attempts died at [54/164] and [90/164] with

ggml-cuda.cu:109 CUDA error (kernel fault under factory OC, not OOM), a third

died at [150/164] to a RAM-OOM (63 GB VM — parallel slots × 8K context).

Cures that held: stock clocks, --parallel 1, --cache-reuse 64.

The closed battery ran on all three.

• Hardware: GTX 1070 8 GB, partial offload (-ngl 20–28), consumer rig, not a datacenter.
• Noise floor: at n=164, gaps below ~13 tasks do not reproduce run to run.

Temp-0 makes the verdict column reproducible (measured: 164/164 identical on re-run).

• Silence scales with weakness: empty completions — 10000: 24/164, 7500: 35/164,

stock Q6: 39/164. Weaker builds go silent rather than babble; the empties column

separated the 10000-vs-Q6 duel at p=0.0041 all by itself.

Protocol (so anyone can re-check)

• Scores: llama-server -ngl {18,20,28} -c 8192 --jinja, sampling

temp 1.0 / top_p 0.95 / top_k 20 / max_tokens 2048, strictly sequential,

fixed seed 7 for PPL (-c 1024 -n 64 -b 512, wikitext-2-raw).

• HE+ via evalplus.evaluate. Three columns or it stays unverified.

Speed tip (interactive use, NOT the benchmark)

For serving (not scoring), ngram speculation lifts codegen from ~60 t/s to

~160 t/s on this model:

llama-server -m MODEL.gguf -ngl 99 -c 8192 --jinja \
  --spec-type ngram-map-k4v --spec-ngram-map-k4v-size-m 12 --parallel 1

All batteries above were scored WITHOUT speculation (strictly sequential,

same protocol as every other build in the lab) — speed tricks never enter

the verdict column.

Reproduce

python main.py --model Mellum2.1-12B-A2.5B-Thinking-BF16.gguf \
  --imatrix Mellum2.1-12B-A2.5B-Thinking-imatrix.gguf \
  --size 7500 --utility smse --output Mellum2.1-12B-A2.5B-Thinking-MERNIK-7500-SMSE.gguf --run

---

If this work saved you a GPU-week: github.com/sponsors/wepiqx