---

base_model: ManniX-ITA/gemma-4-A4B-98e-v7-coderx-it

tags:

- gguf

- imatrix

- quantized

license: apache-2.0

---

gemma-4-A4B-98e-v7-coderx-it-GGUF

GGUF quantizations of ManniX-ITA/gemma-4-A4B-98e-v7-coderx-it.

All quants made using imatrix with calibration data v5.

The imatrix.dat used is included in this repo for reproducibility/audit; mmproj-gemma4.gguf

is the shared Gemma 4 SigLIP vision tower (untouched by pruning) for multimodal use.

Quantizations — score, size & answer length

Every published K-quant and CD tier was scored on HumanEval+ (164) and MultiPL-E-100

(llama.cpp, greedy T=0), with per-problem completion length from token_stats. The table

replaces a bare file-list — it shows, per tier, the actual code score, the exact size, the

true bits-per-weight (bpw = 8 × bytes ÷ 19,877,953,946), and whether the tier stays

length-tight or starts to ruminate at low bit. ⭐ marks a recommended pick.

| Tier | Size | bpw | HE+ % | HE+ tok p50/p90/max | MPE-100 % | MPE tok p50/p90/max |

|---|---:|---:|---:|---|---:|---|

| Q8_0 | 21.16 GB | 8.52 | 92.07 | 230/431/1002 | 88.33 | 85/188/1012 |

| Q6_K_L | 17.98 GB | 7.24 | 92.68 | 229/465/1748 | 89.67 | 84/179/603 |

| Q6_K ⭐ | 17.81 GB | 7.17 | 92.07 | 238/442/3897 | 90.33 | 84/178/975 |

| Q5_K_L | 15.25 GB | 6.14 | 92.07 | 230/483/3374 | 89.00 | 84/193/1011 |

| Q5_K_M | 15.07 GB | 6.07 | 90.24 | 228/445/1715 | 89.67 | 85/184/981 |

| Q4_K_L | 13.42 GB | 5.40 | 92.07 | 251/457/1838 | 89.33 | 86/187/982 |

| Q4_K_M | 13.24 GB | 5.33 | 92.07 | 247/448/1407 | 88.33 | 85/182/1006 |

| Q4_K_S ⭐ | 12.21 GB | 4.91 | 93.29 | 245/445/2187 | 89.00 | 84/190/965 |

| IQ4_NL ⭐ | 11.42 GB | 4.60 | 91.46 | 229/447/1409 | 91.33 | 84/180/902 |

| IQ4_XS | 11.01 GB | 4.43 | 90.85 | 231/426/1321 | 89.67 | 84/180/687 |

| Q3_K_L | 10.94 GB | 4.40 | 92.68 | 240/425/8163 | 88.67 | 86/207/1003 |

| Q3_K_M | 10.51 GB | 4.23 | 92.07 | 239/422/2496 | 89.33 | 86/204/875 |

| CD-Q2_K ⭐ | 8.82 GB | 3.55 | 89.02 | 229/509/3334 | 87.00 | 92/210/1010 |

Recommended picks:

• Q6_K ⭐ (17.81 GB) — max-fidelity tier — 92.07% HE+ / 90.33% MPE-100.
• Q4_K_S ⭐ (12.21 GB) — best K-quant — 93.29% HE+, the highest of any tier.
• IQ4_NL ⭐ (11.42 GB) — best compact — 91.33% MPE-100 (highest of any tier) + 91.46% HE+.
• CD-Q2_K ⭐ (8.82 GB) — smallest tier — 89.02% HE+ at the lowest disk.

Reads: the K-quant and CD tiers hold HE+ in the ~89–93% band and MPE in the 87–91% band

with completion length essentially identical to Q6_K. The 2-bit Q2_K_L / IQ2_XS tiers

are the cliff — HE+ falls into the 80s/70s and the token p90 blows out (budget saturation /

rumination), so the K-quant CD tier(s) (CD-Q2_K) are the recommended low-bit path. The

*i-quant CD bodies (CD-IQ) are not offered** — the pruned MoE degenerates on an IQ-family

body even with critical-tensor protection (token-salad, score → 0).

Head-to-head by file size — v7-coderx vs Qwen2.5-Coder-14B (iso-disk)

Pairing by tier name is misleading — v7-coderx is a ~20.8B-total MoE and Qwen2.5-Coder-14B is a

14.7B dense model, so the same tier name lands at a different file size. The fair comparison is

iso-disk: at a given GB budget, which model scores higher on HumanEval+? Both are measured on

the identical rig (single RTX 3090 24 GB, llama-server -ngl 99 --parallel 2, omk_eval llama

backend, humanevalplus_full, greedy T=0). Qwen GGUFs are bartowski's

Qwen2.5-Coder-14B-Instruct-GGUF;

its ladder sits at 83–85% across the whole stack. At every band below the MoE runs **lower

bpw at the same disk** and still scores higher.

| Disk band | Qwen2.5-Coder-14B (size / bpw / HE+) | v7-coderx best (size / bpw / HE+) | Δ HE+ |

|---|---|---|---:|

| ~21.2 GB | (none — Qwen ceiling is Q8_0 15.70 GB) | Q8_0 21.16 / 8.52 / 92.07% | new top |

| ~17.8 GB | (none — Qwen ceiling is Q8_0 15.70 GB) | Q6_K 17.81 / 7.17 / 92.07% | new top |

| ~15.1 GB | Q8_0 15.70 / 8.54 / 84.76% | Q5_K_M 15.07 / 6.07 / 90.24% | +5.48 |

| ~12.2 GB | Q6_K 12.12 / 6.60 / 84.76% | Q4_K_S 12.21 / 4.91 / 93.29% — ⭐ best K-quant 93.29% | +8.53 |

| ~13.2 GB | Q6_K 12.12 / 6.60 / 84.76% | Q4_K_M 13.24 / 5.33 / 92.07% | +7.31 |

| ~11.4 GB | Q5_K_M 10.51 / 5.72 / 83.54% | IQ4_NL 11.42 / 4.60 / 91.46% — ⭐ best MPE 91.33% | +7.92 |

| ~11.0 GB | Q5_K_M 10.51 / 5.72 / 83.54% | IQ4_XS 11.01 / 4.43 / 90.85% | +7.31 |

| ~10.5 GB | Q5_K_M 10.51 / 5.72 / 83.54% | Q3_K_M 10.51 / 4.23 / 92.07% — iso-disk (~10.5 GB) | +8.53 |

| ~8.8 GB | Q4_K_M 8.99 / 4.89 / 85.37% | CD-Q2_K 8.82 / 3.55 / 89.02% — ⭐ smallest | +3.65 |

Reads:

1. Iso-disk ~10.5 GB. v7-coderx Q3_K_M (10.51 GB / 4.23 bpw / 92.07%) vs Qwen Q5_K_M (10.51 GB / 5.72 bpw / 83.54%): +8.53pp at the exact same file size, −1.49 bpw.
2. Sub-9 GB code-grade. CD-Q2_K (8.82 GB / 3.55 bpw / 89.02%) clears Qwen's whole ladder — its best tier tops out at 85.37% (Q4_K_M, 8.99 GB), so the MoE holds a code-grade ~89% HE+ ~0.2 GB smaller and ~0.87 bpw lower.
3. Every band wins at lower bpw. Across the ladder the MoE uses 1.5–4 bpw less than the

dense Qwen tier at the same disk and still scores higher on HumanEval+ — the point of the iso-disk framing.

> CD recipes are open-source — generator at

> omnimergekit/scripts/generate_cd_maps.py.

How to Use

With llama.cpp:

llama-server -m gemma-4-A4B-98e-v7-coderx-it-Q4_K_M.gguf -c 32768 -ngl 99 \
    --reasoning-format deepseek --reasoning-budget 8192 \
    --temp 1.0 --top-k 64 --top-p 0.95 --min-p 0.05 \
    --repeat-penalty 1.02 --repeat-last-n 2048

The --reasoning-budget flag is required for Gemma 4 thinking — without it the

model emits malformed channel tokens. Keep -c several times larger than the

reasoning budget — with -c equal to the budget the thinking phase can fill the

whole window and the answer degenerates.

Recommended sampling — --repeat-penalty 1.02 --repeat-last-n 2048 is the

field-tested sweet spot for long agentic / tool-calling sessions (validated with

opencode driving multi-turn coding work): it suppresses the intermittent

repetition loops without side effects. Stronger penalties (1.05–1.1) also stop

the loops but cause premature end-of-turn — the model announces a step and then

stops mid-task. Narrower windows (64–1024) at 1.02 let long-period loops through;

keep the full 2048. The remaining sampler values above are the Gemma 4 vendor

defaults baked into the GGUF, stated explicitly so CLI defaults don't override them.

With ollama: ollama pull mannix/gemma4-98e-v7-coderx:Q4_K_M

(:latest = Q4_K_M; :vision-<tier> pairs the tier with the SigLIP vision tower).

---

Gemma 4 A4B 98-Expert v7-coderx — code-maximal prune (~20.8B)

> Eval complete (Q6_K / llama.cpp, greedy, same host). Every cell in the

> scoreboard is read from summary.json under the cohort-pinned greedy recipe

> (temperature 0.0, top_p 1.0, top_k 0). The 128e, v6-coder and v7-coder

> columns are the matching same-host Q6_K runs. The GGUF and NVFP4A16 formats are

> deployment targets and are not separately benchmarked (cohort policy) — the

> Q6_K column is representative.

>

> Headline — the cohort's code specialist. v7-coderx spends its whole prune

> budget on code and short-form reasoning. On the all-hard LiveCodeBench-77 set —

> the most demanding and most discriminating LCB slice — it scores **85.71%, the

> highest in the cohort** (128e 79.22%, v7-coder 84.42%), and it leads on

> HumanEval+ 93.29%, HumanEval 96.95%, MATH-500 95.0% and AIME 76.67%.

> On the easier LCB-medium-v4 slices it sits a little below the generalists

> (LCB-55 92.73 / LCB-100 91.0 vs 128e's 96.36 / 97.0) — those slices no longer

> separate the strongest coders. This is also the loop-fixed build: it

> force-keeps the agentic loop-protection experts and **replaces the earlier looping

> fs2440 prune. The trade is graduate science: GPQA-diamond sits at 51.01%**

> (this recipe carries no targeted_gpqa term). For the broader LCB-medium lead and

> HumanEval, see the sibling v7-coder

> (LCB-55-v4 98.18%, HE 98.17%, LCB-hard-77 84.42%; GPQA ≈ 51 like this model).

A research checkpoint that prunes the unpruned

Gemma 4 26B-A4B-it

(128 experts/layer, top-8 + shared, 30 layers) down to 98 experts per layer. The

drop map concentrates protection on generic-code and LiveCodeBench-medium with

no science or multilingual targeting — the code-maximal member of the v7-coder cohort —

and force-keeps the agentic loop-protection experts so the served model does not

loop. Same 98e shape, same router, same attention, same norms as the rest of the

cohort, plus the mandatory shared-FFN α=1.2 upweight all coder variants carry.

Benchmarks

Q6_K · llama.cpp · greedy (temperature 0.0, top_p 1.0, top_k 0), all four

models scored on the same host from summary.json. Row-max in bold.

This repo = v7-coderx.

| Benchmark | 128e (unpruned) | v6-coder | v7-coder | v7-coderx |

|---|---:|---:|---:|---:|

| GPQA-diamond (198q) | 67.17 | 61.11 | 51.52 | 51.01 |

| AIME (30q) | 73.33 | 56.67 | 80.00 | 76.67 |

| MATH500 (100q) | 92.00 | 89.00 | 95.00 | 95.00 |

| GSM8K (100q) | 89.00 | 88.00 | 91.00 | 93.00 |

| ARC-Challenge (full) | 96.50 | 95.39 | 92.15 | 86.60 |

| IFEval (100q, strict) | 97.00 | 92.00 | 92.00 | 92.00 |

| HumanEval (164) | 97.56 | 98.17 | 98.17 | 96.95 |

| HumanEval+ (164) | 92.07 | 92.68 | 92.07 | 93.29 |

| LCB-medium-55 v4 | 96.36 | 92.73 | 98.18 | 92.73 |

| LCB-medium-100 v4 | 97.00 | 94.00 | 94.00 | 91.00 |

| MultiPL-E (100) | 90.00 | 89.00 | 89.67 | 89.00 |

<sub>Metrics: GPQA & GSM8K = exact_match flexible-extract · MATH500 = math_verify ·

ARC & AIME = exact_match · IFEval = prompt_level_strict_acc · HumanEval/+ = pass@1

chat-extract · LCB-55/100 & MultiPL-E = pass@1. 128e uses the lcb_medium_55/100

templates; the prunes use lcb_medium_*_v4 (corrected harness, equivalent task). The

all-hard LCB-77 cross-model comparison is in the table below.</sub>

v7-coderx leads the cohort on the hardest code slice (LCB-hard-77, below) and on

HE+ / MATH-500 / AIME; the budget is paid on graduate science (GPQA) and on the easier

instruction / ARC axes, which carry no protection term in this recipe.

Coder-field comparison — v7-coderx vs Qwen2.5-Coder-14B / 7B + Qwen3.5-9B (Q6_K, llama.cpp, greedy)

The 9 canonical benches + MultiPL-E-100, all on the identical llama.cpp Q6_K / greedy

recipe (reasoning models served with `--reasoning-format deepseek --reasoning-budget 12288

--parallel 2`). Architectures differ — this is a same-harness comparison, not a same-class one:

• v7-coderx — Gemma-4 26B-A4B MoE pruned to 98 experts (~20.8B total, ~A4B active), reasoning.
• Qwen2.5-Coder-14B / 7B-Instruct — dense, non-reasoning code specialists (bartowski Q6_K).
• Qwen3.5-9B — dense reasoning model (bartowski Q6_K).

| Bench (n) | v7-coderx Q6_K | Qwen2.5-Coder-14B | Qwen2.5-Coder-7B | Qwen3.5-9B |

|---|---:|---:|---:|---:|

| ARC-Challenge-chat (1172) | 86.60% | 90.53% | 85.58% | 96.76% |

| GPQA Diamond flex (198) | 51.01% | 34.85% | 26.26% | 73.74% |

| GSM8K-100 flex | 93.00% | 89.00% | 80.00% | 79.00% |

| MATH-500-100 math_verify | 95.00% | 62.00% | 66.00% | 59.00% |

| AIME 2024 (30) | 76.67% | 10.00% | 10.00% | 56.67% |

| IFEval-100 (prompt_strict) | 92.00% | 68.00% | 54.00% | 93.00% |

| HumanEval-164 chat | 96.95% | 90.85% | 87.20% | 89.02% |

| HumanEval+-164 chat | 93.29% | 84.76% † | 83.54% | 80.49% |

| LCB-medium-55 v4 | 92.73% | 18.18% † | 12.73% | 58.18% |

| MultiPL-E-100 (macro) | 89.00% | 84.67% | 80.67% | 80.33% |

† Qwen2.5-Coder-14B HumanEval+ / LCB-medium-55 are the same-stack GGUF HE+ sweep numbers

(not re-run in this chain). All Qwen cells are the same-host reference runs used on the

v6-coder card — Qwen is a

fixed reference, so the columns are identical across the cohort; only the Gemma column changes.

> Note on Qwen3.5-9B. Qwen3.5-9B is a verbose, slow thinking model: it emits long

> <think> reasoning chains (often ≥1900 tokens even on a trivial GSM8K question), so it runs

> several× slower per question than the non-reasoning Qwen2.5-Coder models — well beyond what

> its 9B size would suggest. Its GSM8K / MATH-500 / GPQA cells were re-run after a harness fix

> (under batched, reasoning-parsed serving the verbose thinking intermittently left the final

> answer inside the reasoning block, mis-scored as empty content).

LiveCodeBench across problem sets

v7-coderx's code score depends on the LiveCodeBench slice. All cells are the **same greedy

Q6_K / imat-Q6 llama.cpp stack** (build provenance verified per run); v4-55/100 mirror the

9-bench above. The all-hard 77q set is the most demanding and the most discriminating

across the cohort.

| LCB problem set | 128e | v7-coder | v7-coderx |

|---|---:|---:|---:|

| LCB-medium-55 (v4, 55q) | 96.36% | 96.36% | 92.73% |

| LCB-medium-100 (v4, 100q) | 97.00% | 97.00% | 91.00% |

| LCB-v6-55 (55q) † | — | 92.73% | 98.18% |

| LCB-hard-77 (all-hard, 77q) | 79.22% | 84.42% | 85.71% |

† LCB-v6-55 is a small, noisier 55-problem slice (no greedy 128e baseline was run); it is

included for completeness, but all-hard 77q is the reference for cross-model comparison.

Answer-length analysis (anti-rumination)

The pruned reasoning model thinks with a bounded thinking_token_budget=12288; the

question is whether that length is productive (long thinking that PASSes) or

rumination (long thinking that fails). Per-problem completion length is measured from

omk_eval token_stats (characters from the raw completion; tokens via the 128e tokenizer)

on the real-n benches, against 128e and v6-coder on the same problems, same greedy

Q6_K / llama.cpp stack.

Per-problem completion length — characters (p50 / p90 / max):

| Bench (n) | 128e | v6-coder | v7-coderx |

|---|---|---|---|

| GPQA Diamond (198) | 2571/16136/27811 | 2582/16100/25243 | 2458/17984/32411 |

| AIME 2024 (30) | 1963/7748/8680 | 2141/7469/9433 | 2061/7449/9660 |

| LCB-medium-55 | 3734/16430/36462 | 31015/36260/43278 | 31167/38631/41953 |

| LCB-medium-100 | 2056/15467/48569 | 29384/35389/43633 | 29883/36439/55504 |

| MultiPL-E-100 (300) | 245/566/3353 | 245/573/2725 | 244/619/1861 |

| MATH-500 (100) | 1083/1873/7899 | 1089/2025/9236 | 1080/1981/2312 |

| GSM8K (100) | 294/746/25989 | 283/780/11378 | 279/676/19868 |

| IFEval (100) | 877/3755/8263 | 855/3489/20908 | 791/3702/8179 |

| HumanEval (164) | 698/1284/5354 | 711/1438/5954 | 745/1427/4044 |

| HumanEval+ (164) | 714/1461/3289 | 694/1390/5282 | 742/1423/4461 |

| ARC-Challenge (1172) | 1210/1633/6254 | 1221/1674/48886 | 1335/2193/45174 |

Per-problem completion length — tokens (p50 / p90 / max):

| Bench (n) | 128e | v6-coder | v7-coderx |

|---|---|---|---|

| GPQA Diamond (198) | 843/8189/8189 | 879/8189/8189 | 837/8189/8189 |

| AIME 2024 (30) | 933/3994/4021 | 946/3993/4011 | 950/3994/4013 |

| LCB-medium-55 | 1005/5622/16022 | 12818/13318/15976 | 12834/13286/16019 |

| LCB-medium-100 | 542/5353/16022 | 12740/13212/15976 | 12750/13374/15976 |

| MultiPL-E-100 (300) | 84/171/1013 | 85/184/965 | 84/182/976 |

| MATH-500 (100) | 431/895/3377 | 424/863/3377 | 403/832/1219 |

| GSM8K (100) | 131/271/8853 | 129/276/4687 | 122/258/6957 |

| IFEval (100) | 219/850/1561 | 222/797/3898 | 186/801/4057 |

| HumanEval (164) | 226/431/1611 | 226/448/2084 | 235/437/1463 |

| HumanEval+ (164) | 226/455/996 | 224/437/2040 | 230/459/1440 |

| ARC-Challenge (1172) | 258/355/1417 | 259/365/16266 | 282/487/16260 |

Budget-saturation incidence — share of problems whose completion reached ≥12k tokens

(at/near the thinking_token_budget=12288 cap). Saturation by itself is not rumination —

a saturated output that PASSes is productive use of the budget; the pruned reasoning model

saturates on nearly every LCB problem, 128e almost never does.

| Bench (n) | 128e | v6-coder | v7-coderx |

|---|---:|---:|---:|

| LCB-medium-55 | 1 / 55 (1.8%) | 54 / 55 (98.2%) | 54 / 55 (98.2%) |

| LCB-medium-100 | 2 / 100 (2.0%) | 98 / 100 (98.0%) | 97 / 100 (97.0%) |

Rumination — long thinking that fails to PASS. The right metric is not median length

(128e looks short only because it answers easy problems fast). It is the share of the model's

budget-saturated outputs that still fail — tokens burned without a correct answer:

| Bench (n) | 128e | v6-coder | v7-coderx |

|---|---:|---:|---:|

| LCB-medium-55 — saturated-and-failed | 1 / 1 (100.0%) | 4 / 54 (7.4%) | 4 / 54 (7.4%) |

| LCB-medium-100 — saturated-and-failed | 2 / 2 (100.0%) | 6 / 98 (6.1%) | 9 / 97 (9.3%) |

| LCB-100 — mean completion tokens, PASS vs FAIL | 1392 vs 13782 | 12698 vs 15051 | 12623 vs 15143 |

Key findings:

• 128e only thinks long when it is lost. Every 128e output that reaches the budget cap is

a failure (1/1 on LCB-55, 2/2 on LCB-100), and its failed problems run several× longer than

its passed ones (mean 13782 vs 1392 tok on LCB-100).

• v7-coderx's long thinking is overwhelmingly productive. It saturates on ~97% of LCB-100

problems but only 9/97 of those saturated outputs fail (9.3%); its PASS and FAIL

completions are nearly the same length (mean 12623 vs 15143 tok), so failures are not

driven by extra rumination. On LCB-55 it is 4/54 saturated-and-failed.

• Comparable to v6-coder's rumination rate. v6-coder ran 4/54 (LCB-55) and

6/98 (LCB-100) saturated-and-failed; v7-coderx is at or below on LCB-55 (4/54) and near

on LCB-100 (9/97). The saturated-fail share tracks the model's LCB pass-rate — these are the

genuinely hard problems, not extra rumination (PASS and FAIL completions are near-equal length).

• Non-LCB benches stay tight. On the short-answer benches (GSM8K / MATH-500 / HE / HE+ /

MultiPL-E) p50/p90 length tracks 128e and v6-coder within a few tokens — the targeted prune

did not trade length for accuracy on the everyday benches.

> Methodology. Per-problem lengths come from omk_eval token_stats over each bench's

> samples_*.jsonl / lcb_result.samples.jsonl; saturation/PASS-FAIL is computed per problem

> from completion_tokens + passed. MultiPL-E measures code length, not reasoning (its

> samples store only the final code block, no <think> trace), so it is a code-conciseness

> reference rather than a thinking-length signal.

Recipe (summary)

98e prune from 128e on the v7 competence map with the generic-code and

LiveCodeBench-medium classes up-weighted (no science or multilingual targeting),

then the mandatory shared-FFN α=1.2 upweight, plus an **agentic loop-protection

force-keep** that pins the experts emitting end-of-turn / answer-channel tokens. This

is the published, loop-fixed build — it replaces the earlier looping fs2440

prune. v7-coderx is the code-maximal sibling of

v7-coder — same map,

minus the targeted_gpqa science term. Full recipe and calibration-class table are on

the bf16 card.

Intended use & limitations

Compact (~12 GB at Q4_K_M, single 12–16 GB GPU) Gemma 4 checkpoint for **maximal

coding throughput and instruction-following**. Serve with the reasoning parser

enabled. A research prune, not an official Google release; generic_multilingual is

de-weighted (0×) and graduate science (GPQA) is the explicit budget axis — at

51.01% it is well below the unpruned 128e (67.17%). Choose v7-coder if science

matters. Prefer Q4_K_M or higher.

Lineage

128e → (v4 → v5 → v6-coder code line) → v7 competence-map rebuild → **code

floor + agentic loop-protection force-keep** = v7-coderx. Built and evaluated on the

omnimergekit toolchain.