---

base_model: ManniX-ITA/gemma-4-A4B-98e-v4-it

tags:

- gguf

- imatrix

- quantized

- gemma4

- moe

- expert-pruning

license: other

license_name: gemma

---

gemma-4-A4B-98e-v4-it-GGUF

GGUF quantizations of ManniX-ITA/gemma-4-A4B-98e-v4-it — Gemma 4 26B pruned to 98 experts per layer (from 128) using a multi-class ContribDynamic (CD) drop map with max-over-classes aggregation.

> Eval results pending: numbers are being re-collected on the locked methodology in omnimergekit/eval/EVAL_PROTOCOL.md. The previous pod-side numbers were discarded due to settings drift and token-budget truncation. See the model card for status.

All quants made using imatrix with calibration data v5. Includes ContribDynamic (CD) quants with per-layer dynamic quantization based on expert contribution analysis.

Available Quantizations

Standard Quants

| Quantization | File | Size |

|---|---|---|

| Q8_0 | gemma-4-A4B-98e-v4-it-Q8_0.gguf | 21.16 GB |

| Q6_K_L | gemma-4-A4B-98e-v4-it-Q6_K_L.gguf | 17.98 GB |

| Q6_K | gemma-4-A4B-98e-v4-it-Q6_K.gguf | 17.81 GB |

| Q5_K_L | gemma-4-A4B-98e-v4-it-Q5_K_L.gguf | 15.25 GB |

| Q5_K_M | gemma-4-A4B-98e-v4-it-Q5_K_M.gguf | 15.07 GB |

| Q5_K_S | gemma-4-A4B-98e-v4-it-Q5_K_S.gguf | 14.19 GB |

| Q4_K_L | gemma-4-A4B-98e-v4-it-Q4_K_L.gguf | 13.42 GB |

| Q4_K_M | gemma-4-A4B-98e-v4-it-Q4_K_M.gguf | 13.24 GB |

| Q4_1 | gemma-4-A4B-98e-v4-it-Q4_1.gguf | 12.61 GB |

| Q4_K_S | gemma-4-A4B-98e-v4-it-Q4_K_S.gguf | 12.21 GB |

| Q4_0 | gemma-4-A4B-98e-v4-it-Q4_0.gguf | 11.42 GB |

| IQ4_NL | gemma-4-A4B-98e-v4-it-IQ4_NL.gguf | 11.53 GB |

| IQ4_XS | gemma-4-A4B-98e-v4-it-IQ4_XS.gguf | 11.12 GB |

| Q3_K_XL | gemma-4-A4B-98e-v4-it-Q3_K_XL.gguf | 10.69 GB |

| Q3_K_L | gemma-4-A4B-98e-v4-it-Q3_K_L.gguf | 10.94 GB |

| Q3_K_M | gemma-4-A4B-98e-v4-it-Q3_K_M.gguf | 10.51 GB |

| IQ3_M | gemma-4-A4B-98e-v4-it-IQ3_M.gguf | 9.82 GB |

| Q3_K_S | gemma-4-A4B-98e-v4-it-Q3_K_S.gguf | 9.68 GB |

| IQ3_XS | gemma-4-A4B-98e-v4-it-IQ3_XS.gguf | 9.22 GB |

| IQ3_XXS | gemma-4-A4B-98e-v4-it-IQ3_XXS.gguf | 8.95 GB |

| IQ2_M | gemma-4-A4B-98e-v4-it-IQ2_M.gguf | 8.22 GB |

| IQ2_S | gemma-4-A4B-98e-v4-it-IQ2_S.gguf | 7.83 GB |

| IQ2_XS | gemma-4-A4B-98e-v4-it-IQ2_XS.gguf | 7.77 GB |

| IQ2_XXS | gemma-4-A4B-98e-v4-it-IQ2_XXS.gguf | 7.37 GB |

| Q2_K_L | gemma-4-A4B-98e-v4-it-Q2_K_L.gguf | 8.58 GB |

| Q2_K | gemma-4-A4B-98e-v4-it-Q2_K.gguf | 8.40 GB |

ContribDynamic (CD) Quants

Per-layer dynamic quantization based on expert contribution analysis. Important layers get higher precision.

| Quantization | File | Size |

|---|---|---|

| CD-Q6_K | gemma-4-A4B-98e-v4-it-CD-Q6_K.gguf | 15.43 GB |

| CD-Q5_K_M | gemma-4-A4B-98e-v4-it-CD-Q5_K_M.gguf | 13.01 GB |

| CD-Q4_K_M | gemma-4-A4B-98e-v4-it-CD-Q4_K_M.gguf | 10.59 GB |

| CD-Q3_K_M | gemma-4-A4B-98e-v4-it-CD-Q3_K_M.gguf | 9.94 GB |

| CD-Q2_K | gemma-4-A4B-98e-v4-it-CD-Q2_K.gguf | 8.40 GB |

Imatrix Calibration

• imatrix.dat — included in this repo (44 MB). bartowski calibration_datav5, ngl=15 on RTX 3090.

How to Use

With llama.cpp:

llama-server -m gemma-4-A4B-98e-v4-it-Q4_K_M.gguf \
    --port 8099 -c 32768 -ngl 99 --no-warmup --jinja \
    --reasoning-format deepseek --reasoning-budget 16384 \
    --temp 1.0 --top-p 0.95 --top-k 64 --dry-multiplier 0.5

With ollama:

ollama run mannix/gemma4-98e-v4:Q4_K_M

All published Ollama tags: mannix/gemma4-98e-v4.

License

Inherits the Gemma license from the base model.