---

license: gemma

base_model: google/gemma-4-E4B-it-assistant

tags:

- gguf

- llama.cpp

- gemma4

- mtp

- speculative-decoding

library_name: gguf

---

Gemma 4 E4B — MTP (assistant) head · GGUF

GGUF conversions of google/gemma-4-E4B-it-assistant, the Gemma 4

Multi-Token-Prediction (MTP) assistant head — architecture gemma4-assistant

in llama.cpp.

> These are not a standalone chat model. This is the small speculative /

> draft-mtp head (~76M parameters) that runs alongside a Gemma 4 E4B main

> model to accelerate generation via speculative decoding. Load it as the draft

> model, not as the primary model.

Files

| File | Type |

|------|------|

| gemma-4-E4B-it-assistant-f16.gguf | F16 |

| gemma-4-E4B-it-assistant-Q4_0.gguf | Q4_0 |

| gemma-4-E4B-it-assistant-Q6_K.gguf | Q6_K |

| gemma-4-E4B-it-assistant-Q8_0.gguf | Q8_0 |

All variants are converted from the original bf16 checkpoint with

llama.cpp (commit f955e39).

Choosing a quantization

Most of this head's weights live in the token-embedding tensor, so quantization

mainly shrinks the embedding:

• F16 — highest draft acceptance rate; recommended when disk isn't a concern.
• Q8_0 — near-F16 quality, ~half the size.
• Q6_K — smaller still, minor acceptance loss.
• Q4_0 — smallest; acceptance drops the most (the embedding is quantized to 4-bit).

Because the head is tiny to begin with, the size savings from lower bit-widths

are modest while acceptance decreases — F16 or Q8_0 are the practical picks.

Usage (llama.cpp speculative decoding)

Use this file as the draft/MTP head together with a Gemma 4 E4B main model in a

llama.cpp build that supports gemma4 MTP (draft-mtp) speculative decoding.

The head shares the KV cache with the target model. Typical speculative-decoding

settings: n_max (draft length) of 3 tends to give the best throughput;

higher values reduce acceptance and can end up slower than no speculation.

Base model & license

Derived from the gated model

google/gemma-4-E4B-it-assistant.

Distributed under the Gemma license;

you are responsible for complying with Google's Gemma Terms of Use.