Mediform/gemma-4-E4B-it-assistant-gguf overview
gemma 4 E4B it assistant MTP draft — GGUF F16 F16 GGUF conversion of google/gemma 4 E4B it assistant — the stock MTP multi token prediction assistant head for …
Runs locally from ~94.1 MB disk (4 GB VRAM class GPUs with llama.cpp / guIDE).
Repository Files & Downloads
| File | Type | Quantization | Size | Link |
|---|---|---|---|---|
| gemma-4-E4B-it-assist-F16.gguf | GGUF | F16 | 94.1 MB | Download |
Model Details
Model README
---
license: gemma
base_model: google/gemma-4-E4B-it-assistant
tags:
- gguf
- speculative-decoding
- mtp
- gemma-4
---
gemma-4-E4B-it assistant (MTP draft) — GGUF F16
F16 GGUF conversion of google/gemma-4-E4B-it-assistant — the stock
MTP (multi-token-prediction) assistant head for Gemma-4 E4B-it. Finetune
revision 65892304d4eb7762acc45257a327885f7535e584.
Purpose
Draft model for speculative decoding against the stock
google/gemma-4-E4B-it-qat-q4_0-gguf target (llama.cpp
--spec-type draft-mtp / ngram-mod,draft-mtp, or Scribion's in-process
LlamaCppGemmaEngine). The assistant reads the target's hidden states, so
draft/target pairing matters:
| target | draft | mean accepted len (measured) |
|---|---|---|
| stock QAT q4_0 | this model | ~2.9 / 3 |
| stock QAT q4_0 | Mediform/gemma4-e4b-v13-assistant-rollout-gguf | ~2.1 / 3 (mismatched) |
Use Mediform/gemma4-e4b-v13-assistant-rollout-gguf when the target is the
v13-plainlora finetune; use this model when the target is stock QAT.
Speculative decoding is lossless — a mismatched draft only costs speed
(~28 vs ~36-40 tok/s decode on M-series), never output quality.
Run Mediform/gemma-4-E4B-it-assistant-gguf with guIDE
Download guIDE — the AI-native code editor with local LLM inference and 69 built-in tools.
Source: Hugging Face · Compare models