Mediform/gemma4-e4b-v13-assistant-rollout-gguf overview
gemma4 e4b v13 assistant rollout — GGUF BF16 + Q8 0 llama.cpp GGUF of Scribion's MTP draft assistant , rollout distilled against the finetuned v13 target E4B p…
Runs locally from ~94.1 MB disk (4 GB VRAM class GPUs with llama.cpp / guIDE).
Repository Files & Downloads
Model Details
Model README
---
license: gemma
base_model: google/gemma-4-E4B-it-assistant
tags:
- gguf
- llama.cpp
- speculative-decoding
- mtp
- gemma4
---
gemma4-e4b-v13-assistant-rollout — GGUF (BF16 + Q8_0)
llama.cpp GGUF of Scribion's MTP draft assistant, rollout-distilled against the finetuned
v13 target (E4B plain-LoRA r16). EAGLE-style multi-step rollout distillation on in-domain German
medical extraction data lifts deep-draft acceptance on long dialogues (froehlich +13.6% accept/step
at draft length 7) — a pure decode-speed win (speculative decoding is exact, output unchanged).
Files
| file | precision | size |
|------|-----------|------|
| assistant-rollout-bf16.gguf | bf16 | 172 MB |
| assistant-rollout-q8_0.gguf | Q8_0 | 99 MB |
78.5M-param 4-layer EAGLE-style draft model. Pair it as the draft model for speculative decoding
with the target:
- target GGUF: Mediform/gemma4-e4b-v13-plainlora-r16-gguf
- MLX-swift assistant: Mediform/gemma4-e4b-v13-assistant-rollout-mlx-bf16
Requires a llama.cpp build with Gemma-4 MTP-assistant / speculative support.
Run Mediform/gemma4-e4b-v13-assistant-rollout-gguf with guIDE
Download guIDE — the AI-native code editor with local LLM inference and 69 built-in tools.
Source: Hugging Face · Compare models