---

license: apache-2.0

library_name: llama.cpp

tags:

• gguf
• gemma
• llama-cpp
• quantized
• MTP
• speculative-decoding

base_model:

• google/gemma-4-E4B-it-qat-q4_0-unquantized-assistant

---

Gemma-4-E4B-it-qat-q4_0-unquantized-assistant - GGUF Quantized Model

This model is a GGUF quantized version of the google/gemma-4-E4B-it-qat-q4_0-unquantized-assistant base model, converted using llama.cpp's https://github.com/ggml-org/llama.cpp/blob/master/convert_hf_to_gguf.py script.

Quantization Details

• Base Model: google/gemma-4-E4B-it-qat-q4_0-unquantized-assistant
• Conversion Tool: llama.cpp https://github.com/ggml-org/llama.cpp/blob/master/convert_hf_to_gguf.py
• Quantization Method: Q8_0 (8-bit quantization)
• Format: GGUF (GPT-Generated Unified Format)

Model Files

| File | Description |

|------|-------------|

| gemma-4-E4B-it-qat-assistant-MTP-Q8_0.gguf | Draft model for speculative decoding (Q8_0) |

Usage with llama-server

Below is the recommended llama-server command to run this model with optimal settings including MTP (Multi-Token Prediction) speculative decoding:

llama-server -hf google/gemma-4-E4B-it-qat-q4_0-gguf \
  --temp 1.0 \
  --top-p 0.95 \
  --top-k 64 \
  --ui-mcp-proxy \
  -c 64000 \
  -fa off \
  --jinja \
  --metrics \
  --spec-type draft-mtp \
  --spec-draft-n-max 2 \
  --parallel 1 \
  --spec-draft-hf amaranus/Gemma-4-E4B-it-qat-assistant-MTP-Q8_0-GGUF

| Parameter | Value | Description |

| --- | --- | --- |

| -hf | google/gemma-4-E4B-it-qat-q4_0-gguf | Load model directly from Hugging Face Hub |

| --temp | 1.0 | Sampling temperature (higher = more creative) |

| --top-p | 0.95 | Nucleus sampling threshold |

| --top-k | 64 | Top-K sampling (only consider top 64 tokens) |

| --ui-mcp-proxy | - | Enable UI MCP proxy interface |

| -c | 64000 | Context length (64K tokens) |

| -fa | off | Flash Attention disabled (use standard attention) |

| --jinja | - | Enable Jinja2 template processing for chat formats |

| --metrics | - | Enable metrics endpoint for monitoring |

| --spec-type | draft-mtp | Speculative decoding type: Multi-Token Prediction |

| --spec-draft-n-max | 2 | Maximum number of speculative tokens per step |

| --parallel | 1 | Parallel sequences (1 = single sequence) |

| --model-draft | gemma-4-E4B-it-qat-assistant-MTP-Q8_0.gguf | Path to draft model (higher quality Q8\_0) |

---

<div align="center">




</div>