ManniX-ITA/gemma-4-12B-it-assistant-GGUF overview
Gemma 4 12B IT Assistant — MTP draft head llama.cpp gemma4 assistant format MTP / speculative decoding draft head for gemma 4 12B it , in the upstream llama.cp…
Runs locally from ~307.9 MB disk (4 GB VRAM class GPUs with llama.cpp / guIDE).
Repository Files & Downloads
| File | Type | Quantization | Size | Link |
|---|---|---|---|---|
| gemma-4-12B-it-assistant.F16.gguf | GGUF | GGUF | 821.6 MB | Download |
| gemma-4-12B-it-assistant.Q4_K_M.gguf | GGUF | GGUF | 312.1 MB | Download |
| gemma-4-12B-it-assistant.Q4_K_S.gguf | GGUF | GGUF | 307.9 MB | Download |
| gemma-4-12B-it-assistant.Q5_K_M.gguf | GGUF | GGUF | 328.5 MB | Download |
| gemma-4-12B-it-assistant.Q8_0.gguf | GGUF | GGUF | 443.6 MB | Download |
Model Details
| Model ID | ManniX-ITA/gemma-4-12B-it-assistant-GGUF |
|---|---|
| Author | ManniX-ITA |
| Pipeline | — |
| License | gemma |
| Base model | — |
| Last modified | 2026-09-09T05:22:16.000Z |
Model README
---
license: gemma
library_name: gguf
tags:
- gguf
- gemma4
- speculative-decoding
- mtp
- draft-model
- opencoti
---
Gemma-4 12B IT Assistant — MTP draft head (llama.cpp gemma4-assistant format)
MTP / speculative-decoding draft head for gemma-4-12B-it, in the upstream
llama.cpp gemma4-assistant GGUF data-plane format (arch gemma4-assistant, with
nextn. (+ masked_embd_ where applicable) tensors and embedding_length_out).
Use it as the drafter for self-speculative decoding against the matching
gemma-4-12B-it target:
--mtp-head gemma-4-12B-it-assistant.Q8_0.gguf --spec-type draft-assistant -ngld 99Quants: F16, Q8_0, Q5_K_M, Q4_K_M, Q4_K_S.
Adapted from google/gemma-4-12B-it-assistant (converted from safetensors). Converted from the official Google safetensors via the AtomicBot converter, then emitted directly in the upstream gemma4-assistant data-plane format. The Gemma license applies.
Run ManniX-ITA/gemma-4-12B-it-assistant-GGUF with guIDE
Download guIDE — the AI-native code editor with local LLM inference and 69 built-in tools.
Source: Hugging Face · Compare models