---

license: apache-2.0

base_model:

- OliviaRossi/Gemma-4-12B-it-QAT-Hybrid

tags:

- gguf

- llama.cpp

- gemma-4

- qat

- mtp

- vision

- multimodal

- speculative-decoding

- q5_k_l

---

<div align="center">

<img src="https://ai.google.dev/gemma/images/gemma4_banner.png" alt="Gemma 4 Banner" width="100%">

<br><br>

<p align="center">

<a href="https://huggingface.co/collections/google/gemma-4" target="_blank"><b>Hugging Face Collection</b></a> •

<a href="https://github.com/google-gemma" target="_blank"><b>GitHub</b></a> •

<a href="https://blog.google/innovation-and-ai/technology/developers-tools/gemma-4/" target="_blank"><b>DeepMind Blog</b></a> •

<a href="https://ai.google.dev/gemma/docs" target="_blank"><b>Documentation</b></a>

</p>

</div>

Gemma-4-12B-it-QAT-Hybrid-GGUF

High-precision Q5_K_L GGUF quantizations of OliviaRossi/Gemma-4-12B-it-QAT-Hybrid, the Geodesic Model Soup merging Google's official unquantized QAT model with its pristine instruction-tuned base.

1:1 Matched Companion Files

All files follow the exact naming convention required for automatic pairing in LM Studio, Jan, and llama.cpp:

| Filename | Role | Size | Description |

| :--- | :--- | :--- | :--- |

| Gemma-4-12B-it-QAT-Hybrid-Q5_K_L.gguf | Target Model | ~9.1 GB | Base model with Q5_K_M attention/MLP and lossless Q8_0 embeddings & LM head. |

| mtp-Gemma-4-12B-it-QAT-Hybrid-Q5_K_L.gguf | MTP Drafter | ~460 MB | Official QAT-matched MTP drafter (arch: gemma4-assistant) in lossless Q8_0. |

| mmproj-Gemma-4-12B-it-QAT-Hybrid-Q5_K_L.gguf | Vision Projector | ~175 MB | Unified multimodal projector for image analysis, OCR, and document understanding. |

---

Verified Working Launch Commands

1. High-Speed Speculative Decoding (MTP Acceleration)

llama-server \
  -m Gemma-4-12B-it-QAT-Hybrid-Q5_K_L.gguf \
  --spec-draft-model mtp-Gemma-4-12B-it-QAT-Hybrid-Q5_K_L.gguf \
  --spec-type draft-mtp \
  --spec-draft-n-max 3 \
  -ngl 99 \
  -c 16384

> GUI Users (LM Studio / Jan): Ensure you specify --spec-type draft-mtp --spec-draft-n-max 3 under Additional Arguments so the engine initializes the non-causal MTP graph rather than attempting standard causal drafting.

2. Multimodal Vision Mode

llama-server \
  -m Gemma-4-12B-it-QAT-Hybrid-Q5_K_L.gguf \
  --mmproj mmproj-Gemma-4-12B-it-QAT-Hybrid-Q5_K_L.gguf \
  --ubatch-size 2048 \
  -c 16384