majentik/gemma-4-12B-it-RotorQuant-GGUF-Q3_K_M overview
TIP KV cache quantization upstream, no fork needed : llama.cpp/Ollama cover this natively — ctk q8 0 ctv q8 0 ~half KV memory, negligible quality loss or ctk q…
Runs locally from ~5.67 GB disk (8 GB VRAM class GPUs with llama.cpp / guIDE).
Repository Files & Downloads
| File | Type | Quantization | Size | Link |
|---|---|---|---|---|
| gemma-4-12B-it-Q3_K_M.gguf | GGUF | Q3_K_M | 5.67 GB | Download |
Model Details
| Model ID | majentik/gemma-4-12B-it-RotorQuant-GGUF-Q3_K_M |
|---|---|
| Author | majentik |
| Pipeline | text-generation |
| License | gemma |
| Base model | google/gemma-4-12B-it |
| Last modified | 2026-07-20T16:21:09.000Z |
Model README
---
base_model: google/gemma-4-12B-it
license: gemma
library_name: gguf
pipeline_tag: text-generation
tags:
- gguf
- quantized
- gemma
- gemma4
- llama.cpp
- text-only
- rotorquant
---
> [!TIP]
> KV-cache quantization (upstream, no fork needed): llama.cpp/Ollama cover
> this natively — -ctk q8_0 -ctv q8_0 (~half KV memory, negligible quality
> loss) or -ctk q4_0 -ctv q4_0 (~quarter memory, small quality cost). In
> Ollama: OLLAMA_KV_CACHE_TYPE=q8_0 with OLLAMA_FLASH_ATTENTION=1.
gemma-4-12B-it — RotorQuant GGUF Q3_K_M
google/gemma-4-12B-it @ 12ace6d648d72bd41519e140f1185f34d38c7e3d quantized pack, published as majentik/gemma-4-12B-it-RotorQuant-GGUF-Q3_K_M.
Method
llama.cpp Q3_K_M quantization, text-only (vision/audio not included).
Release line
Released under the RotorQuant line. RotorQuant and TurboQuant are this project's release labels for this pack, not distinct quantization algorithms — both brand repos for a given tier carry byte-identical weights, produced once and published under two names. No brand-specific speedup is claimed or measured for either label.
Modality
TEXT-ONLY. This is a llama.cpp GGUF conversion of the text tower only — the vision and audio towers are NOT included in this pack. Do not expect image or audio understanding from this repo; use one of the MLX packs for multimodal input.
License
Governed by the Gemma Terms of Use. See the upstream repo for the full license text.
Run majentik/gemma-4-12B-it-RotorQuant-GGUF-Q3_K_M with guIDE
Download guIDE — the AI-native code editor with local LLM inference and 69 built-in tools.
Source: Hugging Face · Compare models