---

license: gemma

base_model: yuxinlu1/gemma-4-12B-coder-fable5-composer2.5-v1-GGUF

base_model_relation: quantized

tags:

- gemma4

- coding

- code

- python

- reasoning

- thinking

- gguf

- imatrix

- llama.cpp

- vulkan

- amd

- cpu

- local-llm

library_name: gguf

pipeline_tag: text-generation

quantized_by: Lna-Lab

---

💻 Gemma-4-12B-Coder (fable5 × composer2.5) — imatrix GGUF ✨

Runs anywhere llama.cpp runs — AMD/Vulkan, CPU, Apple, NVIDIA. No Blackwell, no MTP, just GGUF. 🐧🍎🪟

> Importance-matrix (imatrix) quants of yuxinlu1's coding model, calibrated on real Python coding data so the low-bit builds keep their coding smarts. Text-only (a coding model — no vision baggage). 💚

---

🙏 Credit

Quants of yuxinlu1/gemma-4-12B-coder-fable5-composer2.5-v1 — all thanks to @yuxinlu1 for the model. ⭐ the original and watch it for a v2! The author's recipe: a fine-tune of google/gemma-4-12B-it on execution-verified Python coding chains-of-thought (Composer 2.5 real CoT + a Fable 5 "second-attempt" set for the hard cases). It thinks in Gemma's native channel, then writes clean, runnable code. De-refused; Python/algorithmic focus; English-centric.

Why this repo: the originals are static GGUF. These add an importance matrix (code-calibrated) so IQ4_XS / Q4_K keep more quality at low VRAM — the builds that fly for AMD/Vulkan and CPU folks.

---

📦 Pick your quant (all imatrix)

| Quant | Size | Vibe |

|------|------|------|

| 🟢 Q3_K_S | 5.53 GB | smallest that works — for 8 GB / 6 GB cards (leaves room for context). ~91.7% HumanEval[:12] |

| 🟢 Q3_K_M | 6.09 GB | tiny and sharp — 100% HumanEval[:15] |

| 🔵 IQ4_XS | 6.64 GB | the imatrix 4-bit sweet spot — 100% HumanEval[:15] |

| 🔵 Q4_K_M | 7.38 GB | balanced (embeddings/output at Q6_K) |

| ⚪ Q5_K_M | 8.55 GB | quality-first if you have the RAM/VRAM |

> 💡 8 GB VRAM (or 6 GB): grab Q3_K_S (5.5 GB) — it leaves headroom for context and still codes well. On the Vulkan backend (AMD) all of these fly.

> ⚠️ Avoid IQ3 (i-quant 3-bit) for this model — IQ3_XXS/IQ3_S collapse to gibberish here (gemma-4's special attention layers don't survive 3-bit i-quants). The *Q3_K_ K-quants** stay coherent at the same size — that's why the small tiers are Q3_K, not IQ3.

---

🚀 Run it (llama.cpp — any backend)

# build llama.cpp with your backend (Vulkan for AMD):  cmake -B build -DGGML_VULKAN=ON && cmake --build build
# grab one quant:
hf download sakamakismile/gemma-4-12B-coder-fable5-composer2.5-GGUF gemma-4-12B-coder-fable5-composer2.5-IQ4_XS.gguf --local-dir .

# chat server (OpenAI-compatible at http://localhost:8080)
./llama-server -m gemma-4-12B-coder-fable5-composer2.5-IQ4_XS.gguf \
  -ngl 99 --ctx-size 16384 -fa on --jinja \
  --temp 1.0 --top-p 0.95 --top-k 64 --host 0.0.0.0 --port 8080

> ⚠️ Needs a recent llama.cpp — this is the gemma4 architecture (older builds won't load it).

> 🧠 Thinking is on by default via the chat template (--jinja). The model reasons through edge cases, then writes the code. For deterministic coding use --temp 0.

🦙 Ollama (one line, straight from this repo)

ollama run hf.co/sakamakismile/gemma-4-12B-coder-fable5-composer2.5-GGUF:Q4_K_M

Pick any tag: Q3_K_S Q3_K_M IQ4_XS Q4_K_M Q5_K_M.

> ❗ "manifest not found"? You must include both the hf.co/ prefix and an explicit quant tag. Without a tag, Ollama looks for :latest (which doesn't exist here); without hf.co/, it searches Ollama's own registry instead of this repo. The fix is just …-GGUF:Q4_K_M.

Also works in LM Studio / Jan / KoboldCpp — import the GGUF, pick a quant, go. 🐾

---

📊 How good is it? (greedy pass@1)

| Benchmark | Score |

|---|---|

| HumanEval | 90.2% (148/164) |

| MBPP | 85.7% (366/427) |

Strong at hard algorithms, bug-fixing & refactoring, and faithful open reasoning. Japanese prompts cause no measurable Python-quality drop.

> ⚠️ One honest caveat: on time-series / quant-finance code it can introduce a look-ahead bias (and its reasoning may state the right rule while the code does the opposite). Great algorithm/debug helper — but review its pandas/numpy back-test code before trusting it.

---

🔧 Quant details

• imatrix computed on a code-heavy calibration set (HumanEval + MBPP problems & solutions) so the importance matrix reflects real coding activations.
• Source: the author's Q8_0 GGUF (≈lossless). Text-only gemma4 (no vision/audio).
• Higher tiers keep token-embeddings & output tensors at Q6_K (K-quant default) for fidelity where it matters most; the Q3_K tiers trade a little there for size.
• K-quants over i-quants here: gemma-4's heterogeneous attention (head_dim 256 / 512 layers) survives Q3_K_ but collapses under IQ3_** — verified, so the small tiers ship as Q3_K.

📚 License & use

Gemma Terms of Use (derivatives must comply). De-refused / not safety-aligned — add your own guardrails. Best on Python/algorithmic tasks; double-check general facts and time-series code. Shared as-is. Quants & eval by Lna-Lab; thanks to @yuxinlu1. 🐾✨