---

base_model: tpls/gemma-4-12B-coder-fable5-composer2.5-v1-sft-v5-abliterated

base_model_relation: quantized

library_name: gguf

pipeline_tag: text-generation

language:

- en

license: gemma

quantized_by: triplezrobotics

tags:

- gemma4

- code

- text-generation

- function-calling

- tool-use

- agentic

- gguf

- llama.cpp

- imatrix

- abliterated

- uncensored

model-index:

- name: Gemma-4 12B Coder — SFT v5 + abliterated (GGUF)

results:

- task:

type: text-generation

name: Function calling (tool use)

dataset:

name: gemma4-coder-tool-eval

type: tpls/gemma4-coder-tool-eval

metrics:

- type: pass_rate

value: 1.0

name: Tool-call pass rate (shim, prod path)

- type: pass_rate

value: 0.125

name: Tool-call pass rate (raw llama.cpp --jinja)

---

Gemma-4 12B Coder — SFT v5 + abliterated (GGUF)

Uncensored gemma-4 12B coder for local, agentic tool use — GGUF quantizations for llama.cpp / Ollama.

Run it: llama-server -hf tpls/gemma-4-12B-coder-fable5-composer2.5-v1-sft-v5-abliterated-GGUF:Q4_K_M --jinja (full commands below).

> ⚠️ Tool-calling needs the recovery shim. The model emits gemma-4's native tool markup, which llama.cpp --jinja under-parses — wrap your endpoint with the tool-shim (see Tool-calling below) to get standard tool_calls.

> 💡 Pick this to run locally with the best of both: SFT v5's tool-calling and an uncensored model. Our KL-guarded abliteration on top of SFT v5 — gate SHIM 8/8.

At a glance

| | |

|--|--|

| Type | GGUF quantizations · llama.cpp / Ollama |

| Techniques | sft-qlora → abliteration → imatrix-quant → tool-shim |

| Tool-calling | ✅ 100% gate pass (recovery-shim path) |

| Status | ✅ Active / supported |

| Use | llama-server -hf tpls/gemma-4-12B-coder-fable5-composer2.5-v1-sft-v5-abliterated-GGUF:Q4_K_M --jinja |

Use it

# llama.cpp (server) — tool-calling needs the recovery shim, see below
llama-server -hf tpls/gemma-4-12B-coder-fable5-composer2.5-v1-sft-v5-abliterated-GGUF:Q4_K_M --jinja --ctx-size 16384

# Ollama
ollama run hf.co/tpls/gemma-4-12B-coder-fable5-composer2.5-v1-sft-v5-abliterated-GGUF:Q4_K_M

Files

Sizes and a one-click loader are in the file browser / Quantizations widget above;

the note says which quant to reach for.

| Quant | Notes |

|-------|-------|

| Q4_K_M | good default — fits 12 GB VRAM, best size/quality balance |

| Q5_K_M | higher quality, ~9.5 GB |

| Q6_K | near-bf16 quality, ~10.5 GB |

| Q8_0 | highest GGUF quality, large |

| IQ4_XS | smallest usable — for <8 GB VRAM, slight quality cost |

| Q3_K_M | low-VRAM fallback, noticeable quality drop |

Tool-calling

Tool-calling works — but llama.cpp --jinja doesn't recognise gemma-4's native

tool-call markup, so the bare parser under-reports calls. The model is fine; the

parser is blind to the format. Recover standard tool_calls with a small serve-side

post-processor (no weight change, no latency beyond a regex scan).

Ready-to-use → tpls/gemma4-tool-shim — a drop-in

callback for OpenAI-compatible proxies, a standalone (dependency-free) example, and the pure

parser, all Apache-2.0, with the full recovery algorithm documented. Point your

OpenAI-compatible endpoint through it.

You send tools the usual OpenAI way (tools=[…]); the model emits native markup; the

shim turns it into a standard tool_calls object:

# model completion (raw):
<|tool_call>get_weather{"city": "Paris", "units": "celsius"}

// after the shim:
{"finish_reason": "tool_calls",
 "message": {"role": "assistant", "content": null,
   "tool_calls": [{"id": "call_0", "type": "function",
     "function": {"name": "get_weather", "arguments": "{\"city\": \"Paris\", \"units\": \"celsius\"}"}}]}}

Tool-calling gate

Measured 2026-06-26 on the tool-eval suite (7 tool cases + 1 abstain), the Q4_K_M quant served on llama.cpp --jinja with TOOLS_IN_PROMPT=1 at temp 0. raw = native parser; shim = the prod gemma_tool_parse recovery path. SHIM 8/8 = identical to clean SFT v5: abliteration preserved tool-calling (KL=0.0009). The low raw is gemma-4's known native-parser breakage (the shim is the prod path), not abliteration damage.

The rows are this model under two parse paths (raw and shim); the shim path is how it's served in production.

| Measured on | Pass rate |

|-------------|-----------|

| this model — raw (--jinja) | 0.125 |

| this model — shim (prod path) | 1.000 |

Intended use & limitations

Built for code generation and agentic tool use; serve locally via llama.cpp /

Ollama, or use as a base to fine-tune / merge / quantize. Outputs can be wrong or

fabricated — validate tool arguments before executing, and keep a human in the loop

for anything consequential.

> ⚠️ Uncensored. For this variant the refusal direction has been ablated from the weights — safety guardrails are

> substantially removed and it will attempt requests a stock model would refuse. You

> are responsible for what you generate and how it's used; not suitable where refusal

> behaviour is itself a safety requirement.

Where this sits in the family

• base (upstream) — yuxinlu1/gemma-4-12B-coder-fable5-composer2.5-v1

- SFT v5 (weights)

- SFT v5 + abliterated (weights)

- SFT v5 + abliterated (GGUF) ← you are here

- SFT v5 (GGUF)

---

Provenance & reproduction

How this model was built — technique chain, training mix, and the exact knobs/pins,

so the result is reproducible without any of our tooling.

Mechanics applied

| Step | Technique | What it does | Provenance |

|------|-----------|--------------|------------|

| 1 | sft-qlora | QLoRA supervised fine-tune to keep + improve native tool-calling | — |

| 2 | abliteration | refusal-direction ablation edits the weights to remove refusals | tpls/gemma-4-12B-coder-fable5-composer2.5-v1-sft-v5 |

| 3 | imatrix-quant | llama.cpp quantization with an importance matrix (imatrix) | — |

| 4 | tool-shim | serve-side recovery of structured tool_calls from the model's native markup | — |

1. sft-qlora

• tools_mode: mixed

2. abliteration

> weight ablation degrades the canonical <|tool_call> token — the model tends to leak calls as text markup, so the native llama.cpp parser may not fire. See the tool-call recovery note below to get structured calls back.

3. imatrix-quant

• calibration: code + tool-call markup
• embed/output: kept at f16 (protects tool-call logits)
• eog_patch: tokens 105/106 → EOG (bounds the <|turn> runaway)

4. tool-shim

• where: a thin pre/post wrapper on the OpenAI-compatible endpoint
• format: re-parse <|tool_call>NAME{json-args} (and leaked <|tool>…) into tool_calls

> serve-side only — does not modify the weights; recommended for abliterated variants.

Training data, hyperparameters & environment

The supervised fine-tune (sft-qlora step above) is inherited from

Gemma-4 12B Coder — SFT v5 (weights) — see that card

for the full training mix, exact hyperparameters, and pinned environment. The remaining

step(s) above are what this model adds on top; their measured effect is below.

Quantization environment

The GGUF bytes depend on the quantizer build, not just the weights — a different

llama.cpp release rounds tensors differently and can change the convert mapping. Pins

the toolchain these quants were produced with:

| Step | Tool / setting |

|------|----------------|

| quantizer | llama.cpp tools image ghcr.io/ggml-org/llama.cpp:full |

| convert | convert_hf_to_gguf.py → f16 GGUF |

| imatrix | llama-imatrix over the calibration set (CPU forward pass) |

| quantize | llama-quantize --imatrix, token-embeddings + output tensor kept at f16 |

> The image is the rolling :full tag, not a digest — for byte-exact reproduction pin

> the image digest you build with. The imatrix-quant step above lists the calibration

> set and the EOG patch this build applied.

Other measured metrics

| Metric | Value |

|--------|-------|

| kl_divergence | 0.001 |

| n_trials | 100.000 |

| refusals | 3.000 |

---

Part of the Gemma-4 12B Coder — active collection.

Something not right, or a request? Open a discussion — happy to help.