---

base_model: tpls/gemma-4-12B-coder-fable5-composer2.5-v1-abliterated-sft-v5

base_model_relation: quantized

library_name: gguf

pipeline_tag: text-generation

license: gemma

quantized_by: triplezrobotics

tags:

- gguf

- llama.cpp

- gemma4

- abliterated

- code

---

gemma-4-12B-coder-fable5-composer2.5-v1-abliterated-sft-v5-GGUF

GGUF quantizations of tpls/gemma-4-12B-coder-fable5-composer2.5-v1-abliterated-sft-v5.

Quantized with llama.cpp using an importance matrix (imatrix) calibrated on

code + tool-call markup, with token-embedding and output tensors kept at f16

to protect the tool-call token logits, plus a gemma-4 EOG patch (eot=106) to

bound the <|turn> runaway.

Files

| Quant | File | Notes |

|-------|------|-------|

| Q4_K_M | gemma-4-12B-coder-fable5-composer2.5-v1-abliterated-sft-Q4_K_M.gguf | good default — fits 12 GB VRAM, best size/quality balance |

Use it

# llama.cpp (server, with native tool parsing)
llama-server -hf tpls/gemma-4-12B-coder-fable5-composer2.5-v1-abliterated-sft-v5-GGUF:Q4_K_M --jinja --ctx-size 16384

# Ollama
ollama run hf.co/tpls/gemma-4-12B-coder-fable5-composer2.5-v1-abliterated-sft-v5-GGUF:Q4_K_M

⚠️ Tool-calling note (abliterated weights)

The base model and fine-tune emit canonical tool_calls that llama.cpp --jinja

parses natively. Abliteration degrades the canonical tool token: this model

tends to leak its calls as text markup (<|tool>NAME{...}) instead of the

canonical <|tool_call> token, so the native parser may not fire.

Two ways to get structured tool_calls back:

1. Runtime shim (works now). Put a tolerant recovery shim in front via LiteLLM:

```yaml

# litellm config

model_list:

- model_name: gemma-coder

litellm_params:

model: openai/gemma-coder

api_base: http://127.0.0.1:8096/v1

api_key: sk-none

litellm_settings:

callbacks: ["gemma_tool_shim.shim"] # recovers tool_calls from leaked markup

drop_params: true

```

The shim is published with the source at the repo that produced this model.

1. Native (no shim). Use a control-vector / activation-steered variant

instead of weight-ablation — steering uncensors at inference without editing

the weights, so the canonical tool token survives and bare --jinja works.