---

license: apache-2.0

library_name: gguf

base_model: Kwaipilot/KAT-Coder-V2.5-Dev

base_model_relation: quantized

model_name: KAT-Coder-V2.5-Dev-Cerebellum-GGUF

model_creator: Kwaipilot

model_type: qwen3

quantized_by: deucebucket

pipeline_tag: text-generation

tags:

- GGUF

- qwen3

- qwen

- quantized

- cerebellum

- imatrix

- moe

- mixed-precision

- 3-bit

- coding

- conversational

model-index:

• name: KAT-Coder-V2.5-Dev-Cerebellum-14GB-v2

results:

- task:

name: Text Generation

type: text-generation

dataset:

name: HumanEval+ chat (EvalPlus)

type: openai_humaneval

split: test

metrics:

- name: pass@1 base

type: pass@1

value: 0.9207

source:

name: Local benchmark run (RTX 3090, llama.cpp)

url: https://huggingface.co/deucebucket/KAT-Coder-V2.5-Dev-Cerebellum-GGUF/tree/main/benchmark_results

- task:

name: Text Generation

type: text-generation

dataset:

name: HumanEval+ chat (EvalPlus)

type: openai_humaneval

split: test

metrics:

- name: pass@1 plus

type: pass@1

value: 0.8902

source:

name: Local benchmark run (RTX 3090, llama.cpp)

url: https://huggingface.co/deucebucket/KAT-Coder-V2.5-Dev-Cerebellum-GGUF/tree/main/benchmark_results

- task:

name: Text Generation

type: text-generation

dataset:

name: BigCodeBench hard

type: bigcodebench

split: test

metrics:

- name: pass@1

type: pass@1

value: 0.2805

source:

name: Local benchmark run (RTX 3090, llama.cpp)

url: https://huggingface.co/deucebucket/KAT-Coder-V2.5-Dev-Cerebellum-GGUF/tree/main/benchmark_results

---

<p align="center">

<img src="cerebellum_banner.png" alt="Cerebellum" width="640">

</p>

KAT-Coder-V2.5-Dev — Cerebellum GGUF

Sensitivity-guided mixed-precision quantization of Kwaipilot/KAT-Coder-V2.5-Dev, a fine-tune of Qwen/Qwen3.6-35B-A3B.

This is a standard GGUF that runs on stock llama.cpp.

Variants

| Variant | File | Size | BPW |

|---------|------|------|-----|

| 14 GB v2 | KAT-Coder-V2.5-Dev-Cerebellum-14GB-v2.gguf | 12.1 GB | 2.93 |

Quantization recipe

• Base model: Kwaipilot/KAT-Coder-V2.5-Dev (Qwen3.6-35B-A3B, 35B total / ~3B active MoE)
• Source format: BF16 merged GGUF
• Imatrix: KAT-specific lite coder imatrix built from HumanEval+ / MBPP+ samples
• Base quant: Q3_K_M
• Key override: expert ffn_down weights in layers 20-39 promoted from Q2_K to Q3_K

Benchmarks

Measured on an RTX 3090 with llama-server -ngl 99 --parallel 4 -c 24576 --reasoning off --reasoning-budget 0.

| Benchmark | v2 (12.1 GB) | maxx-v4 (15.6 GB) | Q5_K_M pure (24.0 GB) | Previous 35B A3B Cerebellum 14 GB |

|-----------|--------------|-------------------|-----------------------|-----------------------------------|

| HumanEval+ chat base | 92.07% | 90.85% | 90.85% | 89.63% |

| HumanEval+ chat plus | 89.02% | 87.80% | 87.80% | 85.98% |

| BigCodeBench hard | 28.05% | 27.0% | 27.7% | 25.70% |

v2 is the best coding quant we have produced for this architecture. Both larger experiments (16 GB mixed-precision maxx-v4 and 24 GB pure Q5_K_M base) scored lower, confirming the coding-specific imatrix + targeted late-layer ffn_down promotion in v2 extracts more performance per gigabyte than simply raising the base quant.

Coding ablation

A per-group ablation (demote one group to Q2_K over a Q4_K_M base, measure HumanEval delta) found every tensor group is coding-critical. The largest drops came from attn_q (−76.2%), ffn_up_all (−73.2%), and ssm_beta (−71.9%). Even the "least damaging" group, attn_qkv, dropped coding performance by 56.7%.

This means v2 is close to the practical floor for this model at ~14 GB. Further gains require a higher starting base quant or layer-level drilling, not blanket demotion.

Full logs, samples, eval outputs, ablation data, and one-shot game generations are in benchmark_results/.

Experimental result files for maxx-v4 and the rejected Q5_K_M pure base are also in benchmark_results/ for comparison.

Runtime stats

| Metric | Value |

|--------|-------|

| Single-request TPS | ~78–83 t/s |

| Batched TG | ~109–160 t/s |

| Code-corpus perplexity | 1.6380 |

| Agent probe (tool JSON / repair / patch / completion) | 17/17 |

Notes

• Text-only. The base model tag includes image-text-to-text, but the published source weights used here are the text-only KAT-Coder fine-tune. Vision was not tested.
• Native context is 262,144 tokens. On a 24 GB RTX 3090, 24K–98K context is the practical daily-driver range.

How to run

llama-server \
  --model KAT-Coder-V2.5-Dev-Cerebellum-14GB-v2.gguf \
  -ngl 99 --parallel 4 -c 24576 \
  --reasoning off --reasoning-budget 0

License

Apache-2.0, matching the base model.