---

license: apache-2.0

base_model: Kwaipilot/KAT-Coder-V2.5-Dev

base_model_relation: quantized

pipeline_tag: text-generation

tags:

• gguf
• llama.cpp
• moe
• apex
• mtp
• speculative-decoding
• code
• agentic-coding

language:

• en
• zh

---

KAT-Coder-V2.5-Dev — APEX-MTP GGUF

APEX MoE-aware mixed-precision quantizations of

Kwaipilot/KAT-Coder-V2.5-Dev, with the

Qwen3.6-35B-A3B multi-token-prediction (MTP) head grafted in for single-file speculative

decoding. Stock llama.cpp runs these files; the MTP path additionally works with builds

supporting --spec-type draft-mtp.

Files

| File | Size | Recipe | Fits |

|---|---|---|---|

| Kwaipilot_KAT-Coder-V2.5-Dev-MTP-UD-Q5_K_XL.gguf | 27.4 GB | Highest fidelity here — see measurements | 32 GB |

| Kwaipilot_KAT-Coder-V2.5-Dev-APEX-MTP-I-Balanced.gguf | 26.2 GB | mudler's KAT-native APEX Balanced config, Q8_0 MTP | 32 GB |

| Kwaipilot_KAT-Coder-V2.5-Dev-APEX-MTP-I-Compact.gguf | 17.4 GB | mudler's KAT-native APEX Compact config, Q8_0 MTP | 24 GB (16 GB w/ CPU-MoE) |

| Kwaipilot_KAT-Coder-V2.5-Dev-APEX-MTP-I-Mini.gguf | 14.4 GB | mudler's KAT-native APEX Mini config, Q8_0 MTP — mudler's own I-Mini has no MTP head, this one does | 16 GB (12 GB w/ CPU-MoE) |

| Kwaipilot_KAT-Coder-V2.5-Dev-imatrix.gguf | 184 MB | importance matrix used for the I-variants | — |

These are text-only GGUFs. (KAT-Coder-V2.5-Dev's config declares a vision tower, but image

input does not work through llama.cpp in practice, so no mmproj is shipped.)

Verify your download

sha256sum -c sha256sums.txt --ignore-missing

| File | sha256 |

|---|---|

| …-MTP-UD-Q5_K_XL.gguf | d178621ee21573b5b55cf972951d8f06e5e085d970a86768aa74b3a9cb838a79 |

| …-APEX-MTP-I-Balanced.gguf | cb82e964c72108c03a1436681ab1435e5794fc80e5c6243e0afaff61c4cdd607 |

| …-APEX-MTP-I-Compact.gguf | 5c0056cbc1e4603f4abaf620678211352e37d8389e9c91f30fe8465a1957cf17 |

| …-APEX-MTP-I-Mini.gguf | 8956838bc6f481294b68ed08cff33867336552e3f36317bd7b8188f92b20ed85 |

| …-imatrix.gguf | f5727113280b24cbacc2f5308c970ab5efc8adb15d3ac4feb0e27ae1ac469c7c |

(Build provenance and toolchain versions in MANIFEST.txt.)

What's special here

1. APEX quantization (mudler & Palethorpe 2026):

instead of uniform bit-width, tensors are classified by role — routed experts get a layer-wise

precision gradient (sensitive edge layers high, redundant middle layers low), always-active

shared experts stay at high precision (heavy-tailed distributions), attention stays at Q6_K/Q4_K/Q3_K

depending on tier. On the same architecture APEX Balanced matches Q8_0 perplexity at ~⅔ the size;

see the APEX technical report for full benchmarks.

2. Grafted MTP head. KAT-Coder-V2.5-Dev ships without an MTP head

(mtp_num_hidden_layers: 0), but it is an architecture-identical fine-tune of

Qwen3.6-35B-A3B, whose 19 mtp.* tensors were grafted into the checkpoint before conversion.

llama.cpp (≥ PR #22673) bundles them as blk.40.* with nextn_predict_layers = 1; the whole

draft layer is pinned at Q8_0 in every file here (an under-quantized draft head kills

speculative-decode acceptance).

> ⚠️ Caveat: the draft head was trained against the base trunk, not KAT's fine-tuned

> trunk, so speculative acceptance may be lower than on stock Qwen3.6. If MTP doesn't pay off

> for your workload, run the same file with speculation disabled — nothing else changes.

<a id="measured-quality"></a>

Measured quality

KL divergence against this model's own bf16 weights — not a proxy, not an estimate.

Measured on wikitext-2-raw, context 2048, 40,960 tokens. KAT's bf16 perplexity on this eval

is 5.8024.

| File | Size | Same top token as bf16 | Max KLD | 99.9% KLD | 99.9% Δp |

|---|---:|---:|---:|---:|---:|

| MTP-UD-Q5_K_XL | 27.4 GB | 96.237 ± 0.133 % | 1.423 | 0.352 | 21.59 % |

I-Balanced, I-Compact, and I-Mini were rebuilt on mudler's own KAT-native APEX configs

(kat_coder_v25_{balanced,compact,mini}.txt) rather than the Qwen3.6 stand-in used previously

(mudler had not yet published KAT-specific configs at the time), plus the MTP graft. They have

not been KLD-measured against bf16 yet — treat the file sizes and recipe as current, but

not the quality figures above, which apply only to MTP-UD-Q5_K_XL.

MTP-UD-Q5_K_XL

Built on Unsloth's UD-Q5_K_XL

tensor allocation — all attention and SSM output at Q8_0, shared experts Q8_0, output head

and embeddings Q8_0, routed experts Q5_K with the down-projection at Q6_K. **That role-level

design is Unsloth's work**, read from their published Qwen3.6-35B-A3B GGUF and replayed here

(753 tensors, transfers 1:1).

Nine tensors deviate:

• Six trunk tensors — per-layer expert protection re-derived from KAT's own importance

matrix. Unsloth protect blk.34 and blk.39, which reflect the base model's activation

outliers. KAT's are elsewhere: its blk.0 gate concentrates 46% of activation energy in

1% of channels, the sharpest outlier in the model. Protection was moved to blk.0, 38,

39 — a net-zero change in bytes.

• Three MTP tensors — blk.40 experts pinned Q8_0 (+266 MB). Unsloth leave theirs lower,

but their draft head is native to their trunk; this one is grafted from the base onto a

fine-tuned trunk, and draft acceptance is what the MTP head is for.

Uses KAT's own imatrix (computed on this model), not the base model's.

Usage

Plain llama.cpp (MTP tensors are simply carried, no speculation):

llama-server -m Kwaipilot_KAT-Coder-V2.5-Dev-APEX-MTP-I-Compact.gguf \
  -ngl 99 --n-cpu-moe 30 -c 32768 -fa on

With MTP speculative decoding (builds supporting --spec-type draft-mtp, e.g. TurboQuant+):

llama-server -m Kwaipilot_KAT-Coder-V2.5-Dev-APEX-MTP-I-Compact.gguf \
  --spec-type draft-mtp --spec-draft-n-max 1 \
  -ngl 99 --n-cpu-moe 32 -c 32768 -fa on

Tip from measurement: MTP alone can be a net loss on novel prose; combined with ngram-style

speculation it is strongly additive on copy-heavy agent workloads (diffs, file echoes, JSON).

Keep the draft chain depth at 1 when combining engines.

Build provenance

• Converted from the original bf16 safetensors (convert_hf_to_gguf.py --outtype bf16).

MTP-UD-Q5_K_XL built with llama.cpp c0bc859 / apex-quant a445a12. I-Balanced /

I-Compact / I-Mini built with llama.cpp 64d528b / apex-quant 4e95e47.

• I-Balanced / I-Compact / I-Mini quantized with llama-quantize --tensor-type-file

using mudler's own KAT-native apex-quant configs — kat_coder_v25_balanced.txt (base Q6_K),

kat_coder_v25_compact.txt (base Q4_K_M), kat_coder_v25_mini.txt (base Q3_K_M) — with the

blk.40 MTP block appended (mudler's published configs don't include an MTP head).

• Importance matrix from bartowski/Kwaipilot_KAT-Coder-V2.5-Dev-GGUF
• MTP tensors byte-copied from Qwen/Qwen3.6-35B-A3B (Apache-2.0)
• MTP bundling verified post-conversion (blk.40.nextn.eh_proj present, nextn_predict_layers=1)

Credits

• Kwaipilot for KAT-Coder-V2.5-Dev (Apache-2.0)
• Qwen team for the Qwen3.6-35B-A3B base and its MTP head (Apache-2.0)
• Ettore Di Giacinto (mudler) & Richard Palethorpe / LocalAI team for the APEX method (MIT)
• bartowski for the importance matrix
• Georgi Gerganov & contributors for llama.cpp