---

base_model: Kwaipilot/KAT-Coder-V2.5-Dev

base_model_relation: quantized

library_name: gguf

pipeline_tag: text-generation

language:

- en

- zh

tags:

- gguf

- moe

- pruned

- reap

- expert-pruning

- mtp

- speculative-decoding

- code

license: other

---

KAT-Coder-V2.5-Dev — REAP-205E — MTP — GGUF

**Kwaipilot/KAT-Coder-V2.5-Dev

with its MoE experts pruned from 256 to 205 per layer (19.92%), MTP head

grafted, published as a bf16 master plus imatrix GGUF tiers.**

Pruned with REAP (Router-weighted Expert Activation Pruning,

arXiv 2510.13999), calibrated on **2.6M

tokens of KAT's own generated output** — not on someone else's dataset.

~34.7B → ~28.3B parameters. Same 8 active experts per token, same

architecture, same 262k vocab, same context.

---

Measured quality — read this before downloading

KL-divergence of this model against the unpruned BF16 original, both at

bf16, same text, 39 chunks at n_ctx 4096:

| metric | value |

|---|---|

| Mean PPL (original bf16) | 1.915815 ± 0.010606 |

| Mean PPL (pruned 205E) | 1.985212 ± 0.011368 |

| PPL ratio | 1.0362 (+3.62%) |

| Mean KLD | 0.058822 ± 0.001246 |

| Median KLD | 0.001481 |

| Same top-1 token | 94.599 ± 0.080 % |

| RMS Δp | 8.908 ± 0.108 % |

| Mean Δp | −0.892 ± 0.031 % |

| Correlation of ln PPL | 95.54% |

How to read it. The median KLD of 0.0015 says half of all tokens are

essentially untouched. The mean of 0.0588 is pulled up by a minority of

tokens that diverge sharply (max KLD 14.56). Same-top-1 of 94.6% means roughly

1 token in 18 picks a different most-likely candidate than the original.

**In familiar units: this is approximately the quality cost of a good 4-bit

quantization — mean KLD 0.059 sits in the usual Q4_K_M band. It is not

lossless**, and this card will not claim it is.

Two honest caveats:

1. The KLD text is drawn from the same corpus family used to compute the

saliency. Experts were selected to matter for that distribution, so **94.6%

is an optimistic bound** — expect more divergence further from

code/reasoning text.

1. No downstream coding benchmark (HumanEval/MBPP/SWE-bench) has been run.

Nothing here should be read as a retention claim on those.

---

Why 205 experts and not some rounder number

The prune rate was chosen from the measured saliency distribution, not by

analogy to other models. Measured on the real weights, per layer:

| keep | prune | saliency mass discarded | worst layer |

|---|---|---|---|

| 224 | 12.50% | 6.04% | 7.36% |

| 208 | 18.75% | 9.62% | 11.61% |

| 205 | 19.92% | 10.33% | 12.43% |

| 192 | 25.00% | 13.58% | 16.10% |

| 128 | 50.00% | 33.58% | 37.31% |

KAT's expert saliency is long-tailed but shallow: mean Gini 0.232, and

the coldest quarter of experts still carries 13.6% of routed saliency where

a uniform distribution would give 25%. The marginal cost of each removed expert

rises smoothly from 0.45 to 0.89 of an average expert with no knee anywhere

— there is no natural cut point in this model, so the rate is a judgement, and

205 sits at the top of the cheap region.

Skew is U-shaped in depth: the last blocks (38–39) and block 0 are markedly

long-tailed and cheap to cut, while blocks 16–24 are nearly flat. Layer 20 is

the worst layer to cut at every rate below 50%. A per-layer schedule would beat

a uniform rate, but n_expert is a single scalar in GGUF, so a uniform count is

what the format allows.

Routing is not language-specialised. Across 200k EN and 201k ZH tokens,

zero experts fired in exactly one language and not the other, on all 40

layers — so an English-heavy calibration set does not endanger Chinese.

---

How it was calibrated

• 2,516 rows / 2,607,981 tokens, generated by KAT-Coder-V2.5-Dev itself at

Q6_K on real prompts (code-instruct, repair, tool-use, reasoning, Chinese),

100% carrying a genuine <think> block. Real prompts, native completions —

the model's own token distribution, not a third party's.

• Saliency S_j = (1/N_j) Σ g_j(t)·‖f_j(t)‖₂, accumulated per layer over a

single forward pass, ranked independently per layer.

• Ranking by activation saliency, not by usage frequency: keep-set

agreement between the two is only 0.627, so hit-count pruning would cut a

substantially different set.

Convergence caveat: keep-set Jaccard between half the corpus and the full

corpus is 0.977 against a 0.98 stopping bar — roughly one corpus doubling

short. The binding constraint is prompt diversity (2,516 prompts), not token

count. Experts near the cut line could shift slightly with more calibration.

---

The MTP head — an explicit approximation

KAT ships *no mtp. tensors**; the multi-token-prediction head is grafted

from Qwen/Qwen3.6-35B-A3B

(19 tensors, verified byte-identical to the donor). Because transformers

drops mtp.* on load, no activation saliency exists for the head's experts.

Since n_expert is one scalar for all blocks, the head had to be cut to 205

too. It was pruned by per-expert weight magnitude — data-free, computed from

the actual tensors being cut. This is an approximation and may be revised.

Verify it yourself the cheap way: draft acceptance on the unpruned model

measures 0.90–0.94 with --spec-type draft-mtp,ngram-mod. If the approximation

were bad, acceptance would collapse.

---

Files

| file | notes |

|---|---|

| BF16/*-00001-of-00002.gguf + 00002 | the master, 57.9 GB — load from shard 1 |

| …-MTP-UD-IQ4_XS.gguf | unsloth Dynamic tiers: output / token_embd / shared |

| …-MTP-UD-Q4_K_XL.gguf | experts / attention pinned Q8_0, the bit budget spent |

| …-MTP-UD-Q5_K_S.gguf | on the routed experts only |

| …-MTP-UD-Q6_K.gguf | |

| …-MTP-APEX-I-Balanced.gguf | APEX layered maps — per-block expert types |

| …-MTP-APEX-I-Quality.gguf | |

| …-MTP-APEX-I-Compact.gguf | |

| …-MTP-APEX-I-Compact-v2D-lite.gguf | |

| …-MTP-APEX-I-Mini.gguf | |

| imatrix.dat | computed on the pruned model — see below |

| keep-index.json | the surviving expert ids per layer — reproduces the prune |

| kld-result.txt | the full KL-divergence run above |

Same nine tiers, same per-tensor recipes as

the unpruned release.

Tier ordering is readable from the recipes. UD-Q5_K_S dominates

UD-Q4_K_XL: identical Q8_0 pins on output/embd/shared-expert/attention, and

the only difference is routed gate/up experts held at Q5_K rather than a

Q4_K/Q5_K mix. UD-Q4_K_XL is the size/quality pick, not the quality pick.

The imatrix could not be inherited. ffn__exps.weight is a fused* tensor

whose imatrix entry is dimensioned by expert count, so a 256-expert imatrix —

including the standard APEX blend — cannot apply to a 205-expert model. That is

a shape mismatch, not a preference. This one was computed from scratch on the

pruned weights using the same KAT-native corpus that drove the pruning.

Every quant here is reproducible from the BF16 master; the master is

reproducible from the original weights plus keep-index.json.

---

Usage

Load the master from its first shard; llama.cpp finds the rest.

llama-server -m Kwaipilot_KAT-Coder-V2.5-Dev-REAP-205E-MTP-UD-Q4_K_XL.gguf \
  -ngl 99 -c 32768 -fa on --jinja -ub 2048 -b 2048 \
  --cache-type-k q8_0 --cache-type-v q8_0 \
  --spec-type draft-mtp,ngram-mod \
  --spec-draft-n-max 1 --spec-draft-n-min 0 --spec-draft-p-min 0.75 \
  --spec-ngram-mod-n-min 8 --spec-ngram-mod-n-max 24 --spec-ngram-mod-n-match 48 \
  --reasoning-format deepseek

-ub 2048 roughly doubles generation throughput over the 512 default. With

-np N -c C each slot gets C/N tokens — overrun it and the server drops

connections silently.

---

Provenance

• Base: Kwaipilot/KAT-Coder-V2.5-Dev
• MTP donor: Qwen/Qwen3.6-35B-A3B
• Method: REAP, arXiv 2510.13999 ·

CerebrasResearch/reap

• Chat template: improved variant — a strict superset of the official one

(23 message shapes byte-identical, 4 client-crash cases fixed, notably

tool_call.arguments sent as a JSON string rather than a mapping).

sha256 for every file is in SHA256SUMS.txt.

Not affiliated with Kwaipilot or Cerebras. Pruning necessarily changes model

behaviour; the numbers above are what was measured, and what was not measured is

stated as not measured.