---

license: apache-2.0

base_model:

• Kwaipilot/KAT-Coder-V2.5-Dev

tags:

• gguf
• mtp
• speculative-decoding
• deprecated

---

⚠️ Deprecated — the models here were WORSE than doing nothing

Go to → gbuzhf/KAT-Coder-V2.5-Dev-MTP-GGUF

The GGUFs have been deleted. They carried a **fine-tuned MTP draft head that

measured worse than the untrained one it started from.** Keeping them up would

just cost people speed.

The research artifacts and the discussions are kept — the negative result is the

useful part.

---

What this was

KAT-Coder-V2.5-Dev is an SFT+RL fine-tune of Qwen3.6-35B-A3B and ships

mtp_num_hidden_layers: 0 — no draft head. Qwen3.6 has one, so the obvious

move is to graft it on.

The hypothesis: that head was trained against Qwen's hidden states, and KAT's

trunk has moved. Fine-tune the head on KAT's own output and it should fit better.

Sound reasoning. It was wrong.

---

The measurement

79 live configs. Same quant tier, same flags, only the MTP head differs.

--spec-type draft-mtp, DraftMax 2, temp 1.0 / top_k 20 / top_p 0.95 /

presence_penalty 1.5. Draft acceptance:

| MTP head | COPY | NOVEL | AGENTIC |

|---|---|---|---|

| Qwen donor, untrained | 76% | 48% | 73% |

| this repo — fine-tuned, 450 steps | 50% | 24% | 46% |

| earlier — fine-tuned, 80 steps | 47% | 37% | 45% |

| (reference) Qwen head on Qwen's own trunk | 89% | 53% | 76% |

Two things fall out:

1. Fine-tuning didn't fail to help — it did damage. Acceptance roughly halved.
2. There was almost nothing to gain. The untrained donor on KAT's trunk is

within 3 points of Qwen's own co-trained head on its own trunk. The

"trunk has moved" premise was real but tiny.

The 80-step checkpoint is as damaged as the 450-step one, so the damage happens

almost immediately — the signature of catastrophic forgetting, not undertraining.

---

Why it failed

| cause | detail |

|---|---|

| Corpus far too narrow | 446k tokens, 36 synthetic tasks, ONE fixture repo, ONE system prompt |

| Over-parameterised | ~830M-param head, 3 epochs on that corpus — memorisation, not learning |

| Depth 2 never trained | one head call, one cross-entropy. The recursive draft path existed only as a metric |

| Wrong labels | trained on the token KAT sampled at temp 1.0 under presence_penalty 1.5, not the trunk's mode |

| Eval was a mirage | val set was 5 families from the same 36-task catalogue. It reported 96.8%; reality was 56.5% |

The eval is the one that let the rest survive. It saturated at step 100 of 558

and never discriminated again — a metric that stops separating checkpoints has

stopped measuring learning.

---

What was worth keeping

The serving config found by sweeping those 79 runs, which turned out to matter

far more than the head:

--spec-type draft-mtp,ngram-mod
--spec-draft-n-max 1 --spec-draft-n-min 0 --spec-draft-p-min 0.75
--spec-ngram-mod-n-min 8 --spec-ngram-mod-n-max 24 --spec-ngram-mod-n-match 48

• p-min 0.75 was the single highest-leverage setting — drafting only when

confident turns a mediocre head into a useful one.

• On CPU-offloaded MoE, MTP alone is a net loss. It only pays combined with

ngram-mod. With both, every head reaches 96-97% on copy work — ngram covers

the repeats and the head covers the rest, which is why head quality shows up

on novel and agentic content and nowhere else.

---

Files still here

imatrix_blend.gguf, imatrix_katonly.gguf, mtp_head_trained.{pt,safetensors},

corpus_kat.txt, kat_rollouts.jsonl — the inputs and outputs of the failed

experiment, kept so the result is reproducible and checkable.

The blended imatrix is good and is used by the replacement repo. **The trained

head is the thing that did not work** — it is here as evidence, not for use.

---

Where to go

gbuzhf/KAT-Coder-V2.5-Dev-MTP-GGUF

— same tiers, the original Qwen MTP head, blended imatrix, plus the bf16

master so any tier can be rebuilt without a conversion.

License: apache-2.0, inherited from the base model.