---

license: apache-2.0

base_model:

- jakeroxs/KAT-Coder-V2.5-Dev-35B-A3B-MTP-ABLITERATED

language:

- en

- zh

pipeline_tag: text-generation

tags:

- gguf

- llama.cpp

- qwen3_5_moe

- kat-coder

- kat-coder-v2.5

- qwen3.5

- mixture-of-experts

- coding

- mtp

- speculative-decoding

- abliteration

- uncensored

- conversational

---

KAT-Coder V2.5 Dev 35B-A3B MTP Abliterated GGUF

GGUF builds of KAT-Coder V2.5 Dev 35B-A3B MTP Abliterated, including a transplanted native Multi-Token Prediction layer for llama.cpp draft-mtp speculative decoding.

The merged Hugging Face-format source checkpoint is available here:

jakeroxs/KAT-Coder-V2.5-Dev-35B-A3B-MTP-ABLITERATED

Available GGUFs

| File | Quantization | Size |

| ---------------------------- | ------------ | ------------------: |

| KAT-Philly-MTP-Q4_K_M.gguf | Q4_K_M | ~20.2 GiB |

| KAT-Philly-MTP-BF16.gguf | BF16 | Full precision GGUF |

Both contain the transplanted MTP layer.

Model lineage

Original model

KwaiPilot/KAT-Coder-V2.5-Dev

Abliterated base

KridgeDookie/KAT-Coder-V2.5-Dev-35B-A3B-ABLITERATED-UNCENSORED-PHILADELPHIA-CLASS

MTP head

Myric/KAT-Coder-V2.5-Dev-MTP-head

Merged checkpoint

jakeroxs/KAT-Coder-V2.5-Dev-35B-A3B-MTP-ABLITERATED

The Philadelphia Class checkpoint is used as the target model, with the compatible MTP tensors from Myric's KAT-Coder V2.5 Dev MTP-head checkpoint transplanted into it.

MTP modification

The original Philadelphia Class checkpoint has:

mtp_num_hidden_layers = 0

The merged checkpoint changes this to:

mtp_num_hidden_layers = 1

and adds the 19 MTP tensors from the MTP-head checkpoint.

During GGUF conversion, llama.cpp exposes the transplanted prediction layer as the additional nextn / final block tensors expected by native draft-mtp speculative decoding.

The converted model reports:

architecture:          qwen35moe
model type:            35B.A3B
parameters:            35.51 B
training context:      262144
experts:               256
experts used/token:    8
nextn predict layers:  1

Recommended llama.cpp configuration

A recent llama.cpp build with Qwen3.5 MoE MTP support is required.

For a GPU with sufficient VRAM, the Q4_K_M model works well with full 262K context, Q8 KV cache, and native MTP:

llama-server \
  -m KAT-Philly-MTP-Q4_K_M.gguf \
  --host 0.0.0.0 \
  --port 8080 \
  --parallel 1 \
  --reasoning-preserve \
  --fit-ctx 262144 \
  --fit on \
  --flash-attn on \
  --cache-type-k q8_0 \
  --cache-type-v q8_0 \
  --spec-type draft-mtp \
  --spec-draft-n-max 2

The MTP draft KV can remain F16. With a single MTP prediction layer its memory requirement is relatively small.

RTX 5090 validation

The Q4_K_M model was tested on:

GPU: NVIDIA GeForce RTX 5090 32 GB
CPU: AMD Ryzen 7 9850X3D
Context: 262144
Main KV: Q8_0
MTP draft KV: F16
MTP n_max: 2

At full 262,144-token configured context, llama.cpp loaded all model layers onto the GPU:

offloaded 42/42 layers to GPU

Representative GPU allocation:

| Component | Approx. VRAM |

| --------------- | -----------: |

| Model | 20.4 GiB |

| Main Q8 KV | 2.66 GiB |

| Recurrent state | 0.18 GiB |

| Main compute | 0.80 GiB |

| MTP F16 KV | 0.50 GiB |

| MTP compute | 0.29 GiB |

llama.cpp projected approximately 24.96 GiB total device usage, leaving approximately 6.49 GiB free, with no automatic fit reduction required.

MTP benchmark

A deterministic 2,048-token coding generation was run using the same Q4_K_M model, Q8 KV cache, 262K configured context, and otherwise identical settings.

| Configuration | Generation speed |

| ------------- | ---------------: |

| MTP disabled | 265.45 tok/s |

| MTP n_max=2 | 333.72 tok/s |

This represents approximately a 25.7% increase in generation throughput with MTP enabled.

For the MTP run:

draft acceptance = 0.67239
accepted drafts  = 1174 / 1746
mean length      = 2.34
acceptance/pos   = (0.802, 0.543)

This corresponds to approximately:

• 80.2% acceptance at the first speculative position
• 54.3% acceptance at the second speculative position
• 67.2% overall draft-token acceptance

The transplanted MTP layer therefore provides a measurable throughput improvement when the target model and MTP working set can remain fully GPU-resident.

Lower-VRAM testing

Earlier testing on an RTX 3080 Ti 12 GB confirmed that the MTP transplant functioned correctly, but enabling MTP increased the working set enough to cause additional model offloading.

Representative results on that system were:

| Configuration | Sustained generation |

| --------------------- | -------------------: |

| Non-MTP Q4_K_M | ~38.4 tok/s |

| MTP Q4_K_M, n_max=2 | ~29–30 tok/s |

| MTP Q4_K_M, n_max=1 | ~26.8 tok/s |

This was a memory-residency limitation rather than an indication that the MTP head was ineffective.

Systems with sufficient VRAM to keep the model fully GPU-resident can show substantially better results, as demonstrated by the RTX 5090 benchmark above.

Notes

• The model's native context length is 262,144 tokens.
• The Q4_K_M GGUF is approximately 20.21 GiB.
• MTP is optional; the model can be run normally without --spec-type draft-mtp.
• --spec-draft-n-max 2 has been validated and provides a substantial speedup on the tested RTX 5090 configuration.
• Performance and MTP acceptance depend on workload, prompt, sampling settings, llama.cpp build, and hardware.

About the abliteration

This model inherits the modifications made by the Philadelphia Class checkpoint.

Abliteration attempts to reduce refusal behavior by modifying model representations associated with refusal behavior. It does not retrain the underlying model and does not guarantee any particular behavior.

Credits

• KwaiPilot — KAT-Coder V2.5 Dev
• KridgeDookie — Abliterated / Uncensored Philadelphia Class variant
• Myric — KAT-Coder V2.5 Dev MTP head
• llama.cpp contributors — GGUF conversion, Qwen3.5 MoE support, quantization, and MTP inference