---

license: other

license_name: modified-mit

base_model: vcruz305/Kimi-K3-GGUF

library_name: gguf

pipeline_tag: text-generation

tags:

- gguf

- kimi-k3

- moe

- expert-pruning

- reap

- iq1_s

- llama.cpp

---

Kimi-K3 Neuron REAP-720 — IQ1_S GGUF

**Kimi-K3 Neuron (k=1536 IQ1_S) with 176 of 896 routed experts removed per layer

via REAP-style saliency pruning — 277.47 GB, 9 shards.**

Derived from vcruz305/Kimi-K3-GGUF

by a byte splice: every kept expert slab, and every non-expert tensor, is

byte-identical to the source. Zero requantisation anywhere.

| | source (root) | this build |

|---|---:|---:|

| Routed experts / layer | 896 | 720 |

| Parameters | 1.418 T | ~1.18 T |

| Size | 330.17 GB | 277.47 GB |

| Decode bytes/token | 66.67 GB | ~66.6 GB (unchanged) |

| Tensor types | IQ1_S / Q8_0 / F16 / F32 | identical |

Pruning buys memory, not speed. Per-token traffic depends on top_k (still 16)

and the dense path, not on how many experts are stored. This build exists to fit:

~16 GiB/rank less residency at TP3, and it is the first K3 build that (mostly)

fits a 288 GB-RAM workstation.

Method

Expert saliency in the spirit of REAP (Cerebras, arXiv:2510.13999),

computed without any model forward passes: llama.cpp imatrix files store

per-expert × per-column activation statistics, and five 3.42 GB domain imatrices

(chat / code / prose / reasoning / wiki, 23,552 calibration tokens each) provided

per-expert energy and routing counts directly.

Selection = min-rank consensus: each expert is scored by its rank in its single

most favorable domain; the 176 pruned per layer are those that **no domain ranks

well. Measured safety margin: no pruned expert ranks better than 226 of 896 in

any domain, in any layer** (median layer: 314). Dead experts (zero activations in

all five domains: 11 instances) pruned unconditionally; the model's two

super-experts (blk.12 e160 — 88% of its layer's energy — and blk.28 e132) are

maximally retained. Full method, rankings and receipts ship in this repo

(REAP-KEEPLIST-RECEIPT.md, keeplist_720.json, ranking.npz, reap_splice.py).

Verified

• Splice audit: every kept expert slab and every unchanged tensor re-read from

the output and compared byte-for-byte against source — identical, all 9 shards.

• Loads and generates on llama.cpp (b10325): first 12 greedy tokens on the

smoke prompt are identical to the unpruned source model.

NOT verified — read before using

• No task benchmark has been run on this build. No PPL, GSM8K, HumanEval,

MMLU, anything. Expert pruning CAN change and degrade outputs; the 12-token

smoke match is not evidence of general equivalence. Treat quality as unknown

until measured.

• Calibration corpus is English + code only (chat/code/prose/reasoning/wiki).

No CJK or other-language text was represented, so experts serving those

capabilities were pruned blind. Expect disproportionate multilingual

degradation relative to English/code.

• Saliency was measured on the native-width (k=3072) model's imatrix and

restricted to the shipped k=1536 dims via the width-prune selection plan;

restriction barely reranks (Spearman ≥ 0.988/layer) but the provenance is

native-model routing.

• SparkInfer users: current K3 dist builds hard-require n_experts == 896

(dist_rank / rank_protocol / rank_local_loader). This model needs those guards

patched to read the KV value (kimi-k3.expert_count = 720) before it will load.

llama.cpp needs no changes.

Run (llama.cpp)

hf download vcruz305/Kimi-K3-Neuron-REAP720-IQ1S-GGUF --local-dir ./K3-REAP720
llama-server -m ./K3-REAP720/Kimi-K3-UD-IQ1_S-00001-of-00009.gguf \
  -ngl 99 --jinja -c 8192 -fa on --temp 1.0 --top-p 0.95

All 9 shards required. Requires a Kimi-K3-capable llama.cpp build.