---

base_model: moonshotai/Kimi-K3

license: other

license_name: kimi-k3

license_link: https://huggingface.co/moonshotai/Kimi-K3/blob/main/LICENSE

language:

• ja

tags:

• gguf
• kimi-k3
• moe
• reap

---

Kimi-K3 REAP50 Width50 UD GGUF

日本語

!Kimi-K3 REAP50 and Width50 overview

This is an experimental, reduced GGUF build of Kimi-K3.

Calibration and selection

Prepared Japanese responses for chat, code generation, reasoning, and tool use were tokenized and evaluated with a forward pass of the original Kimi-K3. Only semantic assistant tokens were scored; prompts and control or structure tokens were excluded.

REAP50: expert axis

For expert e, the score is:

REAP(e) = mean[t routed to e](top-16-renormalized router weight × ||unweighted expert output||₂)

Experts are ranked separately in each of the 92 MoE layers. The top 448 of 896 are retained. No experts are reserved by hand.

Width50: intermediate axis

For every calibration token t routed to expert e, Width50 follows the actual expert MLP:

a(t,e)     = SiTU(Wgate,e × h(t), Wup,e × h(t))
y(t,e,b)   = router_weight(t,e) × Wdown,e[:,b] × a(t,e)[b]
score(e,b) = Σ[t routed to e] ||y(t,e,b)||₂²

Here, b is one 32-channel slice of the 3072-channel intermediate activation. Each slice is passed through only the matching columns of Wdown, producing a hidden-size output vector. Its router-weighted squared L2 norm is accumulated over semantic assistant tokens.

Eight adjacent 32-channel scores are summed into one physical QK256 score. The highest 6 of 12 QK256 blocks are retained independently for each expert. Low-coverage experts blend this activation score with a weight-based prior. The width map is measured on the original expert IDs and remapped through the REAP50 keep list before GGUF slicing.

Blocks are scored independently; cross-block cancellation and the post-mixture RMSNorm are not part of this ranking.

GGUF build flow

Kimi-K3 weights
  → Quantization
  → Q1 / Q2 GGUF
  → REAP50 expert-axis slice
  → Width50 QK256-block slice
  → final split GGUF

The REAP50 step slices the expert axis of ffn_gate_inp.weight, exp_probs_b.bias, and ffn_{gate,up,down}_exps.weight from 896 to 448 for every MoE layer.

The Width50 step slices the intermediate axis of ffn_gate_exps.weight, ffn_up_exps.weight, and ffn_down_exps.weight from 3072 to 1536. Complete quantization blocks are copied directly, so retained data is not dequantized or requantized.

| | Original | This build |

|---|---:|---:|

| Routed experts per MoE layer | 896 | 448 |

| Routed expert FFN width | 3072 | 1536 |

| Experts used per token | 16 | 16 |

All other tensor data is unchanged. The shared expert metadata is represented as 4 × 1536 instead of 2 × 3072 so that its physical width remains 6144 in llama.cpp.

Files

| Folder | Shards | Size |

|---|---:|---:|

| UD-IQ1_S | 14 | about 181 GiB |

| UD-IQ1_M | 15 | about 194 GiB |

| UD-Q2_K_XL | 19 | about 243 GiB |

Load the first shard in the selected folder.

Usage

Use the Kimi-K3 Width support branch of llama.cpp.

git clone --branch kimi-k3-width-support https://github.com/mmnga/llama.cpp
cmake -S llama.cpp -B llama.cpp/build -DGGML_CUDA=ON -DCMAKE_BUILD_TYPE=Release
cmake --build llama.cpp/build -j --target llama-server

Example using UD-IQ1_S:

./llama.cpp/build/bin/llama-server \
  -m ./UD-IQ1_S/Kimi-K3-UD-IQ1_S-00001-of-00014.gguf \
  -ot ".*.ffn_.*_exps.*=CPU" \
  -ngl 45 \
  --ctx-size 8192 \
  --flash-attn on \
  --jinja \
  --override-kv kimi-k3.expert_shared_count=int:2 \
  --override-kv kimi-k3.expert_shared_feed_forward_length=int:6144 \
  --override-kv kimi-k3.expert_used_count=int:16

The example was tested with a 32 GB RTX 5090 while keeping routed experts on the CPU. Adjust -ngl and context size for your available VRAM and RAM. Load the first shard when using another quantization folder.

This is a heavily reduced experimental model. Quality and stability may differ from the original Kimi-K3.