---

license: apache-2.0

base_model: Qwen/Qwen3-Coder-30B-A3B-Instruct

tags:

• gguf
• moe
• qwen3moe
• expert-pruning
• structural-derivative
• xyntetik-runner

pipeline_tag: text-generation

---

Qwen3-Coder-30B-A3B keep-120, a structurally pruned derivative (Q4_K_M, GGUF)

<!-- BEGIN runner-toolcalling -->

Serve with xyntetik-runner &mdash; tool calls that still parse when the budget runs out

This GGUF is built to be served with xyntetik-runner, a single-binary, dependency-free C11 inference engine (CPU / CUDA / Metal, OpenAI-compatible server). Its one differentiator you cannot get from another engine:

> When a tool call runs past its token budget, Runner closes it to the smallest schema-legal document, so the arguments still parse.

Most OpenAI-compatible engines return finish_reason: "length" with an empty or malformed tool_calls the caller has to repair or retry. Runner does forced-truncation recovery: once a JSON document starts, it emits a legal ending when the budget expires. On local models &mdash; tight context, slow generation &mdash; that is the difference between an agent loop that finishes and one that retries from scratch.

Measured &mdash; what each engine hands the caller when the token budget cuts a tool call short (same box, same tool schema, same prompt, tool_choice:"required", temperature 0, budgets 1&rarr;64):

| engine | budget too small (1&ndash;16 tokens) | enough budget (64, control) |

|---|---|---|

| Runner | executable tool_calls, arguments parse | completes |

| vLLM 0.27.1 | no call; protocol framing leaks into content | completes |

| llama.cpp b10488 | no call; leak, then tool_calls with unparseable args | completes |

| Ollama 0.32.14 | no call; empty content, then HTTP 500 | completes |

| TensorRT-LLM 1.2.1 &dagger; | no call; <tool_call> leak, then empty content | completes |

| SGLang 0.5.17 &dagger; | no call; <tool_call> leak, then empty content | completes |

Every engine completes at 64 tokens &mdash; the failure is truncation, not misconfiguration. Below that, only Runner returns an executable call. This is the behaviour across every OpenAI-compatible engine measured, not a claim about engines not measured. &dagger; TensorRT-LLM and SGLang were measured on a Qwen3-1.7B substitute; truncation recovery is a property of the runtime, not the model.

Constrained decoding also holds the shape of a tool call under quantization: on a full quant ladder, schema conformance and tool selection stayed 100% down to Q4_0 (argument agreement decayed to 50%) &mdash; it guarantees the shape of a call at any quantization, not its contents.

Quickstart

# build the single binary (no deps), then serve this GGUF:
runner -m Qwen3-Coder-30B-A3B-Instruct-keep120-Q4_K_M.gguf --serve
# point any OpenAI-compatible client at http://localhost:8080/v1

Full recipe, raw responses, and the per-release regression gate: the truncation benchmark.

<!-- END runner-toolcalling -->

Run it

This file is 17.5 GB with expert tensors kept in their native Q4_K / Q6_K

precision, so it wants a machine with roughly 20 GB of usable memory or more.

On a smaller machine, start with

gemma-4-E2B-it-Q4_0

instead.

# Apple Silicon Mac
curl -L https://github.com/Joakimpalm-Zen/xyntetik-runner/releases/latest/download/runner-macos-arm64.tar.gz | tar xz
curl -L -o Qwen3-Coder-30B-A3B-Instruct-keep120-Q4_K_M.gguf \
  https://huggingface.co/Joakimpalm-Zen/Qwen3-Coder-30B-A3B-Instruct-keep120-Q4_K_M-GGUF/resolve/main/Qwen3-Coder-30B-A3B-Instruct-keep120-Q4_K_M.gguf
./runner-macos-arm64 -m Qwen3-Coder-30B-A3B-Instruct-keep120-Q4_K_M.gguf -i

Linux and Windows builds are on the

same release page.

Using a different runtime? That is part of the point of this artifact. The

reduced expert count is declared in the GGUF metadata, so a runtime that

resolves num_experts from the file loads it unchanged, and one that assumes

"qwen3moe means 128 experts" fails. Either result is informative; please

report it either way, including the runtime and its version.

**This is not a quantization of Qwen3-Coder-30B-A3B. It is a structural

modification:** each MoE layer's expert roster is pruned from 128 to its own

top-120 experts, selected from measured routing data. The expert FFN tensors

remain in their native Q4_K / Q6_K precision, untouched, not requantized, and

top-8 routing is retained. Category-wise:

| Model | Architecture change | Experts/layer | Routing | Expert format | Category |

|---|---|---|---|---|---|

| Qwen3-Coder-30B-A3B-Instruct (Qwen) | none | 128 | top-8 | Q4_K / Q6_K | official |

| this file | 8 experts pruned per layer | 120 | top-8 | Q4_K / Q6_K (unchanged) | structural / expert-pruned derivative |

• File: Qwen3-Coder-30B-A3B-Instruct-keep120-Q4_K_M.gguf (17,531,967,808 bytes)
• SHA256: 3ac8568224a15ea49db6affdd195ba48e8b2d7c6325c7849b9829381e89163a1
• Parent GGUF: lmstudio-community/Qwen3-Coder-30B-A3B-Instruct-GGUF, revision

1f4ceb1041258b3fbfe59e1175d1321c6b41863b

(SHA256 79ad15a5ee3caddc3f4ff0db33a14454a5a3eb503d7fa1c1e35feafc579de486, 18.6 GB)

• Base model: Qwen/Qwen3-Coder-30B-A3B-Instruct, revision

b2cff646eb4bb1d68355c01b18ae02e7cf42d120, apache-2.0 verified on the base

repo. The parent file's SHA256 was verified byte for byte against the source

repo's LFS hash before pruning.

Why these eight experts: the methodology

Reproducible pipeline, tooling public in the

Xyntetik Runner repo:

1. Trace: run the unpruned parent over a mixed evaluation corpus with the

runner's MoE trace enabled (RUNNER_MOE_TRACE writes JSONL of every

routing decision: position, layer, selected experts, gate weights,

activation norms). This trace covered 4,441 tokens and 213,216 records.

1. Saliency: scripts/moe-prune-plan.py scores each (layer, expert) pair

by the sum of gate weight times activation L2 norm over every selection, a

REAP-style saliency that weighs how much each pick moved the residual

stream, not merely how often it was chosen.

1. Plan: --keep-n 120; every MoE layer keeps its own top-120 by

saliency, so the discarded eight differ per layer. The corpus did not

select every expert in every layer (observed range 112 to 128); experts

never selected carry zero saliency and are tied at the bottom, and the

plan is padded to a uniform 120 with those zero-saliency ties so the

prune is uniform and the expert_count metadata is correct. **The exact

plan used for this file is published here as

qwen3coder-keep120-plan.json**; with

the parent file and the runner, the build is fully reproducible.

1. Prune: runner --prune-experts plan.json --quantize out.gguf drops

the expert tensors and the matching router rows; quantized blocks are

copied, not recoded.

1. Gate: KL divergence plus top-1 agreement against the unpruned parent

(scripts/kld-compare-raw.py, mixed corpus, temperature 0, word-by-word

teacher forcing, KLD over the union of both sides' top-20 logprobs).

Retained saliency mass is 99.989% mean across layers (worst layer 99.80%).

Note that saliency-mass coverage is not a predictor of the gate result;

a sibling model retained 99.56% of mass at a deeper prune and still missed

the top-1 bar. The gate below is the evidence; the mass figure is context.

Validation: and where the cliff is

Publication bar: top-1 at or above 97%, mean KLD at or below 0.05 vs the

parent, over 400 teacher-forced positions.

| metric | measured | bar | verdict |

|---|---|---|---|

| top-1 agreement vs parent | 97.25% | >= 97% | pass |

| margin-qualified top-1 (bar v2, re-measured 2026-08-14) | 100.00% | >= 97% | pass |

| mean KL divergence | 0.00738 | <= 0.05 | pass |

| mean top-8 overlap | 0.9734 | none | none |

| positions scored | 400 (0 failed) | 400 | none |

Zero-point self-check, run first: the identical protocol with the parent

on both sides returned mean KLD 0.0, top-1 100.0%, top-8 overlap 1.0 over 40

positions. The harness contributes no divergence of its own, so the numbers

above are attributable to the removed experts. A 100-position screen returned

96.0% top-1 / 0.0116 KLD before the full gate; both are reported so the

screen-to-gate movement is visible rather than hidden.

The margin is thin (0.25 points) and the frontier around it is measured:

| Variant | Top-1 vs its parent | Verdict |

|---|---|---|

| keep-120 (this file) | 97.25% (mean KLD 0.00738) | passes, published |

| deeper prunes of this model | not gated | not shipped; only what was measured to pass is published |

| Qwen3-30B-A3B (non-Coder sibling) keep-120 | 95.50% | fails the same bar at the same 6.25% drop |

The sibling row is the interesting one: the base Qwen3-30B-A3B is byte

identical in shape to this model and fails where this one passes. Expert

prunability is a property of the specific training run, not of the

architecture. It was measured here rather than assumed.

Scope: the gate measures agreement with the parent's own distribution,

not downstream capability. A 2.75% top-1 disagreement rate is a real

behavioural difference, and this artifact has not been benchmarked on coding

tasks. If you need the parent's exact behaviour, use the parent.

Re-measured 2026-08-14 under the project's adopted dual-column bar

(margin-qualified top-1 beside plain top-1, 0.5-nat reference-side tie

band): this artifact passes BOTH the original bar and the current one,

and remains the only published file in the account's quality series to

clear the original bar unaided. The same campaign withdrew a different

artifact's claim (see gpt-oss-20b-keep30), so this row stands under the

standard that retired that one.

Measured delta vs the stock Q4_K_M

| | stock (18.6 GB) | keep-120 (17.5 GB) |

|---|---|---|

| Disk / resident weights | 18,632,186,176 B | 17,531,967,808 B (5.90% smaller) |

| Geometry | 48 layers, 128 experts, top-8 | 48 layers, 120 experts, top-8 (unchanged otherwise) |

| Quality | reference | 97.25% top-1, mean KLD 0.00738 |

The 1.1 GB saved is not transformative on its own; it buys KV-cache headroom

on machines where the stock file barely fits. This artifact exists because

the quality frontier is measured and stated, and because no other keep-N of

this model has cleared a published gate.

Running it

Built and certified with

Xyntetik Runner, a

from-scratch C inference engine (CPU AVX2/NEON, CUDA, Metal, OpenAI-compatible

server) whose --caps reports memory limits so a scheduler can check

placement before loading:

./runner -m Qwen3-Coder-30B-A3B-Instruct-keep120-Q4_K_M.gguf --serve --port 8080

Any GGUF runtime with qwen3moe support that reads the expert count from

metadata should also load it (llama.cpp does).

Publication policy

This project publishes only artifacts that passed the quality gate.

Experiments that failed it ship as negative-result documentation in the

runner and suite repos instead of as uploads: the same night this file was

gated, the sibling Qwen3-30B-A3B failed keep-120 at 95.50%, gpt-oss-120b

failed at 85.50%, and none of them were published. An optimization does not

pass because the file got smaller; it has to preserve the model and produce

a configuration worth running.

Measured envelope

Every artifact this account publishes ships a measured-envelope sidecar

(Qwen3-Coder-30B-A3B-Instruct-keep120-Q4_K_M.gguf.envelope.json, in this repo) — the same file

xyntetik-runner reads at

load: download it next to the GGUF and the runner reports the measured state at

load. A measured envelope is a **dated observation, not a standing quality

claim** — it records exactly what this file is, where it came from, and what

was measured, so the claims above stay traceable.

| | |

|---|---|

| artifact sha256 | 3ac8568224a15ea49db6affdd195ba48e8b2d7c6325c7849b9829381e89163a1 |

| what changed | keep-120 expert prune via --prune-experts (each layer keeps its own top-120 by measured routing mass) |

| reference (unpruned parent) sha256 | 79ad15a5ee3caddc3f4ff0db33a14454a5a3eb503d7fa1c1e35feafc579de486 |

| fidelity vs parent | PASSES the bar — plain top-1 97.25%, margin-qualified 100.00%, mean KLD 0.00738 (table above) |

| tool calling | generic constrained envelope (truncation-safe; see the tool-calling section above) |

| measured | 2026-08-14 |

| verdict | experimental — loads with a banner; never a standing certification |