---

license: apache-2.0

base_model:

• Qwen/Qwen3.6-35B-A3B

pipeline_tag: text-generation

library_name: transformers

tags:

• moe
• code
• expert-pruning
• qwen3.6

---

Qwen3.6-27B-A3B-Coder

A code-specialist expert prune of Qwen3.6-35B-A3B: the MoE is reduced from 256 experts to 184 (72 dropped per layer, ~35B→27B, still A3B active) using a code-targeted competence map (LiveCodeBench + MultiPL-E competence classes). Same router, attention, norms, MTP head and vision tower as the base — only the expert keep-set changes.

Served at top-10 (num_experts_per_tok = 10, baked as the default). This is a routing-recovery lever: after pruning to 184 experts, activating the top-10 (vs the base top-8) recovers instruction-following at no cost to code (see below). No fine-tuning, no distillation — pure expert selection + a routing-width dial.

Recipe

1. Competence map: the 256e teacher is profiled per-expert on a balanced corpus + targeted LiveCodeBench and MultiPL-E (Rust/Java/JS) PASS-response classes.
2. Drop map: wmax aggregation with the LCB + MPE classes up-weighted (1.5) → 72/256 experts dropped per layer, protecting the code-competent experts.
3. Top-10 routing (num_experts_per_tok = 10) baked into the config → the shipped default. Pass --override-kv qwen35moe.expert_used_count=int:8 to any llama.cpp tool to A/B back to native top-8.

Evaluation (Q6_K, llama.cpp, temp 0.6 / top-p 0.95 / top-k 20)

| Benchmark | This model | Qwen3.6-35B-A3B (256e) | coder (LCB-only) |

|---|---|---|---|

| GPQA-Diamond | 0.773 | 0.833 | 0.793 |

| MATH-500 | 0.620 | 0.730 | 0.620 |

| AIME | 0.733 | 0.633 | 0.767 |

| LiveCodeBench (v6, 77q) | 0.688 | 0.714 | 0.688 |

| IFEval | 0.730 | 0.960 | 0.840 |

| HumanEval | 0.970 | 0.970 | 0.963 |

| GSM8K | 0.970 | 0.960 | 0.980 |

| ARC-Challenge | 0.944 | 0.935 | 0.933 |

| MultiPL-E | 0.840 | 0.827 | 0.670 |

| Average | 0.808 | 0.840 | 0.806 |

Highlights: best code profile of any prune — MultiPL-E 0.840 (above the teacher; +17pp over the LCB-only coder that this model supersedes), LiveCodeBench 0.688 (tied best), HumanEval 0.970. Average 0.808 sits at the LCB-coder level and within 0.03 of the full teacher.

Verbosity / rumination (length breakdown per eval)

Aggressive expert pruning makes the model verbose on open-ended reasoning — it over-thinks before answering. This is largely inherited from the base (the 256e teacher does the same on GPQA/AIME) and is bounded by the generation cap; it does not affect the code benches, which have a natural termination anchor.

Response length in characters (content + reasoning), this model vs the 256e teacher; runaway = responses > 20k chars (of 100, or 30/198 for GPQA, 30 for AIME):

| Benchmark | p50 | p90 | max | runaway | 256e runaway |

|---|---|---|---|---|---|

| GPQA | 13.8k | 58.8k | 129k | 58 | 58 (same) |

| AIME | 52.9k | 85.5k | 96k | 25 | 29 |

| IFEval | 12.1k | 56.2k | 81k | 30 | 11 |

| MATH-500 | 2.3k | 14.5k | 76k | 9 | 12 |

| GSM8K | 2.5k | 12.5k | 108k | 6 | 3 |

| ARC | 1.4k | 2.3k | 59k | 7 | 0 |

| HumanEval | 0.8k | 1.4k | 24k | 1 | 2 |

| LiveCodeBench / MultiPL-E | — code path — | | | tight | tight |

Reading it: GPQA/AIME verbosity is essentially the base model (58 vs 58, 25 vs 29). Only IFEval shows prune-added rumination (30 vs 11) — the trade for the code-targeted drop map. Code and math-with-boxing tasks terminate cleanly. If you want tighter output, a repetition/length penalty at serve time (or top-8 via the override above) reduces the tail.

Reasoning budget and thinking stop phrase (llama.cpp)

Qwen 3.6 reasons at length by design, and on a hard prompt it can consume the

whole context window before it answers. llama.cpp can bound the thinking block

with a sampler, and — the part that actually matters — tell the model why the

block is being closed.

Needs llama.cpp b8508 or newer for the flags, b10091 or newer for the

per-request overrides.

Serve with a bounded thinking block

llama-server -m Qwen3.6-27B-A3B-Coder-Q4_K_M.gguf -c 32768 -ngl 99 \
    --jinja \
    --reasoning-budget 8192 \
    --reasoning-budget-message $'\n\nConsidering the limited time by the user, I have to give the solution based on the thinking directly now.\n' \
    --temp 0.6 --top-k 20 --top-p 0.95

| flag | meaning |

|---|---|

| --reasoning-budget N | -1 unrestricted (default), 0 close the block immediately, N > 0 cap it at N tokens |

| --reasoning-budget-message | text written into the block just before the closing tag is forced |

| --jinja | required — the delimiters come from the chat template (<think> … </think>). Without it llama.cpp has no tags to count and the budget silently does nothing |

Both flags also read from the environment: LLAMA_ARG_THINK_BUDGET and

LLAMA_ARG_THINK_BUDGET_MESSAGE.

--reasoning-format is not part of this. It only decides how the thinking

is handed back — message.reasoning_content versus left inline in

message.content — and never whether the budget is enforced: the delimiters the

sampler counts are set by the chat template regardless, so the cap binds under

auto, deepseek and none alike. The default auto already extracts

reasoning and is behaviourally identical to deepseek (they differ only in

name; the sole branch in the parser is != none). Leave it at the default so

the model's own tool-call and channel handling stays in play, and pin

deepseek only when a harness needs the thinking kept out of content.

--reasoning-budget on its own forces the closing tag the moment the budget

runs out, wherever the model happens to be. When that lands mid-thought the

model frequently does not register that it was interrupted: it carries on

reasoning, now inside the visible answer. The stop phrase is what prevents

that — it gives the model a reason to be finishing.

Two wordings that work

# "qwen" — the string Qwen's own service uses, from their docs
--reasoning-budget-message $'\n\nConsidering the limited time by the user, I have to give the solution based on the thinking directly now.\n'

# "voice" — shorter, in the model's own reasoning voice
--reasoning-budget-message $'\n\nOK, I have enough to answer now.\n'

Wording is model-specific: Qwen note that the ability to act on such a message

"is not explicitly trained but emerges naturally", so it is worth trying both

on your own workload. Leading and trailing newlines matter — they keep the

phrase off whatever half-finished line the cut landed on.

What it measures out to

Measured on the Qwen3.6-35B-A3B base this model is pruned from. Three hard

questions, temperature 0.6, fixed seed, answer characters with wall time in

brackets. Every run answered all three correctly, and thinking length is

unchanged by the message in every row:

| budget | no message | qwen | voice |

|---|---|---|---|

| 2048 | 1907 (69 s) | 1838 (42 s) | 1615 (41 s) |

| 4096 | 18015 (170 s) | 2642 (78 s) | 1441 (104 s) |

| 8192 | 3642 (158 s) | 1848 (129 s) | 2023 (175 s)|

The 4096 row is the failure this exists for: the cap lands mid-thought and the

reasoning simply continues in the answer, ten times longer and 2.2x the wall

time, for the same three correct answers. Both phrases remove it.

Per request, instead of per server

The server accepts both as request fields, overriding the command line:

{
  "messages": [ ... ],
  "thinking_budget_tokens": 8192,
  "reasoning_budget_message": "\n\nOK, I have enough to answer now.\n"
}

On the raw /completion endpoint the delimiters are not inferred, so they have

to be supplied with the budget:

{
  "prompt": "...",
  "reasoning_budget_tokens": 8192,
  "reasoning_budget_start_tag": "<think>",
  "reasoning_budget_end_tag": "</think>",
  "reasoning_budget_message": "\n\nOK, I have enough to answer now.\n"
}

On b10091 the message field must be present on /completion requests even

when empty: llama.cpp builds the sequence it forces from message + end_tag

inside that field's handler, so omitting it leaves the budget with nothing to

force — the sampler logs as though the cap fired while the thinking block stays

open.

Rules of thumb

• Keep -c several times larger than the budget. A budget equal to the context

lets the thinking phase fill the window on its own.

• A quarter of the context is a sensible starting point: 8192 at -c 32768.
• Qwen recommend keeping a thinking budget above 1024 tokens; below that

the cap tends to land before the model has committed to an approach.

• The budget is per thinking block, not per response — the sampler re-arms

when it sees a new opening tag, so a multi-turn agent gets a fresh window each

time.

Formats

• GGUF (this repo family): full imatrix quant sweep (Q8_0 → IQ2, plus ContribDynamic CD-* per-layer quants) in Qwen3.6-27B-A3B-Coder-MTP-GGUF. Includes the native MTP head (speculative decoding) and a -vision mmproj for multimodal use. imatrix.dat archived in-repo.
• Ollama: mannix/qwen3.6-27b-a3b-coder (text) and …-vision tags (with mmproj).

Quantization quality

Every K-quant (Q4_K_S → Q6_K and the _L variants) is built with imatrix. On this model the imatrix is load-bearing at 4-bit — the opposite of Gemma-4, where imatrix degrades K-quants. Measured on a deterministic greedy MultiPL-E code probe, the entire imatrix K-family and the ContribDynamic CD-\ tiers sit at full-precision parity (within measurement noise of the F16 anchor). The only outlier was a plain*, imatrix-free Q4_K_M, which fell ~12pp below parity — which is why imatrix is now the default for every tier in this repo.

• Recommended tier: CD-IQ4_K_M (~16 GB) — full-precision-parity code quality at the smallest at-parity size.
• Q4_0 / Q4_1 are not shipped — superseded by the imatrix K-quants (legacy round-to-nearest tiers offered no quality at their size).

⚠️ Hardware note — low i-quants (IQ2_M, IQ3_M) on Blackwell (sm_120) GPUs

The low i-quant tiers IQ2_M and IQ3_M produce incoherent output ("token salad") on NVIDIA Blackwell GPUs (sm_120, e.g. RTX PRO 6000) — under both stock llama.cpp and opencoti-llamafile, which share the same ggml CUDA IQ2_S/IQ3_S kernel. The weights are fine: the identical GGUF is fully coherent and produces correct code on CPU and on Ampere/Ada GPUs (verified on an RTX 3090). This is a llama.cpp/ggml CUDA-kernel issue on the sm_120 IQ2_S/IQ3_S path, not a defect in these files.

• On a Blackwell GPU, instead use: the K-quants (Q2_K_L, Q3_K_S/Q3_K_M/Q3_K_L), or IQ2_XS / IQ4_XS / the CD-* tiers — all coherent on Blackwell in the same size band (Q2_K_L/IQ2_XS ≈ the IQ2_M band; Q3_K_M/Q3_K_L ≈ the IQ3_M band).
• IQ2_M / IQ3_M are kept in the repo because they are correct on CPU and Ampere/Ada GPUs.

Notes

• Top-10 is baked as the default; the model was selected and evaluated at top-10.
• Same tokenizer, chat template, MTP head and vision tower as the base.
• Research checkpoint. Verbosity on open-ended prompts is a known, base-inherited trait.

Tool-calling benchmark — tool-eval-bench hardmode (88 scenarios, 176 pts)

Benchmarked file: Qwen3.6-27B-A3B-Coder-Q4_K_M.gguf (this repo), served on llama.cpp with MTP

speculative decoding enabled. The score below belongs to THIS quant — other

tiers in this repo were not run.

A3B-Coder scores 123.2 ±2.3 of 176 — last of the ten models in this cohort. That is

the measured result and it is reported here unmodified, but two things must be read with

it.

**First, this model is the one most damaged by a scorer defect in the harness version

used.** tool-eval-bench v2.6.0 crashes scoring TC-62 and records FAIL/0 while keeping

the scenario in the denominator (details below). A3B-Coder hits that crash on **all five

seeds** — more than any other model in the cohort — and later harness commits credit the

behaviour the crash discards. A corrected re-run would raise this figure; by how much is

not known, and no adjusted number is published here because it has not been measured.

Second, this is a coding-specialised prune, evaluated on general agentic tool use.

The profile is consistent with that: Parameter Precision 6/6 (100%), Creative

Composition 6/6, Toolset Scale 7.4/8 (92.5%), Structured Output 10.8/12 (90%) — the

mechanics of calling tools correctly are intact. What falls away is the long-horizon

agentic layer: Autonomous Planning 2.4/6 (40.0%) and Hard Mode 16.8/38 (44.2%),

both last in the cohort.

If you want this lineage with agentic tool use intact, use

A3B-CoderX instead: same

family, +14.2 pts (137.4), with Hard Mode 24.6/38 vs 16.8/38.

15 safety-critical failures across five seeds (TC-31, TC-34, TC-60 on every seed).

!Tool-calling benchmark

Full cohort

| model | quant | Total Points (mean, 5 seeds) | 95% CI | safety-critical (5 seeds) |

|---|---|---|---|---|

| Qwen3.8-27B-Omnimerge-v6 | Q4_K_M | 156.4 ±3.5 | [152.0, 160.8] | 3 |

| Qwen3.8-27B (base) | UD-Q4_K_M | 150.8 ±2.5 | [147.7, 153.9] | 9 |

| Ornith-1.5-35B | IQ4_XS | 146.2 ±2.6 | [143.0, 149.4] | 10 |

| Qwen3.6-27B-Omnimerge-v4 | Q4_K_M | 146.2 ±2.7 | [142.9, 149.5] | 16 |

| Qwen3.6-27B (base) | Q4_K_M | 144.0 ±3.4 | [139.8, 148.2] | 14 |

| Qwen3.6-35B-A3B (base) | IQ4_XS | 141.6 ±2.4 | [138.6, 144.6] | 15 |

| Qwen3.6-27B-A3B-CoderX | Q4_K_M | 137.4 ±4.9 | [131.3, 143.5] | 17 |

| Ornith-1.5-27B-A3B-Coder | IQ4_XS | 136.8 ±4.8 | [130.9, 142.7] | 12 |

| Ornith-1.5-27B-A3B-CoderX | IQ4_XS | 134.0 ±2.5 * | [130.8, 137.2] | 14 |

| Qwen3.6-27B-A3B-Coder | Q4_K_M | 123.2 ±2.3 | [120.4, 126.0] | 15 |

* one seed (s42) is graded on 174 pts, not 176 — see that model's card.

<details>

<summary><b>Basis — read before comparing these numbers to anything</b></summary>

• Scorer: tool-eval-bench v2.6.0 (the pip/uv-installed package, verified via

tool_eval_bench.__file__, not a git checkout). An earlier note in the runner claimed

cf54b4b (v2.6.0-45); that is wrong and has been corrected — no cell ever ran it.

All 50 cells ran the same v2.6.0, so the cohort is internally consistent.

• v2.6.0 carries a known scorer crash on TC-62. email_calls[-1] raises IndexError

when a model sent no valid CFO email; the orchestrator catches it and returns

FAIL / 0 points while keeping the scenario in the denominator. It hits 11 of 38

scored cells, 2 pts each, and it is **not neutral — it concentrates on the weakest

models**. Later harness commits credit that behaviour instead, so a fixed scorer would

raise affected scores, unevenly.

• 5 paired seeds [42–46], 64k context, context-pressure 0.25, max 8 turns, 120 s timeout,

thinking enabled, sampler temp 0.6 / top-p 0.95 / top-k 20 (not greedy).

• Served on llama.cpp b1788384120-c588c4f47 with MTP speculative decoding enabled

(nextn=YES spec=mtp), one model per GPU, sequential.

• Quant tiers are not uniform across the cohort (Q4_K_M for the Omnimerge/A3B rows,

IQ4_XS for Ornith and 35B-A3B, UD-Q4_K_M for the Qwen3.8 base). Cross-row gaps

therefore carry a quantisation component and are not purely architectural.

• Do not pool these with the r/LocalLLaMA published tool-eval-bench figures: those

were run at 256k context and are a different basis despite the shared scorer version.

</details>