---

license: apache-2.0

base_model:

• Qwen/Qwen3.6-35B-A3B

pipeline_tag: text-generation

library_name: transformers

tags:

• moe
• code
• expert-pruning
• qwen3.6

---

Qwen3.6-27B-A3B-Coder

A code-specialist expert prune of Qwen3.6-35B-A3B: the MoE is reduced from 256 experts to 184 (72 dropped per layer, ~35B→27B, still A3B active) using a code-targeted competence map (LiveCodeBench + MultiPL-E competence classes). Same router, attention, norms, MTP head and vision tower as the base — only the expert keep-set changes.

Served at top-10 (num_experts_per_tok = 10, baked as the default). This is a routing-recovery lever: after pruning to 184 experts, activating the top-10 (vs the base top-8) recovers instruction-following at no cost to code (see below). No fine-tuning, no distillation — pure expert selection + a routing-width dial.

Recipe

1. Competence map: the 256e teacher is profiled per-expert on a balanced corpus + targeted LiveCodeBench and MultiPL-E (Rust/Java/JS) PASS-response classes.
2. Drop map: wmax aggregation with the LCB + MPE classes up-weighted (1.5) → 72/256 experts dropped per layer, protecting the code-competent experts.
3. Top-10 routing (num_experts_per_tok = 10) baked into the config → the shipped default. Pass --override-kv qwen35moe.expert_used_count=int:8 to any llama.cpp tool to A/B back to native top-8.

Evaluation (Q6_K, llama.cpp, temp 0.6 / top-p 0.95 / top-k 20)

| Benchmark | This model | Qwen3.6-35B-A3B (256e) | coder (LCB-only) |

|---|---|---|---|

| GPQA-Diamond | 0.773 | 0.833 | 0.793 |

| MATH-500 | 0.620 | 0.730 | 0.620 |

| AIME | 0.733 | 0.633 | 0.767 |

| LiveCodeBench (v6, 77q) | 0.688 | 0.714 | 0.688 |

| IFEval | 0.730 | 0.960 | 0.840 |

| HumanEval | 0.970 | 0.970 | 0.963 |

| GSM8K | 0.970 | 0.960 | 0.980 |

| ARC-Challenge | 0.944 | 0.935 | 0.933 |

| MultiPL-E | 0.840 | 0.827 | 0.670 |

| Average | 0.808 | 0.840 | 0.806 |

Highlights: best code profile of any prune — MultiPL-E 0.840 (above the teacher; +17pp over the LCB-only coder that this model supersedes), LiveCodeBench 0.688 (tied best), HumanEval 0.970. Average 0.808 sits at the LCB-coder level and within 0.03 of the full teacher.

Verbosity / rumination (length breakdown per eval)

Aggressive expert pruning makes the model verbose on open-ended reasoning — it over-thinks before answering. This is largely inherited from the base (the 256e teacher does the same on GPQA/AIME) and is bounded by the generation cap; it does not affect the code benches, which have a natural termination anchor.

Response length in characters (content + reasoning), this model vs the 256e teacher; runaway = responses > 20k chars (of 100, or 30/198 for GPQA, 30 for AIME):

| Benchmark | p50 | p90 | max | runaway | 256e runaway |

|---|---|---|---|---|---|

| GPQA | 13.8k | 58.8k | 129k | 58 | 58 (same) |

| AIME | 52.9k | 85.5k | 96k | 25 | 29 |

| IFEval | 12.1k | 56.2k | 81k | 30 | 11 |

| MATH-500 | 2.3k | 14.5k | 76k | 9 | 12 |

| GSM8K | 2.5k | 12.5k | 108k | 6 | 3 |

| ARC | 1.4k | 2.3k | 59k | 7 | 0 |

| HumanEval | 0.8k | 1.4k | 24k | 1 | 2 |

| LiveCodeBench / MultiPL-E | — code path — | | | tight | tight |

Reading it: GPQA/AIME verbosity is essentially the base model (58 vs 58, 25 vs 29). Only IFEval shows prune-added rumination (30 vs 11) — the trade for the code-targeted drop map. Code and math-with-boxing tasks terminate cleanly. If you want tighter output, a repetition/length penalty at serve time (or top-8 via the override above) reduces the tail.

Formats

• GGUF (this repo family): full imatrix quant sweep (Q8_0 → IQ2, plus ContribDynamic CD-* per-layer quants) in Qwen3.6-27B-A3B-Coder-MTP-GGUF. Includes the native MTP head (speculative decoding) and a -vision mmproj for multimodal use. imatrix.dat archived in-repo.
• Ollama: mannix/qwen3.6-27b-a3b-coder (text) and …-vision tags (with mmproj).

Quantization quality

Every K-quant (Q4_K_S → Q6_K and the _L variants) is built with imatrix. On this model the imatrix is load-bearing at 4-bit — the opposite of Gemma-4, where imatrix degrades K-quants. Measured on a deterministic greedy MultiPL-E code probe, the entire imatrix K-family and the ContribDynamic CD-\ tiers sit at full-precision parity (within measurement noise of the F16 anchor). The only outlier was a plain*, imatrix-free Q4_K_M, which fell ~12pp below parity — which is why imatrix is now the default for every tier in this repo.

• Recommended tier: CD-IQ4_K_M (~16 GB) — full-precision-parity code quality at the smallest at-parity size.
• Q4_0 / Q4_1 are not shipped — superseded by the imatrix K-quants (legacy round-to-nearest tiers offered no quality at their size).

⚠️ Hardware note — low i-quants (IQ2_M, IQ3_M) on Blackwell (sm_120) GPUs

The low i-quant tiers IQ2_M and IQ3_M produce incoherent output ("token salad") on NVIDIA Blackwell GPUs (sm_120, e.g. RTX PRO 6000) — under both stock llama.cpp and opencoti-llamafile, which share the same ggml CUDA IQ2_S/IQ3_S kernel. The weights are fine: the identical GGUF is fully coherent and produces correct code on CPU and on Ampere/Ada GPUs (verified on an RTX 3090). This is a llama.cpp/ggml CUDA-kernel issue on the sm_120 IQ2_S/IQ3_S path, not a defect in these files.

• On a Blackwell GPU, instead use: the K-quants (Q2_K_L, Q3_K_S/Q3_K_M/Q3_K_L), or IQ2_XS / IQ4_XS / the CD-* tiers — all coherent on Blackwell in the same size band (Q2_K_L/IQ2_XS ≈ the IQ2_M band; Q3_K_M/Q3_K_L ≈ the IQ3_M band).
• IQ2_M / IQ3_M are kept in the repo because they are correct on CPU and Ampere/Ada GPUs.

Notes

• Top-10 is baked as the default; the model was selected and evaluated at top-10.
• Same tokenizer, chat template, MTP head and vision tower as the base.
• Research checkpoint. Verbosity on open-ended prompts is a known, base-inherited trait.