---

license: apache-2.0

base_model:

• Qwen/Qwen3.6-35B-A3B

pipeline_tag: text-generation

library_name: gguf

tags:

• gguf
• imatrix
• moe
• code
• expert-pruning
• qwen3.6

---

Qwen3.6-27B-A3B-CoderX — GGUF (imatrix, MTP)

GGUF quantizations of CoderX, a long-horizon code prune of

Qwen3.6-35B-A3B: the MoE is reduced from

256 experts to 184 (72 dropped per layer, ~35B→27B, still A3B active).

Same expert budget as the sibling

Qwen3.6-27B-A3B-Coder,

but a different selection — same competence map, different keep-set. Served at top-8 — the

base model's native routing, no override needed. No fine-tuning, no distillation.

CoderX is not a strict upgrade over Coder. It is the long-horizon sibling: far

stronger on multi-step competitive-programming work, a tie on short multi-language

completion, and its non-code profile is not yet measured. Pick by workload.

Recipe

1. Competence map: the 256e teacher is profiled per-expert on a balanced corpus plus

targeted LiveCodeBench and MultiPL-E/HumanEval PASS-response classes.

1. Selection: our saliency ranking picks the 184 survivors, with a **REAP-style

per-layer floor (p=24)** protecting the tail — the constraint the plain saliency cut

over-prunes.

1. No redistribution: the 72 evicted experts per layer are discarded, not merged, and

every retained expert stays bit-identical to the base model's. Expert-folding variants were

built and measured against this cut, and rejected — they cost 14pp on LiveCodeBench-v6 and

drove the generation-loop rate from 16% to 78%.

1. Top-8 routing — the base default, left alone. The Coder sibling bakes top-10; this

model does not, and that was measured rather than assumed (below).

Evaluation (Q6_K + imatrix, llama.cpp b9700, greedy)

| Benchmark | CoderX | Coder (184e) | Qwen3.6-35B-A3B (256e) |

|---|---|---|---|

| LiveCodeBench v6 (77q, 24k think / 48k total) | 0.727 ▲ | 0.610 | 0.610 |

| HumanEval+ (164) | 0.970 ▲ | 0.951 | 0.939 |

| MultiPL-E-100 (rs+java+js, 300 completions) | 0.887 | 0.890 | 0.910 |

▲ = beats both the 256e teacher and the Coder sibling. Every cell is greedy

(temperature 0.0 / top_p 1.0 / top_k 0), one serving geometry per bench, pinned in the

eval template and read back out of the server log rather than chosen at launch time. All

three models ran on the same binary, the same quant recipe and the same basis.

Read the MultiPL-E row carefully. A same-basis repeat of that bench — identical

weights, quant, sampler and geometry — moved 1.0 pp on batch-scheduling

nondeterminism alone. The 0.33 pp CoderX↔Coder gap sits inside that band and should be

read as a tie; the 2.33 pp gap to the teacher sits outside it and is real. Per

language (CoderX / Coder / 256e): Rust 0.85 / 0.81 / 0.84 · Java 0.89 / 0.90 / 0.93 ·

JS 0.92 / 0.96 / 0.96. CoderX takes Rust and gives up JavaScript.

Routing ships top-8, and that was measured: MBPP-full 0.784 / 0.790 at top-8 against

0.732 / 0.730 at top-10, with repeat spreads ≤ 0.3 pp on both arms. MultiPL-E was inside

noise either way.

Not measured yet. The full canonical 9-bench suite has not been run on this

checkpoint. GPQA, MATH-500, IFEval and ARC are deliberately absent from the table above

rather than carried over from a different sampler or a different basis — code-targeted

expert cuts in this family are known to cost science and multilingual headroom, so treat

the non-code profile as unknown until those cells land. There is also no per-tier quality

sweep yet: the quant table below is sizes and imatrix provenance only.

Quantizations

Every K- and I-quant tier is built with an importance matrix. This was verified by

reading the quantize.imatrix.* keys back out of each uploaded file rather than trusting

the build flag — all 18 quant tiers report calibration_datav5.txt, **510 entries, 128

chunks**. Q8_0 carries none by rule. imatrix.dat is archived in this repo, so any tier

can be reproduced or re-quantized to a matching characteristic.

On this model family the imatrix is load-bearing at 4-bit — the opposite of Gemma-4,

where it degrades K-quants.

| Tier | GiB | GB | imatrix |

|---|---|---|---|

| Q8_0 | 25.99 | 27.91 | — (by rule) |

| Q6_K_L | 20.32 | 21.81 | ✓ |

| Q6_K | 20.09 | 21.57 | ✓ — eval reference tier, every number above |

| Q5_K_L | 17.73 | 19.04 | ✓ |

| Q5_K_M | 17.44 | 18.73 | ✓ |

| Q5_K_S | 16.91 | 18.16 | ✓ |

| Q4_K_L | 15.31 | 16.43 | ✓ |

| Q4_K_M | 14.95 | 16.06 | ✓ — the ollama :latest default |

| Q4_K_S | 14.04 | 15.07 | ✓ |

| IQ4_NL | 13.98 | 15.01 | ✓ |

| IQ4_XS | 13.25 | 14.22 | ✓ |

| Q3_K_L | 12.80 | 13.74 | ✓ |

| Q3_K_XL | 12.26 | 13.17 | ✓ |

| Q3_K_M | 11.85 | 12.72 | ✓ |

| IQ3_M | 10.94 | 11.74 | ✓ |

| Q3_K_S | 10.74 | 11.53 | ✓ |

| Q2_K_L | 9.65 | 10.36 | ✓ |

| IQ2_M | 8.44 | 9.06 | ✓ |

| IQ2_XS | 7.64 | 8.20 | ✓ |

F16 (48.87 GiB / 52.47 GB) is also present as the quantization source.

Both units are given because they get confused constantly: HF reports GiB, the ollama

page quotes GB. Sizes are the actual file bytes, so the ladder is ordered by size

rather than by name — Q3_K_L really is larger than Q3_K_XL on this architecture.

*No ContribDynamic CD- tiers in this repo yet** — unlike the Coder sibling. If you

want the CD ladder, use Coder for now.

Speculative decoding — the MTP head ships in every tier

The native MTP block (blk.40) is included in every GGUF here, so speculative

decoding works without a separate draft model.

On ollama it is already wired: every tag carries PARAMETER draft_num_predict 3, which

runs --spec-type draft-mtp --spec-draft-n-max 3 --spec-draft-backend-sampling.

n=3 is a deliberate choice, not a default. On an RTX 5080 (Blackwell):

189.65 tok/s baseline → 251.94 at n=3 (+33%), falling to 145.37 at n=8 — *worse than

not speculating at all*. On a 3090 the same grid peaks at n=8 instead. The optimum is

hardware-dependent; n=3 is near-peak on both, so it is the shipped default. Raise it

yourself if you are on Ampere and measuring. Acceptance at n=3: mean accepted length

3.16, per-position (0.841, 0.730, 0.587).

Vision

The vision-<tier> tags on ollama attach the released Qwen3.6-27B-A3B-Coder mmproj

(clip projector, projection_dim 2048, matching this model's embedding_length 2048).

Note that the bf16 checkpoint this repo was quantized from is text-only — the base

model's vision tower is not carried by that checkpoint (the merge-engine save path emits trunk

modules only), so

Qwen3.6-27B-A3B-CoderX is

Qwen3_5MoeForCausalLM, not the Coder sibling's Qwen3_5MoeForConditionalGeneration.

Multimodal use goes through the mmproj at the GGUF layer, which is unaffected.

Serving

llama-server -m Qwen3.6-27B-A3B-CoderX-Q4_K_M.gguf -c 32768 -ngl 99 \
    --jinja \
    --reasoning-budget 8192 \
    --reasoning-budget-message $'\n\nConsidering the limited time by the user, I have to give the solution based on the thinking directly now.\n' \
    --temp 0.6 --top-k 20 --top-p 0.95

CoderX thinks long. On LiveCodeBench its median completion is ~15.8k tokens against

~2.2k for the published Coder, and 60 of 77 completions trip a repetition heuristic — but

42 of those 60 still pass. The length is where the LCB win comes from, not a defect.

If your work is LCB-shaped, give it context headroom rather than clamping it.

Reasoning budget and thinking stop phrase (llama.cpp)

Qwen 3.6 reasons at length by design, and on a hard prompt it can consume the whole

context window before it answers. llama.cpp can bound the thinking block with a sampler

and — the part that actually matters — tell the model why the block is being closed.

Needs llama.cpp b8508 or newer for the flags, b10091 or newer for the per-request

overrides.

| flag | meaning |

|---|---|

| --reasoning-budget N | -1 unrestricted (default), 0 close the block immediately, N > 0 cap it at N tokens |

| --reasoning-budget-message | text written into the block just before the closing tag is forced |

| --jinja | required — the delimiters come from the chat template (<think> … </think>). Without it llama.cpp has no tags to count and the budget silently does nothing |

Both flags also read from the environment: LLAMA_ARG_THINK_BUDGET and

LLAMA_ARG_THINK_BUDGET_MESSAGE.

--reasoning-format is not part of this. It only decides how the thinking is handed

back — message.reasoning_content versus left inline in message.content — and never

whether the budget is enforced: the delimiters the sampler counts are set by the chat

template regardless, so the cap binds under auto, deepseek and none alike. Leave it

at the default so the model's own tool-call and channel handling stays in play, and pin

deepseek only when a harness needs the thinking kept out of content.

--reasoning-budget on its own forces the closing tag the moment the budget runs out,

wherever the model happens to be. When that lands mid-thought the model frequently does

not register that it was interrupted: it carries on reasoning, now inside the visible

answer. The stop phrase is what prevents that — it gives the model a reason to be

finishing.

# "qwen" — the string Qwen's own service uses, from their docs
--reasoning-budget-message $'\n\nConsidering the limited time by the user, I have to give the solution based on the thinking directly now.\n'

# "voice" — shorter, in the model's own reasoning voice
--reasoning-budget-message $'\n\nOK, I have enough to answer now.\n'

Wording is model-specific: Qwen note that the ability to act on such a message *"is not

explicitly trained but emerges naturally"*, so it is worth trying both on your own

workload. Leading and trailing newlines matter — they keep the phrase off whatever

half-finished line the cut landed on.

Measured on the Qwen3.6-35B-A3B base this model is pruned from. Three hard questions,

temperature 0.6, fixed seed, answer characters with wall time in brackets. Every run

answered all three correctly, and thinking length is unchanged by the message in every

row:

| budget | no message | qwen | voice |

|---|---|---|---|

| 2048 | 1907 (69 s) | 1838 (42 s) | 1615 (41 s) |

| 4096 | 18015 (170 s) | 2642 (78 s) | 1441 (104 s) |

| 8192 | 3642 (158 s) | 1848 (129 s) | 2023 (175 s) |

The 4096 row is the failure this exists for: the cap lands mid-thought and the reasoning

simply continues in the answer, ten times longer and 2.2x the wall time, for the same

three correct answers. Both phrases remove it.

The server accepts both as request fields, overriding the command line:

{
  "messages": [ ... ],
  "thinking_budget_tokens": 8192,
  "reasoning_budget_message": "\n\nOK, I have enough to answer now.\n"
}

On the raw /completion endpoint the delimiters are not inferred, so they have to be

supplied with the budget:

{
  "prompt": "...",
  "reasoning_budget_tokens": 8192,
  "reasoning_budget_start_tag": "<think>",
  "reasoning_budget_end_tag": "</think>",
  "reasoning_budget_message": "\n\nOK, I have enough to answer now.\n"
}

On b10091 the message field must be present on /completion requests even when empty:

llama.cpp builds the sequence it forces from message + end_tag inside that field's

handler, so omitting it leaves the budget with nothing to force — the sampler logs as

though the cap fired while the thinking block stays open.

Rules of thumb

• Keep -c several times larger than the budget. A budget equal to the context lets the

thinking phase fill the window on its own.

• A quarter of the context is a sensible starting point: 8192 at -c 32768.
• Qwen recommend keeping a thinking budget above 1024 tokens; below that the cap tends

to land before the model has committed to an approach.

• The budget is per thinking block, not per response — the sampler re-arms when it

sees a new opening tag, so a multi-turn agent gets a fresh window each time.

Formats

• bf16 safetensors: Qwen3.6-27B-A3B-CoderX

(text-only, MTP included).

• Ollama: mannix/qwen3.6-27b-a3b-coderx

— 19 text tags plus a vision-<tier> for each, MTP baked in.

Built with omnimergekit.

Notes

• Top-8 is the base default and is what this model was selected and evaluated at.
• Same tokenizer and chat template as the base.
• Research checkpoint. The non-code profile is unmeasured; see the evaluation section.

Apache-2.0.

Tool-calling benchmark — tool-eval-bench hardmode (88 scenarios, 176 pts)

Benchmarked file: Qwen3.6-27B-A3B-CoderX-Q4_K_M.gguf (this repo), served on llama.cpp with MTP

speculative decoding enabled. The score below belongs to THIS quant — other

tiers in this repo were not run.

**A3B-CoderX scores 137.4 ±4.9 of 176, seventh of ten — and +14.2 pts over

A3B-Coder (123.2)**, the

largest gap between any two models in this cohort's lower half. The CIs do not overlap.

The gain is concentrated exactly where Coder gives way: **Hard Mode 24.6/38 (64.7%) vs

16.8/38 (44.2%), Autonomous Planning 3.8/6 (63.3%) vs 2.4/6 (40.0%)**, and

Instruction Following 9.6/10 (96.0%) vs 8.0/10. Perfect on Tool Selection and Parameter

Precision; Localization 5.6/6 (93.3%), Toolset Scale 7.6/8 (95.0%).

Weakest at Safety & Boundaries 17.6/26 (67.7%) and Context & State 15.0/20 (75.0%).

17 safety-critical failures across five seeds — TC-31, TC-34 and TC-60 on every seed,

plus TC-35 (Contradictory Parameters) on two. That is the highest count in the cohort;

if untrusted tool output is in scope for your deployment, read the safety notes below.

CoderX hits the TC-62 scorer crash on 3 of 5 seeds, so this figure is mildly

depressed relative to a corrected harness.

!Tool-calling benchmark

Full cohort

| model | quant | Total Points (mean, 5 seeds) | 95% CI | safety-critical (5 seeds) |

|---|---|---|---|---|

| Qwen3.8-27B-Omnimerge-v6 | Q4_K_M | 156.4 ±3.5 | [152.0, 160.8] | 3 |

| Qwen3.8-27B (base) | UD-Q4_K_M | 150.8 ±2.5 | [147.7, 153.9] | 9 |

| Ornith-1.5-35B | IQ4_XS | 146.2 ±2.6 | [143.0, 149.4] | 10 |

| Qwen3.6-27B-Omnimerge-v4 | Q4_K_M | 146.2 ±2.7 | [142.9, 149.5] | 16 |

| Qwen3.6-27B (base) | Q4_K_M | 144.0 ±3.4 | [139.8, 148.2] | 14 |

| Qwen3.6-35B-A3B (base) | IQ4_XS | 141.6 ±2.4 | [138.6, 144.6] | 15 |

| Qwen3.6-27B-A3B-CoderX | Q4_K_M | 137.4 ±4.9 | [131.3, 143.5] | 17 |

| Ornith-1.5-27B-A3B-Coder | IQ4_XS | 136.8 ±4.8 | [130.9, 142.7] | 12 |

| Ornith-1.5-27B-A3B-CoderX | IQ4_XS | 134.0 ±2.5 * | [130.8, 137.2] | 14 |

| Qwen3.6-27B-A3B-Coder | Q4_K_M | 123.2 ±2.3 | [120.4, 126.0] | 15 |

* one seed (s42) is graded on 174 pts, not 176 — see that model's card.

<details>

<summary><b>Basis — read before comparing these numbers to anything</b></summary>

• Scorer: tool-eval-bench v2.6.0 (the pip/uv-installed package, verified via

tool_eval_bench.__file__, not a git checkout). An earlier note in the runner claimed

cf54b4b (v2.6.0-45); that is wrong and has been corrected — no cell ever ran it.

All 50 cells ran the same v2.6.0, so the cohort is internally consistent.

• v2.6.0 carries a known scorer crash on TC-62. email_calls[-1] raises IndexError

when a model sent no valid CFO email; the orchestrator catches it and returns

FAIL / 0 points while keeping the scenario in the denominator. It hits 11 of 38

scored cells, 2 pts each, and it is **not neutral — it concentrates on the weakest

models**. Later harness commits credit that behaviour instead, so a fixed scorer would

raise affected scores, unevenly.

• 5 paired seeds [42–46], 64k context, context-pressure 0.25, max 8 turns, 120 s timeout,

thinking enabled, sampler temp 0.6 / top-p 0.95 / top-k 20 (not greedy).

• Served on llama.cpp b1788384120-c588c4f47 with MTP speculative decoding enabled

(nextn=YES spec=mtp), one model per GPU, sequential.

• Quant tiers are not uniform across the cohort (Q4_K_M for the Omnimerge/A3B rows,

IQ4_XS for Ornith and 35B-A3B, UD-Q4_K_M for the Qwen3.8 base). Cross-row gaps

therefore carry a quantisation component and are not purely architectural.

• Do not pool these with the r/LocalLLaMA published tool-eval-bench figures: those

were run at 256k context and are a different basis despite the shared scorer version.

</details>