---

license: apache-2.0

base_model: Qwen/Qwen3.6-35B-A3B

tags:

• moe
• pruning
• reap
• gguf
• quantized
• coding
• tool-calling

language:

• en

pipeline_tag: text-generation

library_name: llama.cpp

model-index:

• name: Qwen3.6-35B-A3B-REAP-48-Q3K-mixed-GGUF

results:

- task:

type: text-generation

name: Code Generation

dataset:

name: HumanEval

type: openai_humaneval

metrics:

- type: pass@1

value: 0.951

name: pass@1

- task:

type: text-generation

name: Code Generation

dataset:

name: HumanEval+

type: evalplus/humanevalplus

metrics:

- type: pass@1

value: 0.896

name: pass@1

- task:

type: text-generation

name: Code Generation

dataset:

name: MBPP+

type: evalplus/mbppplus

metrics:

- type: pass@1

value: 0.735

name: pass@1

- task:

type: text-generation

name: Function Calling

dataset:

name: BFCL-simple

type: gorilla-llm/Berkeley-Function-Calling-Leaderboard

metrics:

- type: accuracy

value: 0.918

name: accuracy

- task:

type: text-generation

name: Code Generation

dataset:

name: BigCodeBench-Complete (v0.1.4)

type: bigcode/bigcodebench

metrics:

- type: pass@1

value: 0.518

name: pass@1

---

Qwen 3.6 35B-A3B · REAP-48 · Q3K-mixed (imatrix)

A 9.4 GB mixture-of-experts coding/tool-use specialist derived from

Qwen/Qwen3.6-35B-A3B by pruning

48% of routed experts with REAP and quantizing the result with an

attention-aware mixed-precision scheme calibrated by llama-imatrix.

The result fits a 16 GB Mac Mini (M4) — the unmodified base model does not

fit on the same hardware at the same quantization scheme — and on coding

benchmarks it **outperforms a same-quant version of the unmodified base

model** on three of four metrics.

TL;DR

| | Base · bf16 | Base · Q3K-mixed | REAP-48 · bf16 | REAP-48 · Q3K-mixed (this model) |

|---|---:|---:|---:|---:|

| Parameters | 34.66 B | 34.66 B | 21.69 B | 19.17 B |

| File size on disk | ~66 GB | 16.1 GB | 41.4 GB | 9.4 GB |

| Fits 16 GB Mac | no | no | no | yes |

| HumanEval | 96.3% | 91.5% | 95.7% | 95.1% |

| HumanEval+ | 92.1% | 88.4% | 91.5% | 89.6% |

| MBPP+ | 76.7% | 76.5% | 75.9% | 73.5% |

| BFCL-simple | 90.8% | 89.5% | 91.2% | 91.8% |

| BigCodeBench-Complete | 56.3% | — | 53.0% | 51.8% |

Method

Compression: REAP, 48 % expert pruning

Routed experts are scored by router-weighted activation magnitude on a

512-sample agentic-coding calibration mix (~1.04 M tokens, seed=42); the

lowest-scoring 48 % (123 of 256 routed experts per layer) are pruned, leaving

1. The shared expert and attention layers are preserved. See arXiv:2510.13999

for the method.

Plain REAP is used (no routing-aware variant), because Qwen 3.6 has a shared

expert that absorbs ~69 % of layer output norm — it already catches the load

when a routed expert is pruned, so the routing-disruption penalty is

double-counted by the architecture itself.

Result before quantization: 34.66 B → 19.17 B parameters; bf16 size 41.4 GB.

Quantization: Q3K-mixed + imatrix

Selective scheme via llama-quantize:

| Tensor group | Type |

|---|---|

| Token embeddings, output | Q8_0 |

| Attention attn_q, attn_k, attn_v, attn_output | Q8_0 |

| All FFN/expert weights, GDN/SSM tensors | Q3_K |

Importance matrix computed by llama-imatrix over 1.86 M tokens (253 samples,

n_ctx=4096) on the bf16 GGUF, using the same calibration mix as the REAP scoring.

Result: 9.4 GB on disk, 3.93 BPW.

Calibration data

Roughly 60 % agentic / tool-use, 25 % code reasoning, 10 % math, 5 % general:

| Source | Weight | Why |

|---|---:|---|

| SWE-bench/SWE-smith-trajectories | 25 % | Multi-turn agentic coding traces |

| lambda/hermes-agent-reasoning-traces (kimi) | 20 % | Tool-use planning |

| NousResearch/hermes-function-calling-v1 | 15 % | Modern tool-call format |

| open-r1/Mixture-of-Thoughts (code) | 15 % | Code reasoning |

| open-r1/codeforces-cots | 10 % | Competitive coding |

| open-r1/Mixture-of-Thoughts (math) | 10 % | Code-adjacent math |

| allenai/c4 (en) | 5 % | Forgetting anchor |

All datasets retain role structure through the chat template (no flattening).

Inference

llama.cpp / llama-server (recommended)

llama-server \
  -m qwen36-reap-48pct-mixed-q3k.gguf \
  --port 8093 --host 0.0.0.0 \
  -c 8192 -ngl 999 \
  --jinja --reasoning off

Recommended sampling

• temperature: 0 (deterministic for code), or 0.2 for variation
• max_tokens: 2048 for code generation, 4096+ for agentic tool-call chains

KV cache for long context

On a 16 GB Mac, with -ctk q8_0 -ctv q8_0 you can fit the full 262 K native

context window. Without KV quantization, expect ~150 K context comfortably.

Performance (M4 Mac Mini, Metal)

| Context | Prompt processing | Token generation |

|---|---:|---:|

| 4 K | 407 tok/s | 26.6 tok/s |

| 63 K | 139 tok/s | 13.2 tok/s |

Known issues

BigCodeBench-Complete shows a real compression+quant tax

Of all benchmarks measured, BigCodeBench shows the largest gap between this

artifact and the unmodified bf16 base: 51.8 % (REAP-48 Q3K) vs 56.3 % (base

bf16) — a combined compression+quantization tax of ~4.5 pp, equivalent to

~92 % of base performance retained. The other benchmarks (HumanEval, MBPP+,

BFCL-simple) showed taxes well under 1 pp because they are largely saturated.

BigCodeBench exercises real-library multi-step reasoning (matplotlib, pandas,

sklearn, scipy, requests, etc.) where pruned experts contribute marginally;

that's where the cost of removing 48 % of routed experts shows up. The

shipping artifact still lands in the upper end of the "Solid" band of the

public BigCodeBench-Complete leaderboard, but is not frontier-tier.

MBPP+ small regression vs base Q3K

At Q3K-mixed, REAP-48 loses 11 problems on MBPP+ vs same-quant base

(73.5 % vs 76.5 %). MBPP+ specifically stresses Python edge cases

(empty inputs, large inputs, type coercion) on small functions; this is the

one metric where the compressed model is consistently weaker than the base

at this quant level.

Verbose self-correction on the hardest algorithmic problems

On at least one LeetCode-hard problem (median of two sorted arrays in

O(log(min(m,n)))) we observed the model write an incomplete attempt, abandon

it mid-function with pass, and re-implement the solution in a second

function — producing a correct final answer at ~1000 output tokens instead

of the ~500 a clean implementation would need. We have not characterized how

common this is; if you stream code generation for hard algorithmic prompts,

consider max_tokens >= 1500 to allow space for the recovery.

Reproducibility

This artifact was produced by the Crucible compression toolkit (developed

alongside this model; source not yet publicly released — planned). Pipeline:

# 1. REAP scoring (on the bf16 base)
crucible observe Qwen/Qwen3.6-35B-A3B \
  --samples 512 --batch-size 2 --seed 42 \
  -o results/qwen36-base-observation.json

# 2. REAP pruning at 48 %
crucible compress Qwen/Qwen3.6-35B-A3B \
  --method reap --ratio 0.48 \
  --scores-file results/qwen36-base-observation.json \
  -o outputs/qwen36-reap-48pct

# 3. f16 GGUF
python llama.cpp/convert_hf_to_gguf.py \
  outputs/qwen36-reap-48pct/Qwen3.6-35B-A3B-reap-48pct \
  --outfile qwen36-reap-48pct-f16.gguf --outtype f16

# 4. imatrix
python crucible/scripts/gen_imatrix_data.py \
  -o /tmp/imatrix_calibration.txt --samples 256
llama-imatrix \
  -m qwen36-reap-48pct-f16.gguf \
  -f /tmp/imatrix_calibration.txt \
  -o qwen36-48pct.imatrix \
  --n-gpu-layers 999 -c 4096

# 5. Q3K-mixed quantize
llama-quantize \
  --imatrix qwen36-48pct.imatrix \
  --token-embedding-type Q8_0 --output-tensor-type Q8_0 \
  --tensor-type attn_q=Q8_0 --tensor-type attn_k=Q8_0 \
  --tensor-type attn_v=Q8_0 --tensor-type attn_output=Q8_0 \
  qwen36-reap-48pct-f16.gguf \
  qwen36-reap-48pct-mixed-q3k.gguf \
  Q3_K_S

Evaluation methodology

All numbers above were produced by Crucible's own-harness bench subcommand

(no evalplus / lm-eval runtime dependency). Each problem is run once at

temperature=0. Generated code is executed in a subprocess with a 30 s

timeout (60 s for BigCodeBench, which has heavier tests). For HumanEval/+

the harness prepends the original prompt to the model's completion

(preserving imports). For MBPP+ the assertion list is appended to the

model's function. For BFCL-simple, both native tool_calls and in-content

<tool_call> / JSON / {fn: {args}} fallbacks are parsed. For BigCodeBench

(v0.1.4 split, 1140 problems) the model's body completion is appended to the

provided prompt; a unittest.TestLoader-based shim discovers any

TestCase subclasses in the assembled module and runs them via

TextTestRunner, exiting non-zero on failure. The harness has no dependency

on the BigCodeBench evaluator package; it does require the libraries used

by the problems themselves (pandas, numpy, matplotlib, seaborn, scikit-learn,

scipy, requests, Pillow, etc.) to be present in the runtime, or about half

the problems fail at import time.

Note on BFCL on llama.cpp: the BFCL dataset uses Python type names

(dict, float, tuple, any) in its tool parameter schemas. vLLM

silently accepts these; llama.cpp's --jinja schema validator rejects

non-standard types with HTTP 400. The harness's _bfcl_normalize_fn

remaps these to valid JSON-Schema types (object, number, array,

string) before sending. Without this normalization, BFCL-simple drops

~14-16 percentage points on llama.cpp regardless of model — a runtime

fixture issue, not a model regression.

Per-problem result JSONs (task_id / passed / error / raw response,

truncated) are retained locally and can be shared on request.

Intended use & limitations

Designed as a coding/tool-use specialist for local single-user inference

on consumer hardware. Strong at: function-level Python, idiomatic API usage,

multi-step refactors, single- and multi-turn function calling.

Not designed for, and not measured on: multilingual generation, image/audio

input, long-form prose, factual QA. General knowledge benchmarks (MMLU, GPQA)

were not run on this artifact.

The base Qwen 3.6-35B-A3B model includes a model.visual.* vision encoder

(~0.89 GB at bf16) and a multi-token-prediction head (mtp.*, ~1.69 GB). The

Crucible compressor operates only on the language-model component

(model.language_model.*); these other components are not carried forward

into the compressed artifact.

License

Apache-2.0, inherited from

Qwen/Qwen3.6-35B-A3B.

Citation

@article{lasby2025reap,
  title={REAP the Experts: Why Pruning Prevails for One-Shot MoE compression},
  author={Lasby, Mike and Lazarevich, Ivan and Sinnadurai, Nish and
          Lie, Sean and Ioannou, Yani and Thangarasa, Vithursan},
  journal={arXiv preprint arXiv:2510.13999},
  year={2025}
}