---

license: apache-2.0

base_model: barozp/Qwen3.6-29B-REAP-Opus-Reasoning-Distill

base_model_relation: quantized

pipeline_tag: text-generation

tags:

• gguf
• llama.cpp
• quantized
• moe
• qwen3
• reasoning

---

Qwen3.6-29B-REAP-Opus-Reasoning-Distill-GGUF

GGUF quantizations of a REAP-pruned (205/256 experts) Qwen3.6-35B-A3B MoE, merged with an Opus-reasoning-distilled LoRA adapter. This is the plain merge — no Multi-Token Prediction (MTP) head.

A version with an MTP head grafted on (for self-speculative decoding) is available at barozp/Qwen3.6-29B-REAP-Opus-Reasoning-Distill-MTP-GGUF. See the benchmarks below to decide which one fits your hardware.

Converted with llama.cpp's convert_hf_to_gguf.py.

Highlights

• The reasoning LoRA beats the unpruned base model on ARC-Challenge (0.616 vs 0.532) — a net skill gain from distillation, not just recovered pruning loss. See Quality benchmarks.
• Identical speed to the MTP release with speculative decoding off (219.06 vs 219.34 tok/s, full offload) — no downside to picking this file unless you specifically want the MTP speedup. See Benchmarks.

Files

| Quant | Size | Notes |

|---|---:|---|

| BF16 | 56.5 GB | full precision reference |

| Q8_0 | 30.1 GB | near-lossless |

| Q6_K | 23.2 GB | |

| Q5_K_M | 20.2 GB | |

| Q4_K_M | 17.3 GB | most popular K-quant |

| Q4_K_S | 16.2 GB | |

| IQ4_XS | 15.3 GB | imatrix-based, smaller & often better than Q4_K_S |

| Q3_K_M | 13.7 GB | |

| IQ3_M | 12.6 GB | |

| IQ3_XXS | 11.2 GB | |

| Q2_K | 10.6 GB | |

| IQ2_M | 9.6 GB | |

All quants ≤Q4_K_M were calibrated with an importance matrix (imatrix) built from 512 samples of barozp/opus-reasoning-distill-train. K-quants (Q_K) favor broad compatibility and fast CPU inference; IQ-quants (IQ*) require the imatrix and give better quality per bit at ≤4-bit, at some CPU-inference speed cost.

Why choose this over the MTP version?

The MTP-grafted version carries an extra decoder layer used for self-speculative decoding. When that feature is left enabled on hardware that's memory-bandwidth-constrained (e.g. a small/weaker GPU with heavy CPU offload), the draft+verify overhead can compete with an already-scarce resource and result in slower generation than this plain model. This release removes that footgun entirely — no toggle to remember, always the same speed as the MTP file with speculative decoding off.

Benchmarks

Measured with llama-cli (Q4_K_M, flash attention on, greedy decoding, 5 runs per config, mean ± std) on an NVIDIA RTX PRO 6000 Blackwell Server Edition (97 GB VRAM). Full methodology and the MTP comparison are on the MTP-GGUF model card.

| -ngl | tok/s (generation) |

|---:|---:|

| 99 (full offload) | 219.06 ± 0.10 |

| 20 (partial offload) | 58.60 ± 0.52 |

This matches the MTP-GGUF file with speculative decoding disabled (--spec-type none) within measurement noise — confirming the MTP head, when unused, carries no VRAM/compute penalty. If your setup benefits from speculative decoding (compute-bound, strong GPU, full offload), the MTP release may be faster; see its model card for numbers (+39–67% observed in our tests).

Quality benchmarks

Measured with lm-evaluation-harness (HF backend, bfloat16, chat template disabled — see note) on the underlying safetensors checkpoint, against the same base chain: unpruned Qwen3.6-35B-A3B → REAP 205/256 pruning only (no LoRA, RangerX/Qwen3.6-35B-REAP-Pruned-ratio-0.2) → this checkpoint.

| Task | Axis | Qwen3.6-35B-A3B | RangerX REAP-20% | This model |

|---|---|---:|---:|---:|

| wikitext perplexity (lower=better) | sanity | 7.85 | 10.06 | 10.06 |

| MMLU | out-of-calibration | 0.8409 | 0.8152 | 0.8257 |

| HellaSwag (0-shot, acc_norm) | out-of-calibration | 0.7420 | 0.7440 | 0.7340 |

| ARC-Challenge (0-shot, acc_norm) | out-of-calibration | 0.5320 | 0.5340 | 0.6160 |

Key finding: the reasoning LoRA doesn't just recover REAP's pruning loss on ARC-Challenge — it pushes the score above the unpruned 256-expert base model (0.616 vs 0.532), a genuine reasoning-skill transfer from the Opus chain-of-thought training data (ARC-Challenge appears in neither REAP's calibration mixture nor the LoRA's training data). MMLU sees a smaller but real gain (+1.3pp retained vs. the LoRA-less pruned checkpoint); wikitext perplexity is unaffected; HellaSwag is flat within measurement noise.

Note: chat template was tested and found to badly corrupt loglikelihood-based multiple-choice scoring for this model family (MMLU dropped from 0.85 to 0.38 on the base model with it on) — all numbers above are with it off, applied consistently across every checkpoint. HellaSwag/ARC-Challenge are 0-shot, also applied consistently. Full methodology on the safetensors model card.

Credits

• Base architecture: Qwen3.6-35B-A3B (MoE), pruned via REAP (Router-weighted Expert Activation Pruning) to 205/256 experts.
• Reasoning distillation: LoRA fine-tune on Opus-generated reasoning traces (barozp/opus-reasoning-distill-train).