---

language:

- en

- zh

base_model:

- Qwen/Qwen3.6-27B

tags:

- gguf

- llama.cpp

- qwen

- qwen3

- qwen3.6

- mtp

- speculative-decoding

- quantized

- long-context

- chinese

- distillation

- agent

- react

pipeline_tag: text-generation

license: apache-2.0

library_name: gguf

inference:

parameters:

temperature: 0.6

top_p: 0.95

model-index:

- name: Opus-DeepSeek-Distilled-Q4M

results:

- task:

type: text-generation

dataset:

type: benchlocal

name: BenchLocal 6-pack

metrics:

- type: benchlocal-score

value: 86.5

name: 6-pack Total

- task:

type: question-answering

dataset:

type: gpqa

name: GPQA-Diamond-198

metrics:

- type: accuracy

value: 83.84

name: Accuracy

- task:

type: question-answering

dataset:

type: mmlu

name: MMLU-500 (5-shot)

metrics:

- type: accuracy

value: 91.80

name: Accuracy

---

🧠 Opus-DeepSeek-Distilled-Q4M

> A distilled Qwen3.6-27B GGUF optimized for local agentic reasoning, tool use, and long-chain task execution.






---

⚠️ Sampling Parameters

> Please ensure temperature = 0.6 and top_p = 0.95 when using this model.

>

> This model was trained and validated at these specific parameters. Both too high and too low temperatures cause problems:

>

> 🔥 Too high (> 0.6) → Output becomes divergent as token probabilities flatten. This causes malformed tool calls, function name hallucinations, unstable parameter generation, and uncontrollable agent behavior.

>

> 🧊 Too low (< 0.3) → The model almost always picks the highest-probability token. In tool-calling scenarios this manifests as: repeating the same failed tool call instead of trying alternatives, shortened thinking traces leading to insufficient reasoning depth, and reduced robustness to edge cases.

>

> This is an important recommendation based on extensive real-world testing. Please verify these parameters in your inference framework.

---

📢 Highlights

| Area | Score | vs Qwen3.6-27B q4_k_m |

|------|-------|----------------------|

| BenchLocal 6-pack 🏆 | 86.5 | +8.3 |

| GPQA-Diamond-198 🔬 | 83.84% | +10.14% |

| BugFind-15 🐛 | 80 | +20 |

| ToolCall-15 🔧 | 97 | +4 |

| InstructFollow-15 📋 | 94 | +17 |

| StructOutput-15 📊 | 88 | +11 |

| MMLU-500 (5-shot) 📚 | 91.80% | ~tied (+0.2%) |

| DataExtract-15 📄 | 81 | -2 |

> 🚀 Output speed: ~60 tok/s on A100 40GB · ~100 tok/s on RTX PRO 6000 (q4_k_m + mtp=3)

---

🔥 Why This Model?

The original Qwen3.6-27B has solid foundational capabilities, but its agent behavior falls short — prone to infinite loops when thinking, lacks structured agent design, and has room to improve in math reasoning.

This variant tackles all three through targeted distillation:

1. ✅ Infinite loops → Eliminated. ReAct-style reasoning-action orchestration fixes the root cause.
2. ✅ Agent behavior → Structured. Distilled Claude Opus's systematic thinking and organization.
3. ✅ Math reasoning → Strengthened. Absorbed capabilities from strong math/logic models.

The result is a local agent that doesn't just score high on benchmarks — it works reliably in real engineering tasks like BugFind, where it now competes with GLM5.2.

> ⚠️ Note: Side-by-side GLM5.2 comparisons were evaluated using Opus 4.8 as a judge. Opus-as-judge has inherent biases — results are indicative, not definitive.

---

🎯 Design Philosophy

Core insight: Qwen3.6-27B doesn't lack capability — it lacks good agent behavior. That makes it worth iterating on.

We followed the ReAct paradigm (Yao et al., ICLR 2023) — unifying reasoning and action into an alternating, constrained, executable loop — rather than just making the model "think longer" or "call tools better."

| Teacher Model | Capability Distilled |

|---------------|---------------------|

| Claude Opus 🎯 | Systematic thinking, structured organization, concise reasoning |

| DeepSeek 🧭 | Stable agent behavior, tool orchestration, task closure |

| Math/Logic models ➗ | Mathematical reasoning, logical deduction |

---

📊 Performance

BenchLocal 6-pack

| Pack | q4_k_m (ours) | Qwen/Qwen3.6-27B q4_k_m | Delta |

|------|:-:|:-:|:-:|

| BugFind-15 🐛 | 80 | 60 | +20 |

| ToolCall-15 🔧 | 97 | 93 | +4 |

| DataExtract-15 📄 | 81 | 83 | -2 |

| InstructFollow-15 📋 | 94 | 77 | +17 |

| ReasonMath-15 ➗ | 79 | 79 | 0 |

| StructOutput-15 📊 | 88 | 77 | +11 |

| Total 🏆 | 86.5 | 78.2 | +8.3 |

Extended Evals

| Benchmark | Ours | Baseline | Notes |

|-----------|:----:|:--------:|-------|

| GPQA-Diamond-198 🔬 | 83.84% | 73.7% | +10.14%, all 198 graded locally |

| MMLU-500 (5-shot) 📚 | 91.80% | 91.6% | Approximately tied |

---

🛠️ Usage

Recommended Stack

🧩 OpenCode + LM Studio
📐 Temperature: 0.6 · Top-p: 0.95
⚡ q4_k_m + mtp=3

Quick Start (llama.cpp)

# Download the GGUF
huggingface-cli download your-org/Opus-DeepSeek-Distilled-Q4M \
  opus-deepseek-distilled-q4m-q4_k_m.gguf --local-dir ./models

# Run with llama.cpp
./llama-cli -m ./models/opus-deepseek-distilled-q4m-q4_k_m.gguf \
  --temp 0.6 --top-p 0.95 \
  -p "Your prompt here"

Via LM Studio

1. Load the GGUF file in LM Studio
2. Set backend to llama.cpp
3. Enable MTP (set depth=3) under inference options
4. Set temperature = 0.6, top_p = 0.95
5. Start the local API server
6. Connect via OpenCode or any OpenAI-compatible client

> 💡 Pro tip: For coding tasks, the temp 0.6 / top_p 0.95 combo delivers the best balance of creativity and correctness.

---

⚠️ Known Limitations

Evaluation Methodology

• Opus-as-judge biases: GLM5.2 comparisons are judge-evaluated, not absolute rankings
• MMLU: Single-run 5-shot result; variations in shot selection may cause fluctuation

Capability Boundaries

• DataExtract: Scores 81 vs 83 baseline — extraction tasks may have slight regression from distillation
• Closed-source teachers: Risks include inherited biases and TOS compliance — assess for your use case
• 27B scale ceiling: May still hit capacity limits on extremely complex long-chain reasoning

Deployment Notes

• MTP=3: Boosts throughput but adds VRAM overhead — disable or reduce on <24GB hardware
• Quantization: Only q4_k_m tested; q5_k_m may improve accuracy at higher VRAM cost
• Imatrix: Uses importance-matrix quantization, not standard k-quant — better parameter preservation at low bit widths

---

📝 Citation

@misc{opus-deepseek-distilled-q4m,
  title = {Opus-DeepSeek-Distilled-Q4M: A Distilled Agentic GGUF for Local Deployment},
  author = {Yin, Brian and BenchLocal Contributors},
  year = {2026},
  url = {https://github.com/brianyin/BenchLocal}
}

---

🙏 Acknowledgements

• Qwen team — excellent foundational model
• Unsloth — efficient training infrastructure
• Merkyor — identified ReAct as the key to solving agent infinite loops
• Community — built on existing open-source exploration and practical experience

> It is because of this work that came before that we can continue pushing forward, arriving at today's more stable, more practical, and more complete agent — and helping us get closer to the era of local agent AI.