bytkim/Qwen3.6-27B-MTP-pi-reasoning-GGUF overview

---

license: apache-2.0

license_link: https://huggingface.co/Qwen/Qwen3.6-27B/blob/main/LICENSE

library_name: gguf

pipeline_tag: image-text-to-text

base_model:

- Qwen/Qwen3.6-27B

base_model_relation: finetune

datasets:

- bytkim/deepseek-v4-pro-pi-reasoning-sample-traces

model-index:

- name: Qwen3.6-27B-MTP-pi-reasoning-GGUF

results:

- task:

type: text-generation

name: Pi agentic coding

dataset:

name: Terminal-Bench 2.0

type: terminal-bench-2

metrics:

- name: pass@1

type: pass@1

value: 40.45

source:

name: Qwen3.6 27B reasoning writeup

url: https://huggingface.co/blog/bytkim/qwen36-27b-reasoning

tags:

- gguf

- llama.cpp

- qwen

- qwen3_6

- mtp

- multi-token-prediction

- speculative-decoding

- conversational

- image-text-to-text

- image-to-text

- multimodal

- vision

- vision-language

- vlm

- image

- text-generation-inference

- reasoning

- thinking

- reasoning-trained

- agent

- coder

- devops

- tool-use

- function-calling

- long-context

- pi

- qlora

- sft

---

<div style="font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Roboto,sans-serif;border:1px solid #c7d2fe;border-radius:20px;overflow:hidden;background:#ffffff;margin-bottom:28px;box-shadow:0 16px 42px -32px rgba(67,56,202,0.68);">

<div style="background:radial-gradient(circle at 92% 8%, rgba(45,212,191,0.62) 0%, rgba(45,212,191,0) 48%), radial-gradient(circle at 8% 100%, rgba(251,191,36,0.28) 0%, rgba(251,191,36,0) 42%), linear-gradient(125deg,#312e81 0%,#4338ca 46%,#0e7490 100%);padding:34px 34px 30px;color:white;position:relative;">

<div style="display:inline-block;font-size:10px;font-weight:800;letter-spacing:0.18em;text-transform:uppercase;color:#cffafe;background:rgba(15,23,42,0.36);padding:6px 11px;border-radius:999px;border:1px solid rgba(207,250,254,0.36);margin-bottom:14px;">Reasoning · Pi Tune · MTP · GGUF</div>

<h1 style="margin:0;font-size:32px;font-weight:850;color:white;border:none;line-height:1.1;letter-spacing:-0.03em;">Qwen3.6 27B tuned for reasoning-heavy local coding agents.</h1>

<p style="margin:14px 0 0 0;font-size:15.5px;color:#e0f2fe;font-weight:500;line-height:1.58;max-width:780px;">A 4-bit QLoRA SFT reasoning release for Pi-style terminal loops: inspect the repo, plan the fix, run commands, edit files, validate, and recover when the first attempt fails.</p>

</div>

<div style="display:flex;gap:8px;flex-wrap:wrap;padding:16px 34px;background:linear-gradient(180deg,#f8fafc 0%,#ffffff 100%);border-bottom:1px solid #e2e8f0;">

<span style="background:#eef2ff;color:#4338ca;font-size:11px;font-weight:750;padding:6px 12px;border-radius:999px;border:1px solid #c7d2fe;">🧠 Qwen3.6 27B base</span>

<span style="background:#fff1f2;color:#9f1239;font-size:11px;font-weight:750;padding:6px 12px;border-radius:999px;border:1px solid #fecdd3;">🔎 Reasoning-supervised</span>

<span style="background:#ecfeff;color:#0e7490;font-size:11px;font-weight:750;padding:6px 12px;border-radius:999px;border:1px solid #a5f3fc;">⚡ MTP speculative decoding</span>

<span style="background:#fff7ed;color:#c2410c;font-size:11px;font-weight:750;padding:6px 12px;border-radius:999px;border:1px solid #fed7aa;">🛠️ Coding · DevOps · Agents</span>

<span style="background:#f5f3ff;color:#6d28d9;font-size:11px;font-weight:750;padding:6px 12px;border-radius:999px;border:1px solid #ddd6fe;">📦 llama.cpp GGUF</span>

<span style="background:#fdf2f8;color:#be185d;font-size:11px;font-weight:750;padding:6px 12px;border-radius:999px;border:1px solid #fbcfe8;">🖼️ Vision sidecar compatible</span>

<span style="background:#f0fdfa;color:#0f766e;font-size:11px;font-weight:750;padding:6px 12px;border-radius:999px;border:1px solid #99f6e4;">🪟 128k tested context</span>

</div>

</div>

<div style="display:grid;grid-template-columns:repeat(2,1fr);gap:14px;margin-bottom:30px;">

<div style="display:flex;gap:14px;align-items:flex-start;border:1px solid #e2e8f0;border-radius:16px;padding:18px;background:#ffffff;">

<div style="flex:0 0 44px;height:44px;border-radius:13px;display:flex;align-items:center;justify-content:center;background:linear-gradient(135deg,#eef2ff 0%,#ecfeff 100%);font-size:20px;border:1px solid #c7d2fe;">🧭</div>

<div>

<div style="font-size:13.5px;font-weight:800;color:#0f172a;margin-bottom:4px;">Better planning before action</div>

<div style="font-size:12.75px;color:#475569;line-height:1.55;">Trained on passed agent trajectories with reasoning preserved, so the model can decompose tasks before committing to commands and edits.</div>

</div>

</div>

<div style="display:flex;gap:14px;align-items:flex-start;border:1px solid #e2e8f0;border-radius:16px;padding:18px;background:#ffffff;">

<div style="flex:0 0 44px;height:44px;border-radius:13px;display:flex;align-items:center;justify-content:center;background:linear-gradient(135deg,#fff7ed 0%,#fef3c7 100%);font-size:20px;border:1px solid #fed7aa;">🧪</div>

<div>

<div style="font-size:13.5px;font-weight:800;color:#0f172a;margin-bottom:4px;">Verifier-driven debugging</div>

<div style="font-size:12.75px;color:#475569;line-height:1.55;">Strongest in loops where the agent can run tests, inspect failures, patch code, and validate the whole task end-to-end.</div>

</div>

</div>

<div style="display:flex;gap:14px;align-items:flex-start;border:1px solid #e2e8f0;border-radius:16px;padding:18px;background:#ffffff;">

<div style="flex:0 0 44px;height:44px;border-radius:13px;display:flex;align-items:center;justify-content:center;background:linear-gradient(135deg,#ecfeff 0%,#f0fdfa 100%);font-size:20px;border:1px solid #a5f3fc;">⚡</div>

<div>

<div style="font-size:13.5px;font-weight:800;color:#0f172a;margin-bottom:4px;">MTP at every quant</div>

<div style="font-size:12.75px;color:#475569;line-height:1.55;">The MTP next-token draft heads are kept at Q8_0 precision inside each quant, enabling speculative decoding even with low-bit GGUF files.</div>

</div>

</div>

<div style="display:flex;gap:14px;align-items:flex-start;border:1px solid #e2e8f0;border-radius:16px;padding:18px;background:#ffffff;">

<div style="flex:0 0 44px;height:44px;border-radius:13px;display:flex;align-items:center;justify-content:center;background:linear-gradient(135deg,#fdf2f8 0%,#f5f3ff 100%);font-size:20px;border:1px solid #fbcfe8;">🚀</div>

<div>

<div style="font-size:13.5px;font-weight:800;color:#0f172a;margin-bottom:4px;">Start with Q4_K_M</div>

<div style="font-size:12.75px;color:#475569;line-height:1.55;">The default recommendation is Q4_K_M: a practical quality / memory tradeoff for local coding-agent experiments.</div>

</div>

</div>

</div>

Model overview

| Attribute | Details |

| --- | --- |

| Base model | Qwen/Qwen3.6-27B |

| Format | GGUF |

| Runtime target | llama.cpp / OpenAI-compatible local serving |

| Tuning focus | Pi-style terminal agents, repository work, command use, file edits, validation loops |

| Fine-tuning style | 4-bit QLoRA SFT on private passed agent trajectories |

| Reasoning supervision | Preserved reasoning traces for complex agent tasks |

| Recommended first file | Qwen3.6-27B-MTP-pi-reasoning-Q4_K_M.gguf |

| Vision support | Compatible with the Qwen3.6 mmproj-F16.gguf sidecar |

| Technical writeup | Qwen3.6 27B reasoning writeup |

This is the reasoning-oriented sibling to bytkim/Qwen3.6-27B-MTP-pi-tune-GGUF. Use this release for the stronger Pi-style coding-agent behavior. Use the no-thinking / direct tune when you specifically want lower-latency instruct-style turns.

Terminal-Bench 2.0 Pi snapshot

All results below are single-run pass@1 Terminal-Bench 2.0 results on the 89-task set using Pi with llama.cpp and Q4 GGUF. See the technical writeup for the broader training and evaluation context.

| Run | Harness | Runtime | Quantization | Pass | Total | Score |

| --- | --- | --- | --- | ---: | ---: | ---: |

| Base Qwen3.6 27B | Pi | llama.cpp | Q4 GGUF | 38 | 89 | 42.70% |

| Qwen3.6 MTP pi-tune | Pi | llama.cpp | Q4 GGUF | 25 | 89 | 28.09% |

| Qwen3.6 MTP pi-reasoning | Pi | llama.cpp | Q4 GGUF | 36 | 89 | 40.45% |

The base model remained the strongest single Pi run, but the reasoning-tuned model recovered most of the performance lost by the earlier direct/no-thinking tune and showed a different behavioral profile:

• stronger task decomposition;
• more targeted debugging loops;
• better end-to-end validation on some infrastructure and data tasks;
• more willingness to investigate system state before editing.

The tradeoff is that the same exploratory behavior can become over-exploration when the model fails to converge. On some failures it spends too long investigating or validating and can time out.

> [!IMPORTANT]

> These are applied harness-fluency results, not a complete measure of Qwen3.6 27B coding ability. They are single-run pass@1; the base run used a longer timeout than the reasoning run, so timeout-related comparisons should be read carefully.

Why this tune exists

Qwen3.6 already has strong coding ability and supports long-context, reasoning-capable inference. The goal of this release was narrower: improve how the model behaves inside a lightweight coding harness where the next useful action is often a terminal command, file edit, or verifier check.

The training target is the full agent loop:

understand task → inspect files → plan → run commands → edit → test → recover → finish

The reasoning-tuned variant is most useful when tasks are ambiguous, multi-step, or failure-prone — the cases where a little deliberation before action can save several bad edits later.

Pick a file

Recommended starting point: Q4_K_M.

| Quant | File size | VRAM estimate | Suggested use |

| --- | ---: | ---: | --- |

| Q2_K | ~11 GB | ~13 GB | Smallest footprint; expect quality tradeoffs. |

| Q3_K_S | ~12 GB | ~15 GB | Low-memory 3-bit option. |

| Q3_K_M | ~14 GB | ~16 GB | Balanced 3-bit option. |

| Q3_K_L | ~15 GB | ~17 GB | Higher-quality 3-bit option. |

| Q4_K_S | ~16 GB | ~18 GB | Smaller 4-bit option. |

| Q4_K_M | ~17 GB | ~19 GB | Best first choice for most local users. |

| Q5_K_S | ~19 GB | ~21 GB | Higher-quality 5-bit option. |

| Q5_K_M | ~20 GB | ~22 GB | Strong quality/memory tradeoff. |

| Q6_K | ~22 GB | ~25 GB | High-quality local inference if you have memory headroom. |

| bf16 | ~55 GB | ~58 GB | BF16 GGUF reference. |

VRAM estimates are approximate and depend on context length, KV cache type, GPU offload, batch settings, and runtime build. Longer contexts require more memory.

> [!NOTE]

> Every quant in this release keeps the MTP nextn prediction heads at Q8_0 precision. If your llama.cpp build supports draft-mtp, speculative decoding can be used with any quant above.

Quickstart

llama.cpp with MTP speculative decoding

This profile matches the Pi evaluation style most closely: long context, reasoning enabled by the harness/client, and MTP draft decoding.

llama-server -hf bytkim/Qwen3.6-27B-MTP-pi-reasoning-GGUF:Q4_K_M \
  --spec-type draft-mtp \
  --spec-draft-n-max 3 \
  -np 1 \
  --jinja -ngl 99 -fa -c 131072 \
  --cache-type-k q4_0 --cache-type-v q4_0 \
  --temp 1.0 --top-p 0.95 --top-k 0 --min-p 0

Notes:

• -c 131072 advertises a 128k context window. Reduce it if you are memory constrained.
• The Pi evaluations used a max generation budget of 8192 tokens per turn and reasoning budget of 4096 where the client/harness exposed that control.

llama.cpp without MTP

llama-server -hf bytkim/Qwen3.6-27B-MTP-pi-reasoning-GGUF:Q4_K_M \
  --jinja -ngl 99 -fa -c 131072 \
  --cache-type-k q4_0 --cache-type-v q4_0 \
  --temp 1.0 --top-p 0.95 --top-k 0 --min-p 0

Direct / instruct-style turns

For lower-latency direct responses, use Qwen3.6's instruct-style sampling and disable thinking in clients that expose chat-template options.

llama-server -hf bytkim/Qwen3.6-27B-MTP-pi-reasoning-GGUF:Q4_K_M \
  --jinja -ngl 99 -fa -c 131072 \
  --temp 0.7 --top-p 0.8 --top-k 20 --min-p 0 \
  --presence-penalty 1.5

If direct/no-thinking mode is your main use case, the sibling release may be a better fit: bytkim/Qwen3.6-27B-MTP-pi-tune-GGUF.

llama.cpp --jinja template note

Some llama.cpp versions may fail with the embedded Qwen Jinja chat template:

No user query found in messages.

If this happens, use a llama.cpp-compatible fixed Qwen template, for example:

https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates

This appears to be a chat-template compatibility issue rather than a model-weight issue.

Ollama

ollama run hf.co/bytkim/Qwen3.6-27B-MTP-pi-reasoning-GGUF:Q4_K_M

Download one file

hf download bytkim/Qwen3.6-27B-MTP-pi-reasoning-GGUF \
  Qwen3.6-27B-MTP-pi-reasoning-Q4_K_M.gguf \
  --local-dir .

Vision / image-text-to-text

This release is compatible with the Qwen3.6 mmproj-F16.gguf sidecar for vision-language inference. The fine-tune is language-side only; image/video understanding is inherited from the upstream base model.

hf download bytkim/Qwen3.6-27B-MTP-pi-reasoning-GGUF \
  Qwen3.6-27B-MTP-pi-reasoning-Q4_K_M.gguf \
  --local-dir .

hf download unsloth/Qwen3.6-27B-MTP-GGUF \
  mmproj-F16.gguf \
  --local-dir .

llama-server -m ./Qwen3.6-27B-MTP-pi-reasoning-Q4_K_M.gguf \
  --mmproj ./mmproj-F16.gguf \
  --jinja -ngl 99 -fa -c 131072 \
  --temp 1.0 --top-p 0.95 --top-k 0 --min-p 0

For a quick text-and-image session without spinning up a server:

llama-mtmd-cli -m ./Qwen3.6-27B-MTP-pi-reasoning-Q4_K_M.gguf \
  --mmproj ./mmproj-F16.gguf

OpenAI-compatible local API

llama-server exposes an OpenAI-compatible /v1/chat/completions endpoint, so existing OpenAI-style clients and custom harnesses can point at it directly.

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8080/v1",
    api_key="not-needed",
)

resp = client.chat.completions.create(
    model="Qwen3.6-27B-MTP-pi-reasoning",
    messages=[
        {"role": "system", "content": "You are a precise coding agent."},
        {"role": "user", "content": "Inspect this failing test output and propose the next terminal command."},
    ],
    temperature=1.0,
    top_p=0.95,
)

print(resp.choices[0].message.content)

The same endpoint can be used with streaming and OpenAI-style tool definitions, depending on your client and harness.

Training notes

The reasoning release was trained with 4-bit QLoRA SFT on private, passed agent trajectories. The v2 training set contained roughly 1,200 passed trajectories with reasoning preserved as a first-class target.

The dataset emphasized:

• terminal and shell-environment tasks;
• tool / function-calling interactions;
• multi-language code editing and repair;
• repository issue resolution and test-driven patching;
• infrastructure, data, package, migration, and verifier-driven tasks.

The key change from the earlier pi-tune release is that reasoning was not stripped out of the training target. The model was trained to preserve the plan-and-act structure that showed up in successful trajectories, then hand off into concrete commands and edits.

Sample traces

A compact public sample of successful DeepSeek V4 Pro Pi reasoning teacher trajectories is available here:

bytkim/deepseek-v4-pro-pi-reasoning-sample-traces

The sample contains 70 pass-only trajectories across seven task providers, formatted as Qwen-style reasoning-target segments with Pi tool-use structure.

Performance profile

Representative local profile from the llama.cpp / Pi setup:

| Measure | Approximate value | What it means |

| --- | ---: | --- |

| Prompt / prefill | ~615 tok/s | Reading and processing prompt/context tokens. |

| Decode / generation | ~40 tok/s | Raw generated-token speed reported by llama.cpp. |

| End-to-end request | ~71 tok/s | Combined prompt + decode request throughput. |

| MTP draft acceptance | ~78% | Share of drafted tokens accepted by the main decode path. |

| Effective agent output | ~33 tok/s | Output tokens divided by full agent execution time. |

Throughput depends on hardware, context length, KV cache type, llama.cpp build, draft settings, and task shape. Treat these as a practical profile rather than a guarantee.

Recommended use cases

• Local coding-agent experiments.
• Tool-heavy chat and function-calling experiments.
• Repository navigation, patch planning, and test iteration.
• DevOps troubleshooting and runbook drafting.
• Long-context engineering workflows where local inference is preferred.
• Reasoning-heavy debugging, architecture review, data tasks, and verifier-guided repair loops.

Limitations

• This is a community research release, not a guaranteed drop-in replacement for the base model.
• All reported Terminal-Bench results are single-run pass@1.
• The base Pi run used a longer timeout than the reasoning run; timeout-sensitive comparisons should be interpreted carefully.
• Low-bit quantizations may reduce instruction following, tool-call reliability, and long-horizon task success.
• Reasoning mode emits extra tokens before final answers; use direct-response mode or the no-thinking sibling release when latency matters more than deliberation.
• The full training dataset is private; the linked public sample is provided to show representative trace format and task coverage, not to reproduce the full training mixture.
• The language fine-tune does not improve the inherited vision tower.
• Use normal caution for generated code, shell commands, dependency changes, and security-sensitive workflows.

License

Released under the Apache 2.0 license inherited from the upstream Qwen3.6-27B base model. You may use, modify, and redistribute the model and derivatives subject to that license.

Acknowledgements

Thanks to the Qwen team for releasing Qwen3.6-27B, to the llama.cpp / ggml community for local inference and MTP support, and to the open-source infrastructure around GGUF quantization and local model serving.

This work also benefited from the broader ecosystem around Terminal-Bench, Pi, OpenHands, Codex-style terminal agents, Hugging Face hosting, and trajectory-analysis tooling.

<div style="font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Roboto,sans-serif;border:1px solid #c7d2fe;border-radius:20px;overflow:hidden;background:#ffffff;margin:34px 0 8px 0;box-shadow:0 16px 42px -34px rgba(67,56,202,0.68);">

<div style="background:radial-gradient(circle at 10% 90%, rgba(45,212,191,0.48) 0%, rgba(45,212,191,0) 55%), linear-gradient(120deg,#0e7490 0%,#4338ca 100%);padding:26px 28px;color:white;">

<div style="font-size:11px;font-weight:800;letter-spacing:0.18em;text-transform:uppercase;color:#cffafe;margin-bottom:8px;">Built for local agent loops</div>

<div style="font-size:21px;font-weight:850;color:white;line-height:1.28;">Reasoning where it helps · tool use where it matters · GGUF for the terminal loop.</div>

<div style="display:flex;gap:18px;flex-wrap:wrap;margin-top:18px;">

<div>

<div style="font-size:24px;font-weight:850;color:#ffffff;line-height:1;">40.45%</div>

<div style="font-size:11px;color:#c7d2fe;margin-top:3px;letter-spacing:0.06em;">Pi Terminal-Bench 2.0 pass@1</div>

</div>

<div style="width:1px;background:rgba(255,255,255,0.18);"></div>

<div>

<div style="font-size:24px;font-weight:850;color:#ffffff;line-height:1;">Q4_K_M</div>

<div style="font-size:11px;color:#c7d2fe;margin-top:3px;letter-spacing:0.06em;">recommended start</div>

</div>

<div style="width:1px;background:rgba(255,255,255,0.18);"></div>

<div>

<div style="font-size:24px;font-weight:850;color:#ffffff;line-height:1;">MTP</div>

<div style="font-size:11px;color:#c7d2fe;margin-top:3px;letter-spacing:0.06em;">speculative decoding ready</div>

</div>

</div>

</div>

</div>