---

base_model: Qwen/Qwen3.6-27B

license: apache-2.0

language:

• en

library_name: gguf

tags:

• unsloth
• gguf
• llama.cpp
• qwen3_5
• code
• quantized

pipeline_tag: text-generation

datasets:

• trjxter/Kimi-K2.7-CodingTraces-9000x
• trjxter/Kimi-K2.6-Technical-Reasoning-AddOn-3300x
• trjxter/Kimi-K2.6-Reasoning-3300x-WandB
• Jackrong/Claude-opus-4.7-TraceInversion-5000x
• Jackrong/Kimi-K2.5-Reasoning-1M-Cleaned
• minpeter/xlam-function-calling-60k-parsed
• SWE-Factory/DeepSWE-Agent-Kimi-K2-Trajectories-2.8K
• nebius/SWE-agent-trajectories

---

Qwimi-3.6-27B-Coder-MTP-GGUF

<p align="center">

<img src="./27B.png" alt="Qwimi3.6-27B launch overview" width="100%">

</p>

GGUF quantizations of Qwimi-3.6-27B-Coder-MTP-BF16 — a coding-focused SFT of Qwen 3.6 27B (mixed

coding + tool-calling + agentic training; see Qwimi-3.6-27B-BF16 for the full model card,

training configuration, dataset composition, and eval results). This repo exists for

llama.cpp and downstream tools (Ollama, LM Studio, text-generation-webui, etc.).

Status: public repository. All 10 planned quants are uploaded and confirmed.

---

Contents

1. Source & conversion
2. Available quants
3. Which quant should I use?
4. Important: llama.cpp version requirement
5. How to run
6. Tool-calling with llama.cpp
7. Training summary
8. Known limitations
9. Benchmarks

---

1. Source & conversion

• Source: trjxter/Qwimi-3.6-27B-BF16 (merged full-precision model)
• Converted with: llama.cpp — convert_hf_to_gguf.py --outtype bf16, then

llama-quantize per target quant

• Build: CPU-only (GGML_CUDA=OFF); quantization is a CPU-bound process and does not

require GPU

• Conversion note: the merged repo's tokenizer_config.json originally carried a

transformers-v5-internal tokenizer_class value that AutoTokenizer (used internally by

the converter) could not resolve. This was corrected before conversion; if you reproduce

this conversion yourself from an older snapshot of the BF16 repo, verify

tokenizer_config.json reflects the base model's tokenizer class first.

2. Available quants

All sizes confirmed from the uploaded files (Hugging Face Hub hardware-compatibility widget,

computed from actual repo blobs). Total repo size: 175.0 GB across 10 quants.

| File | Type | Size | Status |

|---|---|---|---|

| Qwimi-3.6-27B-Q2_K_L.gguf | Q2_K (Q8_0 embed/output tensors) | 12.1 GB | ✅ confirmed |

| Qwimi-3.6-27B-Q3_K_S.gguf | Q3_K_S | 12.3 GB | ✅ confirmed |

| Qwimi-3.6-27B-Q3_K_M.gguf | Q3_K_M | 13.5 GB | ✅ confirmed |

| Qwimi-3.6-27B-Q3_K_L.gguf | Q3_K_L | 14.6 GB | ✅ confirmed |

| Qwimi-3.6-27B-Q4_K_S.gguf | Q4_K_S | 15.8 GB | ✅ confirmed |

| Qwimi-3.6-27B-Q4_K_M.gguf | Q4_K_M | 16.8 GB | ✅ confirmed |

| Qwimi-3.6-27B-Q5_K_S.gguf | Q5_K_S | 19.0 GB | ✅ confirmed |

| Qwimi-3.6-27B-Q5_K_M.gguf | Q5_K_M | 19.5 GB | ✅ confirmed |

| Qwimi-3.6-27B-Q6_K.gguf | Q6_K | 22.4 GB | ✅ confirmed |

| Qwimi-3.6-27B-Q8_0.gguf | Q8_0 | 29.0 GB | ✅ confirmed |

Q2_K_L naming note: this is not a standard llama-quantize type — it's the community

recipe of a Q2_K base with the token-embedding and output (LM head) tensors held at Q8_0

precision (--token-embedding-type Q8_0 --output-tensor-type Q8_0), which meaningfully

improves quality over plain Q2_K at a small size cost. All other quants above are standard

llama.cpp K-quant types.

Not produced: Unsloth "UD" (Dynamic 2.0) quants and sub-3-bit i-quants (IQ2_*,

IQ3_XXS) are not included — these require Unsloth's proprietary calibrated quantization

pipeline, not plain llama-quantize, and are out of scope for this repo.

3. Which quant should I use?

General guidance for this model family and size (27.8B):

• Best balance of quality/size: Q4_K_M — the standard recommended default for most use.
• Near-lossless, if you have the VRAM/RAM: Q8_0 (~30 GB) or Q6_K (~23 GB).
• Constrained VRAM: Q4_K_S or Q3_K_M/Q3_K_L — expect a noticeable but usually

acceptable quality drop versus Q4_K_M.

• Extreme constraint: Q2_K_L — usable, but at 2-bit the model's coding accuracy and

instruction-following will degrade more than the size savings suggest; only recommended if

no larger quant fits your hardware.

This model was trained and merged in BF16; no quant-specific fine-tuning was performed, so

relative quality ordering follows standard llama.cpp quantization behavior for models this

size — it has not been separately benchmarked per quant here (see §9).

4. Important: llama.cpp version requirement

Qwen3.6-27B (model_type: qwen3_5) is a hybrid architecture — standard attention layers

interleaved with GatedDeltaNet linear-attention layers. Support for this architecture in

llama.cpp and downstream tools (Ollama, LM Studio, text-generation-webui) depends on how

recently they've pulled in qwen3_5/GatedDeltaNet support upstream. **Verify your llama.cpp

build (or wrapping tool) explicitly supports this architecture before assuming compatibility**

— an outdated build may fail to load the model or silently misinterpret the linear-attention

layers. If you hit load errors, check for a llama.cpp update first.

5. How to run

# llama.cpp CLI
./llama-cli -m Qwimi-3.6-27B-Q4_K_M.gguf -p "Write a Python function to reverse a linked list." -n 512

# llama-server (OpenAI-compatible API)
./llama-server -m Qwimi-3.6-27B-Q4_K_M.gguf --ctx-size 16384

Set --ctx-size to at most 16384 — that is the sequence length this model was trained

and validated at, even though the base architecture supports a much longer native context.

Behavior beyond 16,384 tokens is unvalidated.

6. Tool-calling with llama.cpp

This model uses Qwen 3.6's native XML-style tool-call format (`<tool_call><function=...>

<parameter=...>`), not the JSON-object format from earlier Qwen versions — see the full

example in the Qwimi-3.6-27B-BF16 README §6. For llama-server to apply the model's chat

template (including tool definitions) correctly, you need a recent build with Jinja chat

template support enabled (commonly a --jinja flag) — verify against your installed

llama.cpp version's documentation, since flag names and defaults change across releases.

7. Training summary

Condensed from the full model card (Qwimi-3.6-27B-BF16) — see that repo for complete

details, dataset composition, and eval curves.

• Base: unsloth/Qwen3.6-27B, dense 27.8B, text-only SFT (vision frozen)
• Method: 4-bit QLoRA, r=64/alpha=64, language layers only
• Data: mixed single-run SFT — coding (82.5% of tokens) + tool-calling (4.8%) + agentic

SWE-agent trajectories (12.7%), 22,359 total training rows, 72.5M training tokens

• 1 epoch, effective batch 16, 1,398 optimizer steps, A100 80GB
• Final eval loss: coding 0.4761, tool 0.0208, agentic 0.2892 — all three domains improved

monotonically through the full run with no overfitting signal

8. Known limitations

See Qwimi-3.6-27B-BF16 §9 for the full list (vision untouched, thin agentic coverage, one

capped low-quality coding source, 3.6% of coding examples dropped for length rather than

truncated, 16,384-token validated context). Additionally for this GGUF repo specifically:

• Quantization introduces its own quality loss on top of the base model's characteristics,

increasing with lower bit-width — see §3 for guidance.

• This is a fresh/bleeding-edge architecture in llama.cpp (§4) — expect rougher edges in

tooling support than for long-established architectures.

9. Benchmarks

Qwimi was compared against the base Qwen 3.6 27B model using equivalent

Q6 GGUF builds with MTP enabled and the same evaluation harness.

Two benchmark suites were run:

1. A fully auto-scored 300-task custom benchmark covering coding, tool-calling, and

agentic workflows.

1. A broader 200-task mixed benchmark containing standard coding tasks and personal

Qwimi-oriented evaluations.

> Important: the latency figures below measure average end-to-end wall time per task,

> not raw tokens per second. SFT can reduce task completion time by producing shorter, more

> direct, better-formatted responses and reaching the stop condition sooner. The results do

> not prove that the underlying Q6 inference kernels became faster.

---

9.1 Custom 300-task benchmark

This benchmark contains 300 custom-authored, fully auto-scored tasks:

| Category | Tasks |

|---|---:|

| Coding | 150 |

| Tool-calling | 75 |

| Agentic workflows | 75 |

| Total | 300 |

Both models answered the same 300 tasks, producing 600 total answer rows.

Overall results

| Model | Tasks | Pass rate | Average wall time | Total wall time |

|---|---:|---:|---:|---:|

| Base Qwen 3.6 27B Q6 MTP | 300 | 81.33% | 53.56 s | 267.79 min |

| Qwimi 3.6 27B Coder Q6 MTP | 300 | 82.67% | 27.09 s | 135.45 min |

| Overall change | Result |

|---|---:|

| Absolute pass-rate gain | +1.33 percentage points |

| Relative pass-rate gain | +1.64% |

| Average wall-time reduction | 49.4% |

| Effective task-level speedup | ~1.98× |

| Total evaluation time saved | 132.34 minutes |

The overall result is a **small accuracy improvement combined with a very large reduction

in task-completion time**.

Category results

| Category | Base pass rate | Qwimi pass rate | Accuracy delta | Base avg time | Qwimi avg time | Wall-time reduction |

|---|---:|---:|---:|---:|---:|---:|

| Custom coding | 81.33% | 85.33% | +4.00 pts | 59.33 s | 21.16 s | 64.3% |

| Custom tool-calling | 88.00% | 89.33% | +1.33 pts | 31.77 s | 27.13 s | 14.6% |

| Custom agentic | 74.67% | 70.67% | -4.00 pts | 63.80 s | 38.92 s | 39.0% |

The strongest result was in custom coding, where Qwimi improved pass rate by four

percentage points while completing tasks with 64.3% lower average wall time.

Tool-calling showed a smaller but positive accuracy gain. Agentic accuracy regressed by four

points, although Qwimi still completed the agentic tasks substantially faster.

Pairwise outcomes

| Pairwise result | Tasks | Share |

|---|---:|---:|

| Qwimi passed and base failed | 23 | 7.67% |

| Base passed and Qwimi failed | 19 | 6.33% |

| Both passed | 225 | 75.00% |

| Both failed | 33 | 11.00% |

| Total | 300 | 100% |

Among the 42 decisive tasks where only one model passed:

| Decisive result | Count | Share |

|---|---:|---:|

| Qwimi wins | 23 | 54.8% |

| Base wins | 19 | 45.2% |

Category-level pairwise outcomes

| Category | Qwimi-only wins | Base-only wins | Both pass | Both fail |

|---|---:|---:|---:|---:|

| Coding | 11 | 5 | 117 | 17 |

| Tool-calling | 4 | 3 | 63 | 5 |

| Agentic | 8 | 11 | 45 | 11 |

| Total | 23 | 19 | 225 | 33 |

300-task verdict: Qwimi achieved a slight overall quality win, driven by stronger

coding and tool-calling performance, while also completing the suite in roughly half the

time. The main remaining weakness was agentic-task accuracy.

---

9.2 Mixed 200-task benchmark

A separate 200-task evaluation compared the same Q6 MTP builds across standard coding tasks

and personal Qwimi-oriented tests.

The run produced 400 raw answer rows, with 200 answers from each model. Some categories

were omitted from the scored summary because their scorer/export integration was incomplete;

they were not treated as zero-score failures.

Overall auto-scored results

| Model | Auto-scored rows | Passed | Pass rate | Average wall time |

|---|---:|---:|---:|---:|

| Base Qwen 3.6 27B Q6 MTP | 151 | 46 | 30.46% | 92.86 s |

| Qwimi 3.6 27B Coder Q6 MTP | 166 | 51 | 30.72% | 71.30 s |

| Overall change | Result |

|---|---:|

| Absolute pass-rate gain | +0.26 percentage points |

| Relative pass-rate gain | +0.85% |

| Average wall-time reduction | 23.2% |

Pairwise auto-scored results

On the 146 tasks with a valid paired auto-score:

| Pairwise result | Tasks | Share |

|---|---:|---:|

| Qwimi better | 7 | 4.79% |

| Base better | 2 | 1.37% |

| Same result | 137 | 93.84% |

| Total paired tasks | 146 | 100% |

Among the nine decisive paired tasks, Qwimi won:

\[

\frac{7}{9} = 77.8\%

\]

This corresponds to a net pairwise edge of +3.42 percentage points across all paired

auto-scored tasks.

Category highlights

| Category | Base pass rate | Qwimi pass rate | Delta |

|---|---:|---:|---:|

| HumanEval | 80.00% | 94.29% | +14.29 pts |

| Personal XML tool tasks | 50.00% | 100.00% | +50.00 pts |

| MBPP | 5.77% | 1.92% | -3.85 pts |

| Tool-calling checks | 100% | 100% | Tie |

| Architecture checks | 100% | 100% | Tie |

| Debugging checks | 100% | 100% | Tie |

200-task verdict: the result was directionally positive for Qwimi, particularly on

HumanEval, XML-style tool behavior, and wall time. However, the overall pass-rate gain was

small, most paired results were ties, and MBPP performance regressed.

---

9.3 Combined interpretation

Across both benchmark suites, Qwimi demonstrated:

• Higher overall pass rate in both runs.
• Stronger performance on custom coding, tool-calling, HumanEval, and personal

XML-tool tasks.

• A positive pairwise record:

- 23–19 on the 300-task custom benchmark.

- 7–2 on decisive paired tasks in the 200-task benchmark.

• Substantially lower task-completion time:

- 49.4% lower on the custom 300-task suite.

- 23.2% lower on the mixed 200-task suite.

• A remaining weakness in some areas:

- -4.00 points on custom agentic accuracy.

- -3.85 points on MBPP.

The fairest overall conclusion is:

> **Qwimi is a modest but measurable coding/tool-use improvement over the base model, with

> unusually large task-level latency reductions. Its largest quality gains appear in coding

> and structured tool usage, while agentic and MBPP performance require further work.**

---

9.4 Interpreting the speed results

Both evaluated models used Q6 quantization, so the speed difference should not be interpreted

as Qwimi having fundamentally faster matrix-multiplication kernels.

Autoregressive task time can be approximated as:

\[

T_{\text{task}}

\approx

T_{\text{prefill}}

+

N_{\text{generated tokens}}

\times

T_{\text{decode per token}}

\]

Because both models use the same architecture family and quantization level, their raw decode

cost per token should be broadly similar. SFT can still reduce total wall time by changing

generation behavior:

• reaching the correct solution with fewer tokens;
• producing less unnecessary explanation or repetition;
• following the expected output format sooner;
• avoiding failed attempts, malformed tool calls, or self-correction loops;
• emitting the stop token earlier.

Therefore, the supported claim is:

> Qwimi completed the benchmark tasks faster.

A separate controlled measurement of generated-token counts and tokens per second would be

required to claim that Qwimi is intrinsically faster on a per-token basis.

---

9.5 Benchmark limitations

• Only the Q6 GGUF builds were compared; these results should not be assumed to apply

identically to every quant listed in §2.

• Wall time depends on hardware, llama.cpp build, prompt length, output length, sampling

settings, MTP support, and stop conditions.

• The 300-task suite is custom-authored and is not a replacement for established public

benchmarks.

• Some categories in the 200-task run were excluded because their official scorer/export

integration was incomplete; they were not counted as zero.

• Agentic tasks used keyword/structural auto-scoring and may not capture every aspect of

repository-level software-engineering quality.

• More seeds, repeated runs, and token-level throughput measurements would improve confidence

in the latency conclusions.

<p align="center">

<img src="./B1-Qwimi27B.png" alt="Qwimi Benchmark overview" width="100%">

</p>

<p align="center">

<img src="./B2-Qwimi27B.png" alt="Qwimi benchmark overview" width="100%">

</p>