---

license: apache-2.0

base_model: Qwen/Qwen3.6-27B

tags:

• gguf
• quantized
• imatrix
• ik_llama.cpp
• mtp
• speculative-decoding

language:

• en

pipeline_tag: text-generation

---

Qwen3.6-27B — vram14: an all-VRAM dense quant with native MTP (14.1 GB, imatrix)

A dense 27B that fits entirely in 18 GB of VRAM, keeps its native multi-token-prediction head, and measures statistically lossless against its Q8 source.

Qwen3.6-27B ships with a trained MTP head (nextn_predict_layers=1) that most quants throw away. This build keeps it: with -mtp in ik_llama.cpp you get lossless self-speculative decoding — +58 % decode on code in our tests, from a head that was actually trained for these weights.

File

| File | Size | ctx |

|---|---|---|

| Qwen3.6-27B-vram14-imat.gguf | 14.1 GB | 64 K+ all-VRAM on 18 GB (more with spare VRAM) |

Quality (perplexity, held-out corpus, ctx 512)

| Quant | Size | PPL |

|---|---|---|

| Q8_K_XL (source) | 35.8 GB | 2.5207 ± 0.073 |

| vram14 (this repo) | 14.1 GB | 2.4955 ± 0.069 |

The quant scores at its source's ceiling (inside the error bars) at 39 % of the size — the imatrix + selective-precision recipe is effectively lossless here.

Measured speed (RTX 3060 Ti 8 GB + RTX 3080 10 GB, ik_llama.cpp, q4_0 KV, all-VRAM)

| | decode | prefill @14.5 K |

|---|---|---|

| code (with -mtp) | 52 t/s | 670 t/s |

| prose | 33 t/s | — |

Dense models read every weight per token, so decode scales directly with VRAM bandwidth — a single 3090/4090-class card will be substantially faster than this dual-mid-range split.

Recipe

Requantized from Unsloth's UD-Q8_K_XL with an imatrix (61 K calibration tokens, mixed code/prose):

• Base IQ4_XS
• ffn_up / ffn_gate → IQ3_S (the size lever; imatrix-guided)
• output tensor + token embeddings → q6_K
• MTP/nextn layer preserved intact

Built with ik_llama.cpp llama-quantize (--ffn-up-type iq3_s --ffn-gate-type iq3_s --output-tensor-type q6_K --token-embedding-type q6_K).

Running

ik_llama.cpp required for -mtp; the file loads in mainline llama.cpp without it (head ignored).

llama-server -m Qwen3.6-27B-vram14-imat.gguf \
  --jinja -fa on --cache-type-k q4_0 --cache-type-v q4_0 \
  --ctx-size 65536 -ngl 99 --tensor-split 46,54 --ubatch-size 256 --no-mmap \
  -mtp --ctx-size-draft 8192

Single ≥16 GB GPU: drop --tensor-split. Qwen3.6 is a reasoning model — thinking goes to reasoning_content; disable with "chat_template_kwargs": {"enable_thinking": false}.

When to pick this vs an MoE

On the same held-out corpus, Qwen3.6-35B-A3B (MoE) quantized to a similar all-VRAM size scored better (2.43) and decoded 3× faster — MoE wins when total VRAM is the constraint. This dense build is for when you specifically want dense-model behavior (no routing variance, stronger per-token compute) or have the bandwidth to feed it.

---

Built on a Ryzen 5950X + 3060 Ti + 3080. Base model © Alibaba Cloud, Apache-2.0. Quantized with ik_llama.cpp tooling.