---

license: apache-2.0

base_model:

• Jackrong/Qwopus3.6-27B-v2-MTP-GGUF

tags:

• gguf
• llama.cpp
• magicquant
• hybrid-quantization
• mtp
• multi-token-prediction
• speculative-decoding
• qwen3_6
• conversational

library_name: llama.cpp

pipeline_tag: text-generation

quantized_by: MagicQuant

language:

• en

base_model_relation: quantized

---

Qwopus3.6-27B-v2-MTP — MagicQuant hybrid GGUFs (Q4 / Q5 / Q6)

Per-tensor-group hybrid quantizations of

Jackrong/Qwopus3.6-27B-v2-MTP-GGUF

found by MagicQuant evolutionary search — a measured Predict→Measure→Learn loop

(real perplexity + KL-divergence guard on every surviving candidate, imatrix-weighted

encoding, stream-aware sampling, and a bytes-per-token generation-speed objective).

Each tier's per-group precision layout was selected from 12 fully measured candidate

hybrids rather than applied uniformly.

All files retain the model's embedded MTP (multi-token-prediction) head, so they

work with llama.cpp speculative decoding using the same file as its own draft model.

Files

| File | Size | PPL (wikitext-2) | tg (t/s)* | Layout (group → quant) |

|---|---|---|---|---|

| Qwopus3.6-27B-v2-MTP-GGUF-Q4_K_M.gguf | 14.65 GiB | 6.6596 | 11.52 | attn/FFN/embed Q4_K_M, head Q6_K |

| Qwopus3.6-27B-v2-MTP-GGUF-Q5_K_M.gguf | 20.89 GiB | 6.6314 | 8.05 | uniform Q6_K (search winner for the tier) |

| Qwopus3.6-27B-v2-MTP-GGUF-Q6_K.gguf | 23.14 GiB | 6.6235 | 7.74 | embed Q8_0, head F16, attn-Q MXFP4, K/O Q8_0, FFN-up Q6_K, FFN-down Q5_K, SSM F16 |

Reference points, measured identically (100 chunks, ctx 512, same box/build):

BF16 baseline PPL 6.571 · stock Q4_K_M from the base repo: 15.65 GiB, PPL 6.7022, tg 10.85.

• The Q4 file beats the stock Q4_K_M on quality, size, and speed simultaneously.
• Q5 / Q6 cut the stock file's quantization loss (vs BF16) by 54% / 60%.

\* tg/pp measured with llama-bench -p 128 -n 128 -r 3 -ngl 99 -fa 1 on an AMD Strix Halo

APU (Radeon 8060S iGPU, gfx1151, unified memory). Generation is memory-bandwidth-bound:

expect tg to scale roughly with file size on your hardware.

MTP speculative decoding (~1.5× generation, measured)

The embedded nextn/MTP head is preserved in every file. Serve with the model as its own draft:

llama-server -m Qwopus3.6-27B-v2-MTP-GGUF-Q4_K_M.gguf \
  -md Qwopus3.6-27B-v2-MTP-GGUF-Q4_K_M.gguf --spec-type draft-mtp \
  -c 8192 -ngl 99 -fa on -ctk q8_0 -ctv q8_0

Measured on the Q4 file, same prompt/seed with and without MTP: **10.14 → 14.82 t/s

(1.46×)** at 81% draft acceptance.

> Requires a llama.cpp build with qwen3_6/qwen35 arch + MTP (--spec-type draft-mtp)

> support (recent master).

Notes

• Chat template is embedded (no patching needed).
• Group legend: E=embeddings, H=lm head, Q/K/O=attention, U/D=FFN up/down, S=SSM (mamba) ops.
• SSM conv/1D tensors are kept at F32 where llama.cpp kernels require it; 48-wide mamba rows

fall back to F32 (block-size constraint of this hybrid architecture).

• Sibling repo with AMD-native (ROCmFPX fork-only) builds of the same layouts:

lmcoleman/Qwopus3.6-27B-v2-MTP-ROCmFPX-GGUF.

Built with Foundry + MagicQuant (evolutionary per-tensor-group hybrid quantization;

measured search with imatrix + KL guard).

Vision (image input)

mmproj-F32.gguf is copied unmodified from

Jackrong/Qwopus3.6-27B-v2-MTP-GGUF — pair it with any text

quant here for image input:

llama-server -m Qwopus3.6-27B-v2-MTP-GGUF-Q5_K_M.gguf --mmproj mmproj-F32.gguf -c 8192 --port 8080