---

license: other

base_model: ManniX-ITA/Qwen3.6-27B-Omnimerge-v4-MTP-GGUF

tags:

• gguf
• qwen
• qwen3.6
• mtp
• llama.cpp

---

Qwen3.6-27B-Omnimerge-v4-MTP IQ3_XXS GGUF

IQ3_XXS quantization of ManniX-ITA/Qwen3.6-27B-Omnimerge-v4-MTP-GGUF, generated with llama.cpp using the original imatrix.dat.

Quantization

Source

• Qwen3.6-27B-Omnimerge-v4-MTP-F16.gguf

Output

• Qwen3.6-27B-Omnimerge-v4-MTP-IQ3_XXS.gguf

• The MTP tensors were preserved at Q4_K using tensor-specific quantization overrides due to reasons as documented in the original model.

llama-quantize \
  --imatrix imatrix.dat \
  --tensor-type blk.64.attn_k.weight=q4_K \
  --tensor-type blk.64.attn_q.weight=q4_K \
  --tensor-type blk.64.attn_output.weight=q4_K \
  --tensor-type blk.64.ffn_down.weight=q4_K \
  --tensor-type blk.64.ffn_gate.weight=q4_K \
  --tensor-type blk.64.ffn_up.weight=q4_K \
  --tensor-type blk.64.nextn.eh_proj.weight=q4_K \
  Qwen3.6-27B-Omnimerge-v4-MTP-F16.gguf \
  Qwen3.6-27B-Omnimerge-v4-MTP-IQ3_XXS.gguf \
  IQ3_XXS

Running with MTP

llama-server \
  -m Qwen3.6-27B-Omnimerge-v4-MTP-IQ3_XXS.gguf \
  --spec-type draft-mtp \
  --spec-draft-n-max 2

HumanEval

Evaluated using:

• lm-evaluation-harness
• Raw OpenAI-compatible /v1/completions
• Temperature = 0
• Model-specific tokenizer (ManniX-ITA/Qwen3.6-27B-Omnimerge-v4)
• llama.cpp CUDA build b9632

| Model | Tokenizer | HumanEval pass@1 | Size |

|---|---|---:|---:|

| Qwen3.6-27B-Omnimerge-v4-MTP-IQ3_XXS | ManniX-ITA/Qwen3.6-27B-Omnimerge-v4 | 80.49% ± 3.10% | 10.6 GB |

| ggufbench/Qwen3.6-27B-4bpw-16GB-VRAM | unsloth/Qwen3.6-27B | 81.10% ± 3.07% | 12.58 GB |

| unsloth/Qwen3.6-27B-MTP-GGUF:IQ2_M | unsloth/Qwen3.6-27B | 71.95% ± 3.52% | 10.2 GB |

The IQ3_XXS Omnimerge quant is within approximately one HumanEval problem of the larger 4bpw/16GB-VRAM quant in this setup, while remaining much closer in size to the 10GB-class IQ2_M quant.

Evaluation Command

python -m lm_eval \
  --model local-completions \
  --model_args "model=Qwen3.6-27B-Omnimerge-v4-MTP-IQ3_XXS,base_url=http://127.0.0.1:1234/v1/completions,tokenizer=ManniX-ITA/Qwen3.6-27B-Omnimerge-v4,tokenizer_backend=huggingface" \
  --tasks humaneval \
  --batch_size 1 \
  --confirm_run_unsafe_code

Test Hardware

• Google Colab
• NVIDIA L4 (24 GB VRAM)
• Context length: 65,536
• MTP enabled

Notes

• MTP functionality is preserved.
• Code-completion benchmarks such as HumanEval should use raw /v1/completions rather than chat-completions.
• Benchmark results are sensitive to tokenizer selection; use the Omnimerge tokenizer for reproducible results.

Credits

• Base model: ManniX-ITA
• Quantization: kairusama
• Runtime: llama.cpp