---

license: apache-2.0

language:

- en

- zh

base_model:

- Qwen/Qwen3.5-122B-A10B

library_name: llama.cpp

tags:

- gguf

- qwen

- qwen3.5

- moe

- mxfp4

- mtp

- speculative-decoding

- llama.cpp

- vulkan

- amd

pipeline_tag: text-generation

---

Qwen3.5-122B-A10B MXFP4_MOE GGUF with MTP Donor

This is a community-merged GGUF build of Qwen3.5-122B-A10B MXFP4_MOE with an injected MTP donor layer for llama.cpp MTP speculative decoding.

This repository is not an official release from Qwen, Unsloth, or LaikaFramework.

> This is an experimental community merge. Please report loading issues, MTP compatibility problems, or runtime results in the Discussions tab.

What this repository contains

This repository provides a merged GGUF file:

Qwen3.5-122B-A10B-MXFP4_MOE-MTP-merged.gguf

The model is intended for testing with a custom llama.cpp build that supports MTP speculative decoding.

Sources and attribution

This file was produced from the following sources:

• Original base model: Qwen/Qwen3.5-122B-A10B
• Base GGUF quantization: unsloth/Qwen3.5-122B-A10B-GGUF, MXFP4_MOE
• MTP donor: LaikaFramework/Qwen3.5-122B-A3B-MTP-Donor
• Merge method: convert.py merge <single-base.gguf> <donor.gguf> <output.gguf>
• Runtime: custom llama.cpp build with MTP support, PR #22673 style

Why this merge exists

At the time of testing, complete Qwen3.5-122B-A10B GGUF files with working MTP support were not widely available.

This merge adds the missing MTP/NextN layer metadata and tensors to an existing MXFP4_MOE GGUF base.

The key metadata expected after merge is:

general.architecture = qwen35moe
qwen35moe.block_count = 49
qwen35moe.nextn_predict_layers = 1

Tested environment

Local test environment:

OS: Ubuntu 24.04
Hardware: AMD Ryzen AI Max+ 395 / Radeon 8060S
Memory: 128GB unified memory
Backend: llama.cpp Vulkan / RADV
Context tested: 262144
MTP setting: --spec-type mtp --spec-draft-n-max 2

Local test results

On the above AMD/Vulkan system, the model successfully loaded as:

qwen35moe_mtp

Observed local performance:

1200-token test:
  decode speed: around 31 tok/s
  MTP draft acceptance: around 72%

3000-token test:
  decode speed: around 31 tok/s
  MTP draft acceptance: around 76%

These numbers are only local observations. Your performance may vary significantly depending on backend, hardware, memory bandwidth, context size, and llama.cpp version.

Recommended llama-server command

Example command:

LLAMA="/path/to/llama-server"
MODEL="/path/to/Qwen3.5-122B-A10B-MXFP4_MOE-MTP-merged.gguf"

"$LLAMA" \
  -m "$MODEL" \
  --host 0.0.0.0 \
  --port 1236 \
  --alias qwen35-122b-mxfp4-mtp \
  -ngl 999 \
  -c 262144 \
  -np 1 \
  -b 1024 \
  -ub 256 \
  --cache-type-k q8_0 \
  --cache-type-v q8_0 \
  --jinja \
  --reasoning off \
  --reasoning-format deepseek \
  --temp 0.3 \
  --top-p 0.95 \
  --top-k 20 \
  --spec-type mtp \
  --spec-draft-n-max 2

Low-memory KV option

For lower KV cache memory usage at very long context, you may test:

--cache-type-k q4_0 \
--cache-type-v q4_0

In local testing, q4 KV reduced memory pressure and remained usable, but q8 KV is recommended as the default quality-first configuration.

Important notes

• This model requires a llama.cpp build with MTP support.
• This model is not guaranteed to work with stock llama.cpp releases that do not support MTP.
• This file is a community merge and should be treated as experimental.
• Do not use this repository as evidence of official Qwen, Unsloth, or LaikaFramework support.
• MTP speculative decoding primarily improves decode/generation speed, not prefill speed.
• Very long context still requires substantial KV cache memory.

Verification

A SHA256 checksum is included in SHA256SUMS.

To verify after download:

sha256sum -c SHA256SUMS

License

The upstream Qwen model is released under Apache-2.0.

This repository follows Apache-2.0 and retains attribution to the upstream projects.

Please also review and respect the license terms of the upstream model and source repositories.