tomngdev/MTP-Qwen3.5-122B-A10B-GGUF overview
MTP Qwen3.5 122B A10B GGUF This repository contains MTP GGUF files only for Qwen/Qwen3.5 122B A10B https://huggingface.co/Qwen/Qwen3.5 122B A10B . To be used w…
Runs locally from ~2.99 GB disk (4 GB VRAM class GPUs with llama.cpp / guIDE).
Repository Files & Downloads
Model Details
Model README
---
license: apache-2.0
license_link: https://huggingface.co/Qwen/Qwen3.5-122B-A10B/blob/main/LICENSE
base_model:
- Qwen/Qwen3.5-122B-A10B
tags:
- mtp
- gguf
- qwen3_5_moe
- speculative-decoding
---
MTP-Qwen3.5-122B-A10B-GGUF
This repository contains MTP GGUF files only for Qwen/Qwen3.5-122B-A10B.
To be used with models without MTP included, such as mudler/Qwen3.5-122B-A10B-APEX-GGUF
It does not include the full base model. These files are intended to be used as the MTP / speculative decoding component together with the matching full model GGUF.
Q4_K_M and Q8_0 quants used unsloth/Qwen3.5-122B-A10B-MTP-GGUF for recipe.
Files
| File | Quant | Size |
|---|---:|---:|
| mtp-BF16.gguf | BF16 | 7.55 GiB |
| mtp-F16.gguf | F16 | 7.55 GiB |
| mtp-Q8_0.gguf | Q8_0 | 4.02 GiB |
| mtp-Q4_K_M.gguf (Recommended) | Q4_K_M | 2.99 GiB |
Usage
Example with llama-server:
llama-server \
-m /path/to/Qwen3.5-122B-A10B.gguf \
-md /path/to/mtp-Q4_K_M.gguf \
--spec-type draft-mtp \
--spec-draft-n-max 2Notes
- These files are not usable standalone.
- Use only with the matching base model.
- License follows the original base model license.
Run tomngdev/MTP-Qwen3.5-122B-A10B-GGUF with guIDE
Download guIDE — the AI-native code editor with local LLM inference and 69 built-in tools.
Source: Hugging Face · Compare models