tomngdev/MTP-Qwen3.5-35B-A3B-GGUF overview
MTP Qwen3.5 35B A3B GGUF This repository contains MTP GGUF files only for Qwen/Qwen3.5 35B A3B https://huggingface.co/Qwen/Qwen3.5 35B A3B . To be used with mo…
Runs locally from ~1.51 GB disk (4 GB VRAM class GPUs with llama.cpp / guIDE).
Repository Files & Downloads
Model Details
Model README
---
license: apache-2.0
license_link: https://huggingface.co/Qwen/Qwen3.5-35B-A3B/blob/main/LICENSE
base_model:
- Qwen/Qwen3.5-35B-A3B
tags:
- mtp
- gguf
- qwen3_5_moe
- speculative-decoding
---
MTP-Qwen3.5-35B-A3B-GGUF
This repository contains MTP GGUF files only for Qwen/Qwen3.5-35B-A3B.
To be used with models without MTP included, such as deepreinforce-ai/Ornith-1.0-35B-GGUF
It does not include the full base model. These files are intended to be used as the MTP / speculative decoding component together with the matching full model GGUF.
Q4_K_M and Q8_0 quants used unsloth/Qwen3.5-35B-A3B-MTP-GGUF for recipe.
Files
| File | Quant | Size |
|---|---:|---:|
| mtp-BF16.gguf | BF16 | 3.48 GiB |
| mtp-F16.gguf | F16 | 3.48 GiB |
| mtp-Q8_0.gguf | Q8_0 | 1.85 GiB |
| mtp-Q4_K_M.gguf (Recommended) | Q4_K_M | 1.51 GiB |
Usage
Example with llama-server:
llama-server \
-m /path/to/Qwen3.5-35B-A3B.gguf \
-md /path/to/mtp-Q4_K_M.gguf \
--spec-type draft-mtp \
--spec-draft-n-max 2Notes
- These files are not usable standalone.
- Use only with the matching base model.
- License follows the original base model license.
Run tomngdev/MTP-Qwen3.5-35B-A3B-GGUF with guIDE
Download guIDE — the AI-native code editor with local LLM inference and 69 built-in tools.
Source: Hugging Face · Compare models