---

license: apache-2.0

base_model: Qwen/Qwen3.5-2B

pipeline_tag: image-text-to-text

library_name: gguf

tags:

- gguf

- qwen3.5

- lm-studio

- mtp

- speculative-decoding

- cuda

- windows

---

Qwen3.5-2B MTP fast

Fast MTP profiles for Bartowski's Qwen3.5-2B Q4_K_M GGUF in LM Studio and llama.cpp Vulkan.

The GGUF weights are unchanged. The speed-up is enabled by the included LM Studio MTP load profile.

Files

| File | Description |

|---|---|

| Qwen_Qwen3.5-2B-Q4_K_M.gguf | Q4_K_M model with built-in MTP heads |

| mmproj-Qwen_Qwen3.5-2B-f16.gguf | Vision projector |

| load_profile.ps1 | LM Studio baseline/fast profile loader |

| benchmark-summary.json | Machine-readable benchmark summary |

| benchmark-summary.csv | Compact context-speed table |

| cloudpersonal-benchmark-summary.json | Strix Halo server benchmark summary |

| cloudpersonal-benchmark.csv | Strix Halo context-speed table |

LM Studio fast profile

• LM Studio 0.4.20 or newer
• context: up to 262,144 tokens
• parallel: 1
• Flash Attention: on
• KV cache GPU offload: on
• MTP: on
• maximum draft tokens: 2
• minimum continue probability: 0.20

Start LM Studio's local server on port 1234, then run:

powershell.exe -NoProfile -ExecutionPolicy Bypass -File .\load_profile.ps1 -Profile ornith -ContextLength 262144

RTX 4060 Laptop measurements

Mean decode speed, three runs per context, 128 generated tokens:

| Prompt context | MTP off | MTP fast | Change |

|---:|---:|---:|---:|

| 1K | 128.86 tok/s | 177.92 tok/s | +38.08% |

| 4K | 126.74 tok/s | 175.16 tok/s | +38.20% |

| 8K | 122.89 tok/s | 172.74 tok/s | +40.57% |

| 16K | 117.35 tok/s | 165.21 tok/s | +40.78% |

| 32K | 106.28 tok/s | 148.93 tok/s | +40.13% |

| 64K | 88.71 tok/s | 132.06 tok/s | +48.86% |

| 128K | 67.85 tok/s | 105.60 tok/s | +55.64% |

| 250K | 48.27 tok/s | 76.82 tok/s | +59.14% |

• maximum single run: 179.09 tok/s
• maximum repeated mean: 177.92 tok/s
• geometric mean speed-up over all contexts: 1.4497x
• maximum loaded context: 262,144 tokens

Test system: Windows 11, Intel Core i7-13650HX, 63.74 GiB RAM, NVIDIA RTX 4060 Laptop 8 GiB, LM Studio 0.4.20, llama.cpp CUDA runtime 2.27.1.

Ryzen AI MAX+ 395 / Radeon 8060S server measurements

Direct llama.cpp b9994 Vulkan A/B on the same GGUF and prompts. The server used a 262,144-token window, Q8_0 K/V cache, Flash Attention, full GPU offload, parallel 1, and three runs per context. Other resident AI services were idle during the test.

| Prompt context | MTP off | MTP fast | Change |

|---:|---:|---:|---:|

| 1K | 125.68 tok/s | 203.82 tok/s | +62.18% |

| 4K | 123.79 tok/s | 197.10 tok/s | +59.22% |

| 8K | 121.23 tok/s | 188.57 tok/s | +55.55% |

| 16K | 115.64 tok/s | 180.21 tok/s | +55.84% |

| 32K | 108.47 tok/s | 159.48 tok/s | +47.02% |

| 64K | 94.00 tok/s | 131.15 tok/s | +39.52% |

| 128K | 75.76 tok/s | 95.64 tok/s | +26.24% |

| 250K | 56.06 tok/s | 63.67 tok/s | +13.59% |

• maximum single run: 205.20 tok/s
• maximum repeated mean: 203.82 tok/s
• geometric mean speed-up over all contexts: 1.4393x
• deterministic output hashes matched at every context

Server: AMD Ryzen AI MAX+ 395 with Radeon 8060S, 128 GiB UMA, Ubuntu, llama.cpp b9994 (14d3ba45f), Vulkan.

Credits

• Base model: Qwen/Qwen3.5-2B
• GGUF quantization: bartowski/Qwen_Qwen3.5-2B-GGUF