---

base_model:

• z-lab/Qwen3.5-9B-DFlash

license: apache-2.0

inference: false

tags:

- dflash

- speculative-decoding

- speculative-decoding-draft

- block-diffusion

- draft-model

- diffusion-language-model

- efficiency

- qwen

- qwen3

- qwen3.5

- llama.cpp

---

Quantized version of z-lab/Qwen3.5-9B-DFlash.

Works with mainline llama.cpp, will NOT work with forks.

Their model card is pasted as is below.

Files:

• model.safetensors - original unquantized safetensors
• model.gguf - unquantized bf16 GGUF
• Qwen3.5-9B-DFlash.gguf - GGUF quantized to Q5_K_M

---

Qwen3.5-9B-DFlash

Paper | Github | Blog

This DFlash draft model is a joint retrain from Z-Lab and Modal, trained with 40k sequence length and sliding-window attention for improved long-context performance. It is mirrored across the following Hugging Face repositories:

• z-lab/Qwen3.5-9B-DFlash
• modal-labs/Qwen3.5-9B-DFlash

This repository contains a DFlash draft model for Qwen/Qwen3.5-9B. It is not a standalone language model. It is intended to be paired with the target model in a speculative decoding server.

DFlash uses a lightweight block diffusion draft model to propose multiple tokens in parallel. The target model verifies those proposals, improving serving throughput while preserving the target model's output distribution.

<div align="center">

<img src="assets/dflash_system.png" alt="DFlash Architecture" width="85%">

</div>

Quick Start

Installation

SGLang

Install a recent SGLang build with DFlash support:

uv pip install --upgrade "sglang[all]"

For best performance on Blackwell GPUs, use an SGLang build that includes DFlash, FA4/TRT-LLM attention, and FlashInfer support.

vLLM

For vLLM support, please refer to vllm-project/vllm#40898. We will update the PR to make it merge-ready soon.

Launch Server

This model should be used with an inference server that supports DFlash speculative decoding. An example SGLang deployment is:

export SGLANG_ENABLE_OVERLAP_PLAN_STREAM=1

python -m sglang.launch_server \
  --model-path Qwen/Qwen3.5-9B \
  --trust-remote-code \
  --speculative-algorithm DFLASH \
  --speculative-draft-model-path z-lab/Qwen3.5-9B-DFlash \
  --speculative-dflash-block-size 8 \
  --speculative-draft-attention-backend fa4 \
  --attention-backend trtllm_mha \
  --linear-attn-prefill-backend flashinfer \
  --linear-attn-decode-backend flashinfer \
  --mamba-scheduler-strategy extra_buffer \
  --tp-size 1 \
  --max-running-requests 32 \
  --cuda-graph-max-bs-decode 32 \
  --cuda-graph-backend-prefill tc_piecewise \
  --enable-flashinfer-allreduce-fusion \
  --mem-fraction-static 0.8 \
  --host 0.0.0.0 \
  --port 30000

Block size 8 is the recommended default for higher-concurrency serving. Block size 16 gives longer accept lengths and strong concurrency-1 throughput in most workloads.

Benchmark Results

We benchmarked DFlash against the autoregressive baseline and Qwen's built-in MTP draft path. DFlash reaches up to 5.01x speedup at concurrency 1 and 2.58x at concurrency 32. Across the benchmark suite, DFlash delivers higher throughput than MTP at every matched setting where both completed.

Setup

• Runtime: SGLang on 1x NVIDIA B200 GPU, tensor parallel size 1, bfloat16
• Backends: trtllm_mha target attention, fa4 DFlash draft attention, flashinfer linear-attention prefill and decode
• Workloads: GSM8K, MATH500, HumanEval, MBPP, and MT-Bench with the Qwen chat template
• Decoding: greedy, thinking enabled, max output length 4096 tokens
• Measurement: 5 independent runs per configuration at concurrency 1 and 32 with continuous batching
• Throughput: generated output tokens / wall-clock benchmark time, including prefill and scheduling
• Accept length: completion_tokens / spec_verify_ct per generation turn, averaged across generation turns

Throughput and Speedup

Each cell is output tok/s (speedup). Bold marks the fastest speculative configuration in each row.

Concurrency 1

| Workload | Baseline | MTP steps=3 | DFlash block=4 | MTP steps=7 | DFlash block=8 | MTP steps=15 | DFlash block=16 |

| --- | --- | --- | --- | --- | --- | --- | --- |

| gsm8k | 245.2 (1.00x) | 537.7 (2.19x) | 609.8 (2.49x) | 573.4 (2.34x) | 890.7 (3.63x) | 435.5 (1.78x) | 1027.3 (4.19x) |

| math500 | 244.6 (1.00x) | 558.3 (2.28x) | 636.3 (2.60x) | 617.3 (2.52x) | 987.6 (4.04x) | 485.3 (1.98x) | 1225.7 (5.01x) |

| humaneval | 243.4 (1.00x) | 537.6 (2.21x) | 633.2 (2.60x) | 586.9 (2.41x) | 959.5 (3.94x) | 447.6 (1.84x) | 1195.8 (4.91x) |

| mbpp | 244.7 (1.00x) | 526.4 (2.15x) | 624.4 (2.55x) | 543.6 (2.22x) | 935.7 (3.82x) | 403.6 (1.65x) | 1092.6 (4.46x) |

| mt-bench | 243.7 (1.00x) | 501.7 (2.06x) | 560.2 (2.30x) | 494.6 (2.03x) | 757.5 (3.11x) | 368.0 (1.51x) | 834.3 (3.42x) |

Concurrency 32

| Workload | Baseline | MTP steps=3 | DFlash block=4 | MTP steps=7 | DFlash block=8 | MTP steps=15 | DFlash block=16 |

| --- | --- | --- | --- | --- | --- | --- | --- |

| gsm8k | 5837.6 (1.00x) | 10421.0 (1.79x) | 11882.9 (2.04x) | 10132.5 (1.74x) | 13718.0 (2.35x) | 8332.8 (1.43x) | 11019.7 (1.89x) |

| math500 | 5885.7 (1.00x) | 11124.9 (1.89x) | 12534.0 (2.13x) | 11213.3 (1.91x) | 15198.3 (2.58x) | 7785.0 (1.32x) | 13227.4 (2.25x) |

| humaneval | 5513.0 (1.00x) | 9645.9 (1.75x) | 11882.7 (2.16x) | 9701.0 (1.76x) | 14229.9 (2.58x) | 6901.7 (1.25x) | 12406.1 (2.25x) |

| mbpp | 5538.8 (1.00x) | 9116.4 (1.65x) | 11561.2 (2.09x) | 8701.6 (1.57x) | 13460.3 (2.43x) | 6220.5 (1.12x) | 11338.9 (2.05x) |

| mt-bench | 5491.7 (1.00x) | 9135.0 (1.66x) | 10072.2 (1.83x) | 8436.9 (1.54x) | 10718.2 (1.95x) | 5917.8 (1.08x) | 8495.7 (1.55x) |

Accept Length

Mean accept length at concurrency 1. Bold marks the higher value in each matched MTP/DFlash pair.

| Workload | MTP steps=3 | DFlash block=4 | MTP steps=7 | DFlash block=8 | MTP steps=15 | DFlash block=16 |

| --- | --- | --- | --- | --- | --- | --- |

| gsm8k | 3.464 | 3.452 | 5.276 | 5.400 | 6.388 | 6.949 |

| math500 | 3.541 | 3.555 | 5.466 | 5.757 | 6.728 | 7.721 |

| humaneval | 3.493 | 3.571 | 5.326 | 5.798 | 6.399 | 7.927 |

| mbpp | 3.338 | 3.454 | 4.790 | 5.376 | 5.508 | 6.820 |

| mt-bench | 3.229 | 3.193 | 4.551 | 4.606 | 5.438 | 5.716 |

Citation

If you find DFlash useful, please cite the original paper:

@article{chen2026dflash,
  title   = {{DFlash: Block Diffusion for Flash Speculative Decoding}},
  author  = {Chen, Jian and Liang, Yesheng and Liu, Zhijian},
  journal = {arXiv preprint arXiv:2602.06036},
  year    = {2026}
}