---

license: apache-2.0

base_model: z-lab/Qwen3.8-27B-DFlash2

base_model_relation: quantized

pipeline_tag: text-generation

tags:

- gguf

- dflash2

- speculative-decoding

- draft-model

- rocmfp4

- vulkan

- strix-halo

---

Qwen3.8-27B DFlash2 draft model, ROCmFP4-FAST GGUF

A 4.25 bpw ROCmFP4 requantisation of z-lab/Qwen3.8-27B-DFlash2,

for use as a speculative decoding sidecar with a Qwen3.8-27B target.

Measured on an AMD Strix Halo (Radeon 8060S), paired with

julianmb/Qwen-3.8-27B-ROCmFP4-FAST-GGUF:

65.6 t/s on structured output, 4.7x bare decode.

Two things to know before you download

1. This will not load in stock llama.cpp. Q4_0_ROCMFP4_FAST is GGUF file type 103, which

exists only in the fork linked below. Upstream llama.cpp fails with

failed to load model before it reaches the GPU. Get a prebuilt binary here:

• https://github.com/LaurentZuijdwijk/llama.cpp/releases

2. This is a draft model, not a standalone one. It has no full graph of its own. Loading it by

itself fails with failed to create context, which is expected. It must be passed with -md

alongside a target model.

Usage

llama-server \
  -m  Qwen3.8-27B-ROCmFP4-FAST.gguf \
  -md Qwen3.8-27B-DFlash2-Q4_0_ROCMFP4_FAST.gguf \
  --spec-type draft-dflash --spec-draft-adaptive \
  --spec-draft-n-min 3 --spec-draft-n-max 7 --spec-draft-ngl 99 \
  -ngl 999 -fa on -b 2048 -ub 512 -c 32768

--spec-draft-adaptive sizes the draft from measured acceptance instead of a fixed n. It is

what makes n-max 7 safe here: at a fixed n=7 acceptance collapses to 18 % and throughput

drops to 20.2 t/s, while adaptive holds 96 % acceptance and reaches 65.6.

| Qwen3.8-27B, FP4 target + this sidecar, greedy, 300 tokens | structured output | prose |

|---|---|---|

| bare decode | 14.0 t/s | 14.1 t/s |

| fixed draft n=3 | 41.6 | 25.4 |

| fixed draft n=7 | 20.2 | 24.8 |

| adaptive, n_max 7 n_min 3 | 65.6 t/s - 4.7x | 26.1 |

Speculative decoding raises throughput, not quality: the target model verifies every token, so

output matches what the target would have produced on its own.

Requirements

• The fork above, or any build with the ROCmFPx quant types
• A Vulkan 1.3 driver. Measured on Mesa RADV 26.0.8 on gfx1151.
• The FP4 path is tuned for AMD Strix Halo. It should run anywhere the fork builds, but the

numbers above are specific to this hardware.

A note on requantising

Do not requantise a DFlash2 sidecar to Q8_0_ROCMFPX expecting parity with Q8_0. At identical

bits per weight, ours scored 53.5 % acceptance against z-lab's 60.2 %. The cause is the block

scale, not the codes: Q8_0 stores an fp16 scale, ROCmFPx stores a UE4M3 byte. At 8 bits per

weight the coarse scale is what costs you. FP4 is a different trade and is the one measured here.

Credits

• z-lab/Qwen3.8-27B-DFlash2, the source model (Apache-2.0)
• ciru-ai/ROCmFPX for the FPx quant formats
• DFlash2 support in llama.cpp, PR #27342 by Jian Chen
• ggml-org/llama.cpp

Quantized and published by Agention.

Licensed Apache-2.0, inherited from the base model.