---

license: gemma

base_model: google/diffusiongemma-26B-A4B-it

base_model_relation: quantized

pipeline_tag: text-generation

tags:

• gguf
• rocm
• rocmfp4
• amd
• strix-halo
• gfx1151
• diffusion-llm
• llama.cpp

---

DiffusionGemma-26B-A4B-it — ROCmFP4 / ROCmFPX GGUF

AMD-native FP4 / FP8 GGUF builds of google/diffusiongemma-26B-A4B-it for RDNA3.5 /

Strix Halo (gfx1151). MoE, 26B total / ~4B active, 128 experts / top-8.

This is a block-diffusion model: it denoises a 256-token canvas in parallel over N steps.

llama-server is the wrong binary — use llama-diffusion-cli / llama-diffusion-gemma-server.

Text-only; the source ships no mmproj and no MTP draft head, so neither is included here.

Variants

| file | ftype | size | actual steps | e2e decode |

|---|---|---|---|---|

| …-Q4_0_ROCMFP4_COHERENT.gguf | 102 | 13.46 GiB | 21 | 26.03 t/s |

| …-Q6_0_ROCMFPX_AGENT.gguf | 114 | 21.82 GiB | 18 | 21.08 t/s |

| …-Q8_0_ROCMFPX.gguf | 111 | 24.32 GiB | 16 | 37.85 t/s |

| …-Q8_0_ROCMFPX_AGENT.gguf | 115 | 24.68 GiB | 15 | 41.18 t/s |

All four: 3/3 correctness, [692/692] tensors, constant 15.09 MiB header delta (spread 5.6 KB).

Heads protected — tie_word_embeddings: true, so --output-tensor-type is a silent no-op and

--token-embedding-type is the only lever: q6_K on the 4-bit, q8_0 on the rest.

⭐ The 4-bit is NOT the fastest here — and that is the interesting part

On autoregressive models our 4-bit always wins, because decode is bandwidth-bound. Diffusion

denoises 256 positions at once, so each step is a large GEMM and the workload is compute-bound.

gfx1151 has no FP4 hardware path, so FP4 is unpacked per element in software — and at batch 256

that unpack costs more than the memory it saves.

A 24.68 GiB file runs a step faster than a 13.46 GiB one (414 ms vs 468 ms). Take the 8-bit

unless you are short on disk.

⭐⭐ Scheduler tuning is worth more than the quant choice

The step scheduler is entropy-bound: --diffusion-steps is ignored, and the model stops when

its own entropy budget is met. The stock budget stops conservatively. Loosening it:

--diffusion-eb-entropy-bound 0.8 --diffusion-eb-confidence 0.05

took a mainline Q8_0 build of these same weights from 45.3 → 53.56 t/s (12 steps x 398 ms,

median of 5, spread 1.012) with all quality gates still passing — 1.30x over the baseline above.

Adding -n 512 (two blocks) reached 55.3 t/s. Since e2e = steps x ms/step, removing steps beats

shaving milliseconds.

⛔ There is a quality cliff between 10 and 12 steps. At 8 steps the model reaches 78.5 t/s

and produces degenerate output ("list1111 *") — and the three factual gates still pass*,

because short factual answers converge early. Only a code-generation check caught it. **12 steps is

the floor for clean full-canvas output.* If you tune this, gate on generated prose or code*, not

on needle-in-haystack questions.

⚠️ Diffusion cost is decoupled from output length

"Say hi" still costs 9 steps / ~4 s, because the whole 256-token canvas is denoised

regardless. An autoregressive model emits that reply in a fraction of a second. Diffusion is

competitive only when you actually want a full canvas — compare it as a curve over output length,

never as a single ratio.

For reference, the autoregressive sibling of these weights

(kingjones777/Gemma-4-26B-A4B-it-ROCmFP4-GGUF), same box and same recipe, measures

54.39 t/s without a drafter.

Notes

• HIP works on ROCm 7.2.4 (ldd shows no Vulkan libraries linked). An older 0.73x figure for this

model came from a box whose rocBLAS could not load the gfx1151 Tensile library, forcing a Vulkan

fallback whose sampler ran on CPU — that number does not describe this build.

• Answers sometimes double-print (391391); long code replies can be cut mid-canvas when the

scheduler stops. Decoder behaviour, not quantization damage.

Credits

FP4/FP8 tensor types from the ROCmFPX fork of llama.cpp. These types do not exist in mainline

llama.cpp — a ROCmFPX-capable build is required to load them.