---

license: other

base_model: meta-models/Muse-Glimmer-30B

tags:

- gguf

- llama.cpp

- rocm

- rocmfp4

- strix-halo

- gfx1151

- speculative-decoding

- dflash

pipeline_tag: text-generation

---

Muse-Glimmer-30B — Q4_0_ROCMFP4_STRIX GGUF

A ROCmFP4 quantisation of Meta's Muse-Glimmer-30B, built for AMD Strix Halo

(gfx1151) unified-memory machines.

14.17 GiB @ 4.36 BPW — 22.6% smaller than Meta's K-Quant-Dynamic (18.30 GiB)

and smaller than their 17GB build (15.61 GiB), while measuring faster and holding

parity on a task smoke-check.

To my knowledge this is the first ROCmFP4 build of this model.

Why this needed building

ROCmFP4 quant types and Muse Glimmer support did not exist in the same tree:

• ROCmFP4 (ggml types 100–106) lives only in the ROCmFPX fork of llama.cpp,

which is based on an older upstream.

• Muse Glimmer support landed in current upstream llama.cpp.

Neither could produce this file alone. The model was ported forward into the

ROCmFPX base in three stages:

• Text graph, arch registration and converter, bridging three API gaps

(is_swa_impl → swa_layers, n_layer() from method to field, and the

NVFP4-only output-scale argument, which is null on the ROCmFP4 path).

• Vision tower, which additionally required teaching the older base's

build_vit to accept per-layer attention masks at all — it previously took no

mask parameter. Added as an overload so the ~32 other vision models calling it

are untouched. Those masks are Muse's sparse-window pattern (3 sparse layers

then 1 global, repeating); dropping them compiles cleanly and produces

confident, wrong image understanding, so they were ported rather than skipped.

• Chat-format parser, so harmony-style channel output is parsed instead of

leaking control tokens such as to=self<|message|> into content.

Conversion ran from the BF16 safetensors (1436 tensors) through the upstream

converter, producing a 55.7 GB BF16 GGUF (731 text tensors), then quantised with

the ported ROCmFPX quantiser.

Measurements

AMD Ryzen AI MAX+ 395 (Strix Halo, gfx1151), 128 GB unified memory, ROCm 7.2.4.

All arms: DFlash drafter, --spec-draft-n-max 15, greedy, batch 1, -c 8192.

Median of 9 generations per arm; speculative decoding swings run-to-run, so medians

are reported rather than means or best-of.

| build | size | tok/s (wall-clock) | tok/s (server tg) | smoke-check |

|---|---|---|---|---|

| Meta K-Quant-Dynamic | 18.30 GiB | 18.04 | 21.51 | 6/6 |

| This build (ROCmFP4-STRIX) | 14.17 GiB | 20.23 | 23.38 | 6/6 |

1.122× wall-clock, 1.087× generation-rate, 22.6% smaller.

Both arms ran on the same binary, so the delta is the quantisation rather than a

runtime difference. As a separate control, the K-Quant was also run on stock

upstream and scored identically (6/6, 18.21 tok/s) — confirming the forward-port

of the model graph does not alter output quality.

Workload sensitivity

Throughput on this model depends heavily on the task. Measured on the same build:

| workload | tok/s |

|---|---|

| code transform (edit/annotate an existing function) | 27.49 |

| open-ended explanation | 19.21 |

A single blended figure would misrepresent both, so both are given.

Speculative decoding

DFlash proposes a 16-token block per forward pass (15 speculative + 1 accepted),

and measured 4.17 tokens accepted per target pass (median, range 2.85–6.65)

at 21.5% per-token acceptance. Note that per-token acceptance is a misleading

figure for a block drafter — the tokens-per-pass number is the meaningful one.

For contrast, n-gram speculation was tested on the same build and is not a

substitute here: ngram-map-k reached a much higher 42.9% acceptance yet ran 45%

slower on code-transform work (15.20 vs 27.49 tok/s), because it proposes far fewer

tokens per pass.

Usage

Requires a llama.cpp built with ROCmFP4 support (ggml types 100–106); stock

llama.cpp will reject the tensor types.

llama-server \
  -m muse-glimmer-30B-ROCmFP4-STRIX.gguf \
  --mmproj mmproj-kquant.gguf \
  --spec-type draft-dflash \
  --model-draft dflash-kquant.gguf \
  --spec-draft-n-max 15 \
  -ngl 999 -fa off -dio --jinja \
  -c 32768 --host 127.0.0.1 --port 8080

Text, vision, and DFlash speculative decoding all run together in this

configuration. Drop --mmproj and switch to -fa on for a text-only server.

--spec-draft-n-max 15 is correct because the DFlash block size is 16, with one

position taken by the previously accepted token.

The DFlash drafter (dflash-kquant.gguf, 1.52 GiB) comes from Meta's official

GGUF release and is used unmodified.

Known limitations

Please read these before using this build — they are real and current.

1. Runtime-specific. ROCmFP4 is a runtime ggml type, not a portable container

format. This file only loads on a ROCmFP4-capable build. It cannot run under

Vulkan, CUDA, CPU, or stock llama.cpp.

1. Vision requires -fa off. Vision works and is verified correct, but flash

attention must be disabled: ggml_flash_attn_ext aborts on Muse's per-layer

sparse-window attention masks, which do not satisfy its mask shape/padding

constraints. The cost is negligible on text — measured 0.985× on code-transform

and 1.018× on open-ended work, both inside run-to-run noise. Use -fa on only

if you are running text-only.

1. Reasoning model — give it room. Output is split into reasoning_content

and content. With a small max_tokens the model spends the whole budget

deliberating and returns an EMPTY content, which looks like a broken build but

is not. Allow several hundred tokens.

1. Eval scope. The 6/6 figure is a smoke-check across code generation, SQL,

factual recall, arithmetic, and instruction-following, scored by substring

match. It is a regression guard against a broken quantisation, not a

benchmark suite. Vision was verified separately on images with unambiguous

ground truth, including a four-quadrant colour test that checks spatial

placement rather than mere colour naming — a misapplied attention mask names

colours confidently but puts them in the wrong corners.

Files

| file | size |

|---|---|

| muse-glimmer-30B-ROCmFP4-STRIX.gguf | 14.17 GiB |

Pair it with dflash-kquant.gguf and mmproj-kquant.gguf from Meta's official

GGUF repository.

Credit

Base model and the DFlash drafter are Meta's. ROCmFP4 quantisation types are from

the ROCmFPX fork of llama.cpp. This repository contains only the quantised weights

and the measurements above.