---

license: other

base_model: meta-models/Muse-Glimmer-30B

base_model_relation: quantized

pipeline_tag: text-generation

library_name: gguf

tags:

- gguf

- llama.cpp

- rocm

- amd

- rocmfp4

- rocmfpx

- strix-halo

- amd-strix-halo

- gfx1151

- ryzen-ai-max

- ryzen-ai-max-395

- radeon-8060s

- dflash

- speculative-decoding

- reasoning

- tool-calling

- multimodal

- vision

- quantized

---

> ### ⚠️ PATCH CORRECTED — 2026-08-16

>

> The patch previously published here was incomplete: 9 files, text-only. It was missing the

> vision projector, the chat-template parser, and the conversion scripts. If you used it you would

> have seen unknown projector type: muse-glimmer, a failing mmproj conversion, and

> to=self<|message|> leaking into responses.

>

> **patches/muse-glimmer-complete.patch in this repo is the corrected, complete port (20 files,

> all three stages).** Please re-apply it. Apologies for the trouble.

✅ The patch you need is in this repo

patches/rocmfpx-3edc3d3-add-muse-glimmer.patch — applies to

charlie12345/ROCmFPX at commit 3edc3d3,

verified with git apply --check.

git clone https://github.com/charlie12345/ROCmFPX.git && cd ROCmFPX
git checkout 3edc3d3
git apply patches/rocmfpx-3edc3d3-add-muse-glimmer.patch
cmake -B build -S . -DGGML_HIP=ON -DAMDGPU_TARGETS=gfx1151 -DCMAKE_BUILD_TYPE=Release
cmake --build build -j$(nproc)

Full build notes, per-architecture details and licence: patches/README.md in this repo.

⚠️ If you add files under src/models/, re-run cmake -B build -S . — the models/*.cpp GLOB

is configure-time, so cmake --build alone will not link them.

---

Muse-Glimmer-30B — ROCmFP4 for AMD Strix Halo (gfx1151)

<!-- LEAD-CLAIM:START -->

> ✅ most quant variants of any public build — 4 ftypes in one repo (next: 3)

>

> Verified against Hugging Face repository metadata for all 2 public ROCmFP4 builds of this base model. Size and file facts only — no third-party build was benchmarked.

<!-- LEAD-CLAIM:END -->

Four ROCmFP4 quantisations of Meta's Muse-Glimmer-30B built for AMD Ryzen AI Max+ 395 /

Radeon 8060S / gfx1151, bundled with the DFlash speculative drafter and vision projector.

ROCmFP4 is a runtime tensor format that exists only in the ROCmFPX fork of llama.cpp;

Muse Glimmer support exists only in current upstream — this build ports the model forward

into the ROCmFPX base so the two can meet.

| Metric | Result |

| --- | ---: |

| Quantization | ROCmFP4 (ggml types 100–106), 4 variants |

| Model size | 13.80 – 16.87 GiB |

| Effective BPW | 4.25 – 4.50 (see table) |

| Tested hardware | AMD Ryzen AI Max+ 395 (Strix Halo), 128 GB unified |

| GPU | Radeon 8060S, gfx1151 |

| ROCm version | 7.2.4 |

| 32K decode (FAST, real-world) | ~17-45 tok/s depending on workload; ~30 typical |

| Prompt processing | not separately instrumented — see Not yet measured |

| DFlash decode | 4.13 tokens accepted per target pass, n_max=15 |

| Peak memory | 19.99 GiB resident (model + drafter + projector + 32K KV) |

| Context validated | 32768 only — see Not yet measured |

| Tool calling | 6/7 on a 7-case suite (parity with upstream) |

| Reasoning | yes — reasoning_content / content split |

| Vision | yes — verified on spatial ground truth, requires -fa off |

Why this build?

• 20.31 tok/s on the FAST variant vs 16.65 tok/s for Meta's fastest official

GGUF (kquant-17gb) — same box, same flags, same drafter: 1.22×

• 13.80 GiB vs Meta's 15.61 GiB — 1.8 GiB smaller and faster
• Four ftypes published so you can pick the size/speed/verbosity point you want
• Text, vision and DFlash speculative decoding all work from one download
• Chat-format parser ported, so no to=self<|message|> control tokens leak into output
• Every published file validated before upload; nothing shipped unverified

Which file should I use?

Ryzen AI Max+ 395, ROCm 7.2.4, DFlash drafter at --spec-draft-n-max 15, -fa on,

ctx 32768, batch 1, temperature 0. Warm medians of 9 generations; the first call after

load is discarded.

| Build | ftype | Size | BPW | TG 32K | Quality |

| --- | --- | ---: | ---: | ---: | ---: |

| ROCmFP4-FAST | 103 | 13.80 GiB | 4.25 | 20.31 | 3/3 |

| ROCmFP4-STRIX_LEAN | 106 | 14.00 GiB | 4.38 | 18.72 | 3/3 |

| ROCmFP4-STRIX | 105 | 14.17 GiB | 4.36 | 17.24 | 3/3 |

| ROCmFP4-BASE | 100 | 16.87 GiB | 4.50 | 15.90 | 3/3 |

| Meta kquant-17gb (reference) | — | 15.61 GiB | — | 16.65 | 3/3 |

| Meta kquant-17gb on Vulkan | — | 15.61 GiB | — | 6.10 | 3/3 |

> Why two different speed figures? The comparison table above is a controlled A/B: every

> build ran the same fixed prompt set, so the numbers are directly comparable to each other and

> to Meta's reference (that is where the 1.22x comes from). Real-world decode on the FAST

> build varies with workload by about 2.6x - measured on the live deployment at ~45 tok/s

> on code-transform/edit work, ~17 tok/s on freeform prose, and ~30 tok/s in typical mixed

> use. Quote the range, not a single number.

Start with FAST. BASE is both the slowest and the largest — it is published for

completeness, not because anyone should choose it.

⚠ The faster files write shorter answers

Some of the speed comes from terser output, not only from faster decode. Median words

per answer on identical prompts:

| Build | Median words |

| --- | ---: |

| STRIX (105) | 377 |

| STRIX_LEAN (106) | 306 |

| BASE (100) | 301 |

| FAST (103) | 259 |

The quality check is substring-based and **cannot distinguish "more concise" from "less

thorough."** If answer depth matters more than throughput, prefer STRIX. This is a real

trade, not a free win.

Quick start

hf download kingjones777/Muse-Glimmer-30B-ROCmFP4-Strix-Halo-DFlash-GGUF --local-dir muse

llama-server \
  -m muse/muse-glimmer-30B-ROCmFP4-FAST.gguf \
  --mmproj muse/mmproj-kquant.gguf \
  --spec-type draft-dflash \
  --model-draft muse/dflash-kquant.gguf \
  --spec-draft-n-max 15 \
  --chat-template-kwargs '{"reasoning_strength":"medium"}' \
  -ngl 999 -fa off -dio --jinja \
  -c 32768 --host 127.0.0.1 --port 8080

Requires a llama.cpp built with ROCmFP4 support (ggml types 100–106). Stock llama.cpp

rejects these tensor types.

Three flags that matter more than which file you pick

| Flag | Why |

| --- | --- |

| --chat-template-kwargs '{"reasoning_strength":"..."}' | Template defaults to high. At high this model spent an entire 1200-token budget deliberating on a real refactor task and returned no visible answer at all. medium answered in 38.1s, low in 32.6s. |

| -fa on (text) / -fa off (vision) | -fa off costs +21% at ctx 32768 but is mandatory for images. Run separate endpoints if you serve both. |

| --spec-draft-n-max 15 | DFlash block size is 16; one slot holds the previously accepted token. |

⛔ --reasoning-budget does not work on this model. Values 256 and -1 produced

byte-identical runs at temperature 0 — the flag is not enforced on peg-native format.

Use reasoning_strength instead.

Verified hardware

| Hardware | GPU | ROCm | Status | TG 32K | Notes |

| --- | --- | --- | --- | ---: | --- |

| Ryzen AI Max+ 395 (Strix Halo) | Radeon 8060S / gfx1151 | 7.2.4 | ✅ Tested by KingJones | ~30 (17-45) | 128 GB unified |

| Any Vulkan backend | — | — | ❌ Known incompatible | — | rejects ggml type 101 at parse time |

| gfx1201 / RDNA4 | — | — | ❓ Untested | — | |

| NVIDIA / CUDA | — | — | ❌ Known incompatible | — | ROCmFP4 is a ROCm-only tensor format |

Vulkan is impossible, not merely slow. The backend rejects these files at parse time:

gguf_init_from_reader: tensor 'output.weight' has invalid ggml type 101. should be in [0, 43)

Vulkan's type table ends at 43; ROCmFP4 types are 100–106. No flag changes this. For

reference, Meta's k-quant does run on Vulkan and measured 6.10 tok/s versus 16.65 on

ROCm on this box, so Vulkan is not a useful path for this model in any case.

Speculative decoding (DFlash)

Muse Glimmer has no MTP tensors — zero in both the base checkpoint and the quants.

It speculates using DFlash against a separate 5-layer drafter, which is what Meta's own

recipe prescribes.

| Metric | Value |

| --- | ---: |

| Setting | --spec-type draft-dflash --spec-draft-n-max 15 |

| Tokens accepted per target pass | 4.13 (median, range 2.85–6.65) |

| Per-token acceptance | ~21% |

| Drafter memory | 1.52 GiB |

⚠️ Per-token acceptance is a misleading statistic for a block drafter. DFlash proposes

15 tokens in one forward pass; ~21% acceptance means ~4.13 tokens land per target pass,

which is healthy. Judge block drafters on tokens-per-pass.

n-gram speculation is not a substitute here. Measured on the same build:

ngram-map-k reached 42.9% acceptance — double DFlash's — yet ran 45% slower on

code-transform work (15.20 vs 27.49 tok/s), because it proposes far fewer tokens per pass.

A ROCmFP4 drafter is included but is not the default. It measured 1.008× against

Meta's k-quant drafter on a quiet box — inside noise, acceptance unchanged. The drafter is

~1.5 GiB of a ~17 GiB working set, so shrinking it 8.5% moves total memory traffic by well

under 1%. Shipped because it is valid, not because it is faster.

Tool calling

7-case suite, run against this build and against Meta's k-quant on the upstream binary

as a reference:

| Case | This build | Upstream reference |

| --- | --- | --- |

| multi-arg (string/int/bool) | ✅ | ✅ |

| nested object argument | ✅ | ✅ |

| enum constraint | ✅ | ✅ |

| correctly declines (no spurious call) | ✅ | ✅ |

| multi-turn tool-result follow-up | ✅ | ✅ |

| streaming tool call | ✅ | ✅ |

| two parallel calls in one turn | ❌ | ❌ |

| Total | 6/7 | 6/7 |

The parallel-call failure is the model's, not the quantisation's — Meta's own weights on

upstream's own parser fail identically. Sequential agent loops are unaffected.

Raw example:

{"name": "book_flight",
 "arguments": {"passenger": {"name": "Alice Smith", "age": 34},
               "route": "LHR-JFK", "cabin": "business"}}

Agentic loop

A multi-step loop (list → move → observe → finish) over a directory of loose files,

3 runs at temperature 0.7: 3/3 completed the task correctly, 0 cases of claiming an

action without emitting a tool call.

Vision

Works, and is verified for spatial correctness rather than plausible-sounding output:

a four-quadrant colour image is scored on whether each colour lands in the right corner.

A misapplied attention mask names colours confidently but places them wrongly, so this

test distinguishes a working port from a fluent-but-broken one. 3/3.

Requires -fa off — ggml_flash_attn_ext aborts on Muse's per-layer sparse-window masks.

Quantization methodology

# 1. convert BF16 safetensors -> GGUF (upstream tree; only it has the muse-glimmer converter)
python convert_hf_to_gguf.py <MODEL_DIR> --outtype bf16 --outfile muse-glimmer-30B-BF16.gguf

# 2. quantize with the ROCmFPX build (only it has ggml types 100-106)
llama-quantize muse-glimmer-30B-BF16.gguf muse-glimmer-30B-ROCmFP4-FAST.gguf 103

Source: meta-models/Muse-Glimmer-30B BF16 safetensors, 1436 tensors → 55.7 GB BF16 GGUF

(731 text tensors) → ROCmFP4.

The model was ported forward into the ROCmFPX base in three stages:

1. Text graph, arch registration and converter. Three API gaps bridged:

is_swa_impl → swa_layers, n_layer() from method to field, and the NVFP4-only

output-scale argument (null on the ROCmFP4 path).

1. Vision tower — required teaching the older base's build_vit to accept per-layer

attention masks at all; it previously took no mask parameter. Added as an overload so

the ~32 other vision models calling it are untouched.

1. Chat-format parser, so harmony-style channel output is parsed rather than leaking

to=self<|message|> into content.

Files

Three distinct networks, not parts of one — llama.cpp loads them via --model,

--model-draft and --mmproj. There is no merged single-file format.

| File | Size | Role |

| --- | ---: | --- |

| muse-glimmer-30B-ROCmFP4-FAST.gguf | 13.80 GiB | model — fastest, recommended |

| muse-glimmer-30B-ROCmFP4-STRIX_LEAN.gguf | 14.00 GiB | model |

| muse-glimmer-30B-ROCmFP4-STRIX.gguf | 14.17 GiB | model — most verbose output |

| muse-glimmer-30B-ROCmFP4-BASE.gguf | 16.87 GiB | model — not recommended |

| dflash-kquant.gguf | 1.52 GiB | DFlash drafter (Meta's, unmodified) — use this |

| dflash-ROCmFP4-STRIX.gguf | 1.39 GiB | our ROCmFP4 drafter — works, 1.008× (a wash) |

| mmproj-kquant.gguf | 1.30 GiB | vision projector (Meta's, unmodified) |

Not yet measured

Listed explicitly so nobody mistakes absence for a pass. These are genuine gaps, not

claims:

| Test | Status |

| --- | --- |

| Context scaling (2K / 8K / 16K / 64K / 128K) | ❓ only 32768 measured |

| Prompt-processing tok/s, isolated | ❓ not separately instrumented |

| Sustained generation (1K / 4K tokens) | ❓ not measured |

| Perplexity / KL divergence vs BF16 | ❓ not measured |

| MMLU-Pro, GPQA, GSM8K, HumanEval+, MBPP+ | ❓ not run |

| Long-context needle retrieval | ❓ not run |

| DFlash n-max sweep (2 / 4 / 8 / 24) | ❓ only n=15 measured |

| 5-run statistics with std dev | ⚠️ 9 samples per arm, median reported; std dev not published |

| Independent reproduction | ❓ none yet |

Measurement conditions: the tok/s figures were taken on a machine that also served

other traffic during the run. The ordering across builds is wide enough to be reliable;

the exact ratios are not trustworthy to three significant figures. A re-run on a quiesced

box is planned.

Quality caveat: the 3/3 figure is a smoke check over factual recall, arithmetic and

instruction-following, scored by substring match. It is a regression guard against a

broken quantisation, not a benchmark suite, and it does not measure answer depth.

No claim of "no quality loss" is made — that would require the perplexity and standardized

evaluations listed above.

Independent results

None yet. If you run this build, please open a discussion with: hardware, GPU, OS, ROCm

version, runtime commit, exact command, context, prompt-processing tok/s, generation

tok/s and peak RAM. Independent reproductions will be listed separately from author

benchmarks and carry more weight.

Known issues

1. Vulkan/CUDA/CPU cannot load these files — ROCmFP4 is a ROCm-only tensor format.
2. Vision requires -fa off, costing ~21% on text at 32K context.
3. Parallel tool calls fail — model-level, reproduced identically on Meta's own weights.
4. Small max_tokens returns empty content — the budget goes to reasoning_content.

Allow several hundred tokens.

1. --reasoning-budget is not enforced on this model; use reasoning_strength.

License and attribution

Base model, DFlash drafter and vision projector are Meta's, under the base model's licence.

ROCmFP4 quantisation types are from the ROCmFPX fork of llama.cpp. This repository contains

the quantised weights and the measurements above.

<!-- PEER-TABLE:START -->

Other public builds of this model

Compiled from Hugging Face repository metadata — file sizes, shipped files, quant variant as named by each repo. No third-party build was run or benchmarked here, so this table makes no speed or quality claim about any of them. It is here so you can see the size and format options at a glance and pick what fits your hardware.

| Repository | Largest model file | Variant | Ships | Downloads | Likes |

| --- | ---: | --- | --- | ---: | ---: |

| RadixArk/Muse-Glimmer-NVFP4 | 4.00 GiB | NVFP4 | safetensors | 40 | 5 |

| kingjones777/Muse-Glimmer-30B-ROCmFP4-Strix-Halo-DFlash-GGUF (this repo) | 16.87 GiB | STRIX | 4 model files, drafter, vision | 0 | 1 |

| Preyazz/Muse-Glimmer-30B-NVFP4 | 17.37 GiB | NVFP4 | safetensors | 0 | 5 |

| cloudnathan5/Muse-Glimmer-30B-NVFP4 | 18.63 GiB | NVFP4 | safetensors | 0 | 2 |

| RedHatAI/Muse-Glimmer-30B-NVFP4 | 18.63 GiB | NVFP4 | safetensors | 0 | 7 |

| vmlinux/Muse-Glimmer-30B-ROCmFPX-GGUF | 26.77 GiB | ROCmFPX | 3 model files, drafter, vision | 0 | 13 |

Base model: meta-models/Muse-Glimmer-30B. Generated from Hub metadata; download counts move over time.

<!-- PEER-TABLE:END -->

<!-- CREDITS:START -->

Acknowledgements

This build would not exist without the work below. Please star and follow these

projects — the quantisation format used here is their engineering, not mine.

**ROCmFPX — maintained by

charlie12345 / caf**

The ROCmFP4 / ROCmFPX tensor formats (ggml types 100–106) exist only in this fork.

Every ROCmFP4 file in this repository was produced with its llama-quantize, and

runs on its runtime. The fork also credits collaborators ciru-ai, Tom Turney,

PlunderStruck and Aydan S., and acknowledges AMD for hardware support.

Licensed MIT, based on upstream llama.cpp.

llama.cpp — ggml-org and contributors

The inference engine, GGUF format and conversion tooling everything here is built on.

AMD ROCm

The compute platform these builds target — ROCm 7.2.4 on gfx1151 / Radeon 8060S.

Base model authors — see base_model in the metadata above; all model weights,

licences and capabilities are theirs. This repository contributes quantisation and

measurement only.

If you use these files, please credit ROCmFPX alongside this repository.

<!-- CREDITS:END -->

All quant variants

⚠️ Decode on this model is workload-dominated, not variant-dominated. DFlash proposes long

runs on repetitive/code text and very little on freeform prose, so a single tok/s number is

misleading. Measured on one Ryzen AI MAX+ 395, median of 3, -fa on, with the DFlash head

(--spec-type draft-dflash --model-draft dflash-ROCmFP4-STRIX.gguf --spec-draft-ngl 99):

| variant | ftype | size | prose | code-transform | acceptance len (code) |

|---|---|---|---|---|---|

| 4-bit FAST | 103 | 13.80 GiB | 15.07 | 39.35 t/s | 7.12 |

| 4-bit STRIX | 105 | 14.17 GiB | 14.96 | 37.55 t/s | 6.80 |

| 8-bit plain | 111 | 26.85 GiB | 11.31 | — | 2.65 |

| 8-bit AGENT | 115 | 27.23 GiB | 11.27 | — | 2.51 |

The 2.6× spread between prose and code is the same model and the same binary — acceptance

length moves 2.9 → 7.1. Quote a range for this model, not a point.

⛔ Serve it with the draft head. Without --model-draft the 8-bit build drops 11.62 → 7.65

(−34%). ⚠️ -fa off is required only for the vision path; text-only can run -fa on.

The two 8-bit builds are within noise of each other — AGENT lifts draft acceptance on MTP

models, and this one uses DFlash, so there is nothing for it to win here.