---

license: other

base_model: meta-models/Muse-Glimmer-30B

base_model_relation: quantized

pipeline_tag: image-text-to-text

library_name: gguf

tags:

• gguf
• llama.cpp
• rocm
• amd
• rocmfp4
• rocmfpx
• strix-halo
• amd-strix-halo
• gfx1151
• ryzen-ai-max
• ryzen-ai-max-395
• radeon-8060s
• dflash
• speculative-decoding
• tool-calling
• multimodal
• vision
• muse
• glimmer
• quantized

---

> ### 🔧 Runtime: build the ROCmFPX fork below

> Stock llama.cpp will not load this file. You need both the muse-glimmer architecture

> and the ROCmFP4 tensor types in one tree. Upstream

> charlie12345/ROCmFPX has the ROCmFP4 types but

> not muse-glimmer. Our fork has both:

>

> kingjones30/ROCmFPX — a fork of charlie12345/ROCmFPX, branch main.

>

> ```bash

> git clone https://github.com/kingjones30/ROCmFPX.git

> cd ROCmFPX

> cmake -B build -DGGML_HIP=ON -DGPU_TARGETS=gfx1151 -DGGML_NATIVE=ON -DCMAKE_BUILD_TYPE=Release

> cmake --build build --target llama-server llama-quantize -j$(nproc)

> ```

>

> Verified 2026-08-27 on gfx1151: clean clone → 0 build errors → llama-server loads a

> muse-glimmer ROCmFP4 GGUF from this family and generates coherent text.

Muse-Glimmer-30B — ROCmFPX 8-bit AGENT for AMD Strix Halo (gfx1151)

<!-- LEAD-CLAIM:START -->

> ✅ the complete muse-glimmer port — text graph, vision projector and chat parser — ships as a single applyable patch in this repo

>

> muse-glimmer is not an upstream llama.cpp architecture. Running it end-to-end takes three independent pieces of work; all three are in patches/muse-glimmer-complete.patch (20 files, 81,968 bytes, git apply --check clean).

<!-- LEAD-CLAIM:END -->

An 8-bit ROCmFPX quantisation of Muse-Glimmer-30B for AMD Ryzen AI Max+ 395 / Radeon 8060S / gfx1151, quantised from the BF16 GGUF — a lossless source, not a requantisation of a smaller file. ROCmFPX is a runtime tensor format that exists only in the ROCmFPX fork of llama.cpp.

| Metric | Result |

| --- | ---: |

| Quantization | Q8_0_ROCMFPX_AGENT (ggml ftype 115) |

| Model size | 27.23 GiB (29,235,965,312 bytes) |

| Effective BPW | 8.39 (measured, not advertised) |

| Source | BF16 GGUF, 55,725,514,112 bytes |

| Tested hardware | AMD Ryzen AI Max+ 395 (Strix Halo), 128 GB unified |

| GPU | Radeon 8060S, gfx1151 |

| Decode, no draft head | 7.22 tok/s (median of 3: 7.22, 7.21, 7.26) |

| Decode with DFlash, prose | 11.31 tok/s |

| Correctness | ✅ 3/3 — 391 / Tokyo / 366 |

| Vision (256×256 placement) | ✅ pass, requires -fa off |

| Chat parser | ✅ clean content — no control-token leak |

| Speculative head | DFlash, not MTP |

Why this build?

• Quantised from BF16, source revision verified by fetch (a4e59da52a7bc87ae7251dd5545c0dd437c44b68) rather than assumed
• Output head and token embeddings pinned to q8_0 explicitly, not left to inherit the block type
• The full architecture port is in the repo — you do not have to reconstruct it
• Vision works through the BF16 projector built by stage 2 of that patch
• Sizes are reported from stat, and checked against the --dry-run projection: the ~12.5 MiB delta is GGUF header, which is the signature of a complete file

Which file should I use?

If you want speed, take a 4-bit build — on this model the 4-bit files are roughly 3.5× the code-transform throughput of the 8-bits and less than half the size. The 8-bits are here for bit-count, not for speed.

| Build | ftype | Size | BPW | prose | code-transform | accept len (code) |

| --- | --- | ---: | ---: | ---: | ---: | ---: |

| ROCmFP4-FAST | 103 | 13.80 GiB | 4.25 | 15.07 | 39.35 | 7.12 |

| ROCmFP4-STRIX | 105 | 14.17 GiB | 4.39 | 14.96 | 37.55 | 6.80 |

| Q8_0_ROCMFPX | 111 | 26.85 GiB | 8.28 | 11.31 | — | 2.65 |

| Q8_0_ROCMFPX_AGENT | 115 | 27.23 GiB | 8.39 | 11.27 | — | 2.51 |

> ⚠️ Decode on this model is workload-dominated, not variant-dominated. DFlash proposes long runs on repetitive and code-like text and very little on freeform prose, so a single tok/s figure is misleading — mean accepted length moves 2.5 → 7.1 across the same binary and the same weights. Quote a range for this model, not a point.

muse-glimmer-30B-Q8_0_ROCMFPX_AGENT.gguf and muse-glimmer-30B-Q8_0_ROCMFPX.gguf are within noise of each other (7.22 vs the other build's figure on the same harness). The AGENT routing lifts draft acceptance on MTP models; this model uses DFlash, so there is nothing for it to win here. Choose on size.

Quick start

hf download kingjones777/Muse-Glimmer-30B-ROCmFPX-Q8_0-AGENT-GGUF --local-dir glimmer

The DFlash drafter is not duplicated here — pull it from the 4-bit repo, which carries every

drafter and projector variant:

hf download kingjones777/Muse-Glimmer-30B-ROCmFP4-Strix-Halo-DFlash-GGUF \
  --include "dflash-ROCmFP4-STRIX.gguf" --local-dir glimmer

⚠️ hf download silently ignores --include when given more than one pattern — one call per file.

llama-server \
  -m glimmer/muse-glimmer-30B-Q8_0_ROCMFPX_AGENT.gguf \
  --mmproj glimmer/mmproj-muse-glimmer-30B-BF16.gguf \
  --spec-type draft-dflash --model-draft glimmer/dflash-ROCmFP4-STRIX.gguf \
  --spec-draft-ngl 99 \
  -ngl 999 -c 4096 -fa off -fit off --jinja \
  --host 127.0.0.1 --port 8080

Three flags that matter more than which file you pick

| Flag | Why |

| --- | --- |

| --model-draft | Serve it with the DFlash head. Without one, the 8-bit build drops 11.62 → 7.65 tok/s (−34%). This single flag outweighs the quantisation choice. |

| -fa off | Required for the vision path on gfx1151. Text-only serving can use -fa on. |

| -fit off | llama.cpp's autofit reads MemAvailable on integrated GPUs, which is at its lowest right after a model unload — leaving it on can silently shrink context or push tensors to CPU. |

⚠️ This head is DFlash, not MTP. Read mean accepted length, and do not pass MTP flags to it.

Vision

Verified on spatial placement rather than plausible-sounding output: a solid-colour image is scored on whether the model names the colour that is actually there.

| Input | Result |

| --- | ---: |

| 256×256 solid red | ✅ red |

Serve with -fa off and the BF16 projector from this repo (mmproj-muse-glimmer-30B-BF16.gguf, 3,849,174,048 bytes, 809 tensors).

⚠️ Minimum useful image size is 28×28 px. The preprocessor snaps to patch 14 × merge 2, so anything smaller collapses to a single merge token, carries no spatial signal, and the model reports the dominant colour of the padded canvas. Feed 256×256 or larger. llama-mtmd-cli behaves identically — this is the preprocessing geometry, not the projector.

Files

| File | Size | Role |

| --- | ---: | --- |

| muse-glimmer-30B-Q8_0_ROCMFPX_AGENT.gguf | 27.23 GiB | model — this repo |

| mmproj-muse-glimmer-30B-BF16.gguf | 3.58 GiB | vision projector — use this one |

| patches/muse-glimmer-complete.patch | 80 KiB | 20-file architecture port |

| dflash-ROCmFP4-STRIX.gguf | 1.39 GiB | DFlash drafter — lives in the 4-bit repo, not here |

🩹 The muse-glimmer architecture port — complete, three stages

patches/muse-glimmer-complete.patch — 81,968 bytes, 20 files, git apply --check clean

against charlie12345/ROCmFPX.

git clone https://github.com/charlie12345/ROCmFPX.git && cd ROCmFPX
git apply --check ../patches/muse-glimmer-complete.patch && \
git apply           ../patches/muse-glimmer-complete.patch
cmake -S . -B build-muse -DGGML_HIP=ON -DAMDGPU_TARGETS=gfx1151 \
      -DLLAMA_BUILD_WEBUI=OFF -DCMAKE_BUILD_TYPE=Release
cmake --build build-muse --target llama-server --target llama-quantize \
      --target llama-mtmd-cli -j 6

muse-glimmer is not an upstream architecture. Running it end-to-end takes three independent

pieces of work, and all three are in this patch.

Stage 1 — text graph and conversion

| File | Role |

| --- | --- |

| src/llama-arch.{h,cpp} | LLM_ARCH_MUSE_GLIMMER + tensor-name table |

| src/llama-model.cpp | model factory case |

| src/models/muse-glimmer.cpp | the graph itself |

| src/models/models.h | declaration |

| gguf-py/gguf/{constants,tensor_mapping}.py | GGUF constants + tensor map |

| conversion/{__init__,muse_glimmer}.py | safetensors → GGUF converter |

⚠️ convert_hf_to_gguf.py does not know MuseGlimmer. The converter is

conversion/muse_glimmer.py, driven through conversion.get_model_class.

Stage 2 — vision projector (clip / mtmd)

| File | Role |

| --- | --- |

| tools/mtmd/models/muse-glimmer.cpp | projector graph |

| tools/mtmd/clip-impl.h | PROJECTOR_TYPE_MUSE_GLIMMER |

| tools/mtmd/clip-model.h | hparams + LANCZOS enum |

| tools/mtmd/clip-graph.h | build_vit_opts + 7-arg overload |

| tools/mtmd/clip.cpp, models/models.h, CMakeLists.txt | wiring |

| tools/mtmd/mtmd-image.{cpp,h} | preprocessor; LANCZOS → bicubic_pillow fallback |

| tools/mtmd/mtmd.cpp | <\|image_start\|> / <\|image_end\|> markers |

| gguf-py/gguf/tensor_mapping.py | model.vision_tower.layers.{bid}.attn.{q,k,v,proj}, norm1/2, mlp.fc1/2, ln_post |

| gguf-py/gguf/constants.py | VisionProjectorType.MUSE_GLIMMER |

This stage is what makes --mmproj work. Use the BF16 projector in this repo

(mmproj-muse-glimmer-30B-BF16.gguf, 3,849,174,048 bytes, 809 tensors,

clip.projector_type = muse-glimmer, merge 2, patch 14, image_size 896).

Stage 3 — chat parser

common/chat.cpp — common_chat_params_init_muse_glimmer, PEG_NATIVE.

Without it the model's to=self<|message|> control sequence is emitted into content. With it,

content is clean: "The capital of Japan is Tokyo."

Build note

cmake's source GLOB is configure-time. After the patch adds src/models/muse-glimmer.cpp you

must re-run the cmake -S . -B build-muse configure step, not just --build.

-DLLAMA_BUILD_WEBUI=OFF avoids a node/npm requirement.

Applying to a different base commit

The patch header names commit 3edc3d3, and it applies cleanly to later revisions

(verified on b41ce12). On trees where cohere2moe and bailing_hybrid model sources are absent,

their factory cases in llama-model.cpp reference symbols that do not exist in that tree — build

those two out, or apply on 3edc3d3 where their .cpp files are present. The muse-glimmer factory

case and graph are independent of both.

Quantization methodology

# 1. convert BF16 safetensors -> GGUF (the muse-glimmer converter is not in
#    convert_hf_to_gguf.py; it ships as conversion/muse_glimmer.py in the patch)
python -c "from conversion import get_model_class; ..."   # driver, see patch README

# 2. measure the real BPW before committing
llama-quantize --dry-run muse-glimmer-30B-BF16.gguf /tmp/x.gguf Q8_0_ROCMFPX 8

# 3. quantize with the patched ROCmFPX build (only it has ggml types 100-106)
llama-quantize --output-tensor-type q8_0 --token-embedding-type q8_0 \
  muse-glimmer-30B-BF16.gguf muse-glimmer-30B-Q8_0_ROCMFPX.gguf Q8_0_ROCMFPX 16

Source: meta-models/Muse-Glimmer-30B, revision

a4e59da52a7bc87ae7251dd5545c0dd437c44b68 (fetched and verified, not assumed) →

muse-glimmer-30B-BF16.gguf 55,725,514,112 bytes. Quantized from BF16 only — this is not a

requantization of a smaller file.

--output-tensor-type and --token-embedding-type are both set explicitly so the output head and

token embeddings land at q8_0 rather than inheriting the block type.

Not yet measured

Listed so nobody mistakes absence for a pass:

| Test | Status |

| --- | --- |

| Perplexity / KL divergence vs BF16 | ❓ not measured |

| Long-context behaviour beyond 4096 | ❓ not measured |

| Vision beyond single-colour placement (OCR, charts, documents) | ❓ not measured |

| MMLU-Pro, GPQA, GSM8K, HumanEval+ | ❓ not run |

| Multi-step agentic loop | ❓ not run |

| Sustained multi-thousand-token generation | ❓ not measured |

| Independent reproduction | ❓ none yet |

Known issues

1. Stock llama.cpp cannot load these files. muse-glimmer is not an upstream architecture and

ROCmFP4/ROCmFPX are not upstream tensor types. Both come from the patch in this repo.

1. -fa off is required for the vision path on gfx1151. Text-only serving runs fine with

-fa on.

1. Images below 28×28 px collapse to a single merge token. The preprocessor snaps to

patch 14 × merge 2, so an 8×8 input carries no usable spatial signal and the model reports the

dominant colour of the padded canvas. Feed images at 256×256 or larger.

1. Vulkan / CUDA / CPU cannot load these files — ROCmFP4/ROCmFPX are ROCm-only formats.

Independent results

None yet. If you run this build, please open a discussion with hardware, GPU, ROCm version, runtime

commit, exact command, context, prompt-processing tok/s, generation tok/s and peak RAM. Independent

reproductions will be listed separately from author benchmarks and carry more weight.

License and attribution

Base model and its licence are the Muse team's. ROCmFP4 / ROCmFPX quantisation types are from the

ROCmFPX fork of llama.cpp. This repository contains the quantised weights, the architecture port and

the measurements above.

<!-- CREDITS:START -->

Acknowledgements

**ROCmFPX — maintained by

charlie12345**

The ROCmFP4 / ROCmFPX tensor formats (ggml types 100–106) exist only in this fork. Every file here

was produced with its llama-quantize and runs on its runtime. Licensed MIT, based on upstream

llama.cpp.

llama.cpp — ggml-org and contributors

The inference engine, GGUF format and conversion tooling everything here is built on.

AMD ROCm — the compute platform these builds target, on

gfx1151 / Radeon 8060S.

Muse — the base model and its licence are theirs. This

repository contributes the architecture port, quantisation and measurement only.

If you use these files, please credit ROCmFPX alongside this repository.

<!-- CREDITS:END -->