---

license: mit

base_model: zai-org/GLM-4.7-Flash

base_model_relation: quantized

tags:

- gguf

- rocm

- rocmfp4

- strix-halo

- ryzen-ai-max

- glm

---

GLM-4.7-Flash Q4_0_ROCMFP4_STRIX (GGUF)

First public ROCmFP4 quant of the full GLM-4.7-Flash weights for AMD Ryzen AI Max+ 395

(gfx1151 / Radeon 8060S).

The existing ROCm-flavoured GLM-4.7 quants on the Hub are all of the pruned

REAP-23B-A3B variant. This is the full 30B model, not a pruned one.

> ⚠️ Not compatible with upstream llama.cpp. Requires the

> charlie12345/ROCmFPX fork built with HIP + ROCmFP4 kernels.

Files

| File | Size | Notes |

|------|------|-------|

| GLM-4.7-Flash-Q4_0_ROCMFP4_STRIX.gguf | 14.93 GiB | 4.28 BPW (quantize report) |

Base model: zai-org/GLM-4.7-Flash

BF16 source: unsloth/GLM-4.7-Flash-GGUF

BF16/ 2 shards, 55.79 GiB total (16.00 BPW)

Hardware / stack (validated)

• Ryzen AI Max+ 395, gfx1151, 128 GB unified
• ROCm 7.2.4
• Fork: charlie12345/ROCmFPX @ b41ce12

Build recipe

llama-quantize \
  GLM-4.7-Flash-BF16-00001-of-00002.gguf \
  GLM-4.7-Flash-Q4_0_ROCMFP4_STRIX.gguf \
  Q4_0_ROCMFP4_STRIX

Q4_0_ROCMFP4_STRIX (type 105) is the Strix Halo attention-K/V quality recipe.

Dry-run predicted 15278.74 MiB @ 4.28 BPW and the output matched exactly.

Serving

llama-server --host 127.0.0.1 --port 8080 \
  --model GLM-4.7-Flash-Q4_0_ROCMFP4_STRIX.gguf \
  -dev ROCm0 -ngl 999 -fa on --no-mmap \
  --ctx-size 65536 --parallel 1 -b 2048 -ub 1024 -t 16 --poll 50 --jinja \
  --reasoning-format deepseek --chat-template-kwargs '{"enable_thinking":false}'

⚠️ --chat-template-kwargs '{"enable_thinking":false}' matters. With thinking on, reasoning

consumes the token budget and answers get truncated at ordinary max_tokens values. Also use the

chat endpoint — raw /completion with a bare instruction makes this model emit repetition loops.

Measured

Against the UD-Q4_K_XL GGUF of the same model on the same machine, chat endpoint, thinking off:

| | this quant | UD-Q4_K_XL |

|---|---:|---:|

| size | 14.93 GiB | 16.32 GiB |

| quality battery | 22 / 24 | 22 / 24 |

| tok/s (ROCm) | 56.1 | 48.1 |

The battery is 24 items: 8 code tasks graded by executing the generated function against

assertions, 8 long-tail factual questions, 4 multilingual, 4 maths. Both quants scored 22/24 —

parity — though they fail different items, which is sampling noise rather than quantization damage.

ngram-map-k adds roughly 1.05× on top and is worth enabling for input-grounded work

(code edits, RAG, summarisation).

Note on layout variants

Q4_0_ROCMFP4_FAST (type 103, the "single-scale speed layout") and Q4_0_ROCMFP4_FAST_COHERENT

(104) were both built and benchmarked against this one, same model, same flags, sizes within

0.1 GiB:

| layout | base | + ngram |

|---|---:|---:|

| STRIX (105) | 55.4 | 57.3 |

| FAST (103) | 53.9 | 55.0 |

| FAST_COHERENT (104) | 53.9 | 56.7 |

FAST is ~3% slower than the quality recipe despite its name, on this architecture and hardware.

STRIX is both the faster and the higher-quality choice here, which is why it is the one published.

Runtime knobs (micro-batch 512/2048, 32 threads, q8_0 KV cache, --poll 0) were all

neutral-to-negative — quantizing the KV cache changing nothing indicates the bottleneck is weight

bandwidth rather than KV.

License

Follow the base model (zai-org/GLM-4.7-Flash) license terms.