GLM-5.3-Flash-PaoAI-ROCmFP4-STRIX-BALANCED-GGUF

The balanced GLM-5.3-Flash quant for AMD Strix Halo (Ryzen AI Max+ 395 / gfx1151). Measured, not marketed: loads on ONE Strix Halo box with zero special accommodations — the smallest file (71.8 GiB) and the no-fuss daily driver.

Brand: PaoAI · community quant, not affiliated with Z.AI.

---

> Updated 2026-09-21 — re-measured on the latest version (paoai-strix-engine, strix-main @ 4a8440f58): decode faster at every depth (+32% at 128K), chain re-run stated honestly below, all sweep points integrity-checked. Weights unchanged.

What is this? (30 seconds)

We took the 642 GB GLM-5.3-Flash and compressed it organ by organ: the parts that need precision keep it, the parts that don't get compressed harder. The BALANCED mix stays small enough for one box with no offload flags.

The recipe (what's inside)

| Model part | What it does | Format |

|---|---|---|

| 👁 Attention (attn, indexer) | reads your prompt, tracks context | Q3_K (~3.4 bpw) |

| 🧠 Experts (FFN MoE) | the knowledge (most of the size) | IQ1_M (~1.75 bpw, imatrix) |

| ✍️ Draft head (blk.45) | MTP speculative guessing | Q6_K (~6.6 bpw) |

| 📡 Router, glue, norms | keep signals correct | FP32 (never touched) |

Chain-test results (N=3 medians — Sep 2026)

We run a fixed coding exam ("chain test") where tasks are graded by execution, not opinion: the model reads deceptive code and must trace it, implements a novel state machine, and writes a thread-safe ring buffer that survives a 5,000-operation stress harness. Three identical runs per model; we report medians with spreads (identical runs vary — single scores are unreliable).

| Task (what it measures) | Median of 3 | Runs |

|---|---|---|

| Automaton — implement novel rules + exact state trace | 100 | 100 · 100 · 40 |

| Hunt — read lying code: trace it + find the planted bug | 60 | 60 · 100 · 60 |

| Ring buffer — multi-threaded code under stress | 20 | 20 · 20 · 100 |

Task median: 67/100 (runs 60 · 73 · 67)

Re-run 2026-09-21 on the latest version (4a8440f58): hunt 60 · automaton 40 · ringbuffer 20 — task 40 (runs 40 · 40 · 60). Hunt and ring-buffer match the published profile; automaton dropped from 100 to 40 median (3 runs, never reached 100). Cause not yet isolated — stated plainly rather than hidden.

Plain read: novel-algorithm implementation is reliable (100 median). Multi-threaded code is the known weak spot — for concurrency tasks, generate → run the tests → regenerate once if needed.

Speed (decode, tokens/second, per phase)

| Phase | t/s (3-run range) |

|---|---|

| Spec (planning) | 11.0 – 13.7 |

| Hunt (analysis) | 14.5 – 15.5 |

| Automaton (code writing) | 15.7 – 18.2 ← fastest sustained code output of our builds |

| Ring buffer (code writing) | 10.2 – 12.5 |

| Publish (long output) | 10.3 – 10.7 — filled the full 12,000-token budget every run |

These are short-context speeds (tasks send a few hundred tokens of prompt).

Deep-context sweep — what happens when the window fills (7.7K → 131K)

Latest version — re-measured 2026-09-21 (4a8440f58, defaults, every point integrity-checked):

| Filled context | Decode t/s (n=4) | Prefill t/s | MTP accept |

|---|---:|---:|---:|

| 7.9K | 14.1 | 133.6 | 0.51 |

| 16.4K | 13.3 | 115.5 | 0.59 |

| 32.8K | 11.1 | 90.2 | 0.57 |

| 65.6K | 10.1 | 67.7 | 0.65 |

| 98.3K | 9.1 | 57.0 | 0.64 |

| 131.1K | 8.6 | 49.4 | 0.64 |

Plain read (latest): decode is faster than the previous version at every depth (+32% at 128K); prefill wins mid-range (+9-15% at 16-33K) and is slightly behind at 64-128K.

Previous version (7.7K → 128K)

The phase table above is short-context speed. This is the other question: how fast is it when the context is actually full? One server launch per arm; we filled the window in five steps and measured decode + prefill per point — 300 output tokens each, a fresh prompt every point (no prefix-cache reuse), run at draft depth 4 and depth 6.

!GLM-5.3-Flash PaoAI BALANCED — decode (draft depth 4 vs 6), prefill and MTP acceptance vs filled context

| Filled context | Decode t/s (n=4) | Decode t/s (n=6) | Prefill t/s | MTP accept (n=4) | Memory (GTT) |

|---|---|---|---|---|---|

| 7.7K | 13.7 | 12.6 | 134.8 | 0.55 | 82.3 GB |

| 30.6K | 11.5 | 10.4 | 100.0 | 0.55 | 82.3 GB |

| 61.2K | 10.8 | 9.2 | 82.5 | 0.61* | 82.4 GB |

| 91.8K | 8.4 | 6.8 | 71.8 | 0.54 | 82.4 GB |

| 122.4K | 6.5 | 7.1 | 64.7 | 0.46 | 82.5 GB |

One Strix Halo, nothing else serving, flash attention, q8_0 KV — measured on the previous version (ROCmFPX port build). Prefill reproduced within 1–2% across both arms. \±0.03 sampling noise.*

Plain read: the decay is clean and predictable. Per-token cost grows straight-line at +1.8 ms per 1K of context (linear within ±3% across the whole range) — no cliff, nothing unexpected biting. Memory stays flat across 16× context (82.3 → 82.5 GB GTT), so nothing is spilling. Prefill declines gently but never collapses (134.8 → 64.7 t/s). Single-pass acceptance wobbles run-to-run (±0.05), so don't over-read any one acceptance point. Even at 122K of packed context, you still get 6.5 t/s on one box.

Draft depth 4 vs 6: depth 6 is slower at short context (12.6 vs 13.7 t/s at 7.7K — deeper guesses cost more than they return), but the longer draft pays off once the window is packed: at 122K it edged ahead (7.1 vs 6.5 t/s), because fewer verify passes matter more when each pass is expensive. At the top end both arms are within single-pass sampling noise of each other — depth 4 stays the default; depth 6 is only interesting if you live at 100K+ of context.

Facts

| Metric | Result |

|---|---|

| File size | 71.8 GiB (single file) |

| Load | full GPU, no special flags needed |

| Context | tested at 64K (-c 65536); sweep run at -c 131072 |

| Served with | MTP speculative decoding (n-max 4), flash attention, KV q8_0 |

Requirements — which llama.cpp build

Latest version (recommended): paoai-strix-engine, branch strix-main, commit 4a8440f58 — this is what the latest-version numbers below were measured on.

Previous version: the ROCmFPX route below still works and matches the previous-version numbers.

This model uses two things stock llama.cpp doesn't have:

• the glm5next architecture (GLM-5.3-Flash)
• the Q4_0_ROCMFP4_FAST quant type (type 101 — ROCm FP4, gfx1151-native)

Both live in guevae2/ROCmFPX, branch glm5next, commit 3345156 (glm5next arch + FP4 type merged; head of that branch — also buildable from kingjones30/ROCmFPX main once glm5next is upstreamed). Tested with GGML_VULKAN=ON on Strix Halo (gfx1151).

If you build from the wrong tree you'll see exactly one of these:

| Build | Error |

|---|---|

| stock llama.cpp | unknown model architecture: 'glm5next' |

| Unsloth's glm5next branch | invalid ggml type 101. should be in [0, 43) |

How to run (measured, locked)

llama-server -m GLM-5.3-Flash-PaoAI-ROCmFP4-STRIX-BALANCED.gguf \
  --host 0.0.0.0 --port 8080 -ngl 999 -c 65536 --parallel 1 \
  --spec-type draft-mtp --spec-draft-n-max 4 \
  -fa on --cache-type-k q8_0 --cache-type-v q8_0 --lazy-mode off

Add --path <webui assets> if your build serves the WebUI from disk.

Known limits (honest)

• Threading code: ring-buffer implementations crash or deadlock in most chain runs (median 20). Single-threaded generation is strong. Regenerate-once guidance above.
• Long thinking: GLM-5.3-Flash reasons heavily; give it generous token budgets for complex tasks or answers get cut mid-thought
• Driver wall (Strix Halo): Vulkan single allocations near ~98GB can wedge the GPU (needs reboot) — this model at 71.8 GiB is safely under, but don't merge shards into one file

Serving features explained (plain words)

| Flag | What it does | Cost |

|---|---|---|

| --spec-type draft-mtp --spec-draft-n-max 4 | draft head guesses 4 tokens ahead, model verifies in one pass — ~2× decode (acceptance ~50%) | needs the fine draft head we kept (Q6_K) |

| -fa on | flash attention — faster prefill + decode | none |

| --cache-type-k/v q8_0 | half context memory → 64K fits one box | negligible |

| --reasoning-budget 1500 | caps thinking so answers always arrive | complex tasks may need more |

| prompt caching (default) | follow-up turns prefill in ms | none |

Decode speed drops as context fills — now measured in detail: 13.7 t/s at 7.7K → 6.5 t/s at 122K, straight-line cost growth with no cliff (full table + chart above). Stable over time. Full plain-words guide: serving-glossary.md in our repo docs.

Update log

| date | change |

|---|---|

| 2026-09-21 | Re-measured on latest version (4a8440f58): decode faster at every depth; prefill re-run with integrity checks; chain re-run stated honestly; requirements now list the paoai-strix-engine route. |

| 2026-09-12 | Initial release. |

Fleet recipes

Serving configs, ring self-heal infra, and our benchmark battery: paoai-strix-recipes (GitHub).

Credits & license

• Z.AI — GLM-5.3-Flash base model (MIT license; included in repo)
• Unsloth — BF16 source + imatrix
• kingjones30 / charlie12345 — ROCmFPX format & recipes
• ggml-org — llama.cpp
• PaoAI — glm5next port, hybrid recipe, fleet benchmarking, chain test

MIT, same as the base model. Not affiliated with Z.AI.