GLM-5.3-Flash-PaoAI-ROCmFP4-STRIX-MTP-GGUF

The pure-FP4 pool quant of GLM-5.3-Flash for AMD Strix Halo fleets. The whole model on the chip's native 4-bit format (159.2 GiB), designed to be split across two Strix Halo boxes over a direct link when one box isn't enough. The MTP draft head is preserved in the file — but note: on a 2-box pool, speculative decoding is OFF (pool round-trips eat the draft's gains; measured in our fleet).

Brand: PaoAI · community quant, not affiliated with Z.AI.

---

Who is this for?

You have two or more Strix Halo boxes connected directly (USB4/10GbE) and want the full GLM-5.3-Flash in native FP4 split across them. If you have ONE box, use our STRIX-BALANCED instead — a pool of this model loses to a solo smaller quant on both speed and simplicity.

The recipe

| What | Format |

|---|---|

| Everything (attention, experts, draft head) | Q4_0_ROCMFP4_FAST (native FP4, 4.26 bpw) |

| Router, norms, glue | FP32 (keeper list) |

| MTP draft head (blk.45) | preserved (baked into FP4) |

Measured on real hardware

| Metric | Result |

|---|---|

| File size | 159.2 GiB (single file) |

| 2-box pool decode | ~5.4 t/s (stable ×3, both boxes ~80GB each) |

| Solo on one box | ❌ will wedge the gfx1151 driver (98GB+ allocation wall — see BALANCED/HYBRID for solo options) |

| Quality | not battery-scored (pool-only use case; solo score would not represent pool serving) |

How to run (2-box pool, direct link)

# halo-2 (worker):
ggml-rpc-server --host 0.0.0.0 --port 50052
# BG1 (head):
llama-server -m GLM-5.3-Flash-PaoAI-ROCmFP4-STRIX-MTP.gguf \
  --host 0.0.0.0 --port 8081 -ngl 99 -c 32768 --parallel 1 \
  --rpc <worker-ip>:50052 -fa on

⚠️ Pool endpoint rule (learned the hard way): use the DIRECT box-to-box link addresses, never routes through a third machine — RPC traffic via a middle box deadlocks after load (head stuck in send, worker in recv).

Honest position

This is the largest and least convenient of our three models: it needs two boxes, delivers ~5.4 t/s, and can't run solo. We publish it because it completes the recipe set for fleet owners — and because the measurement behind it (pools reward lean bytes-per-token over small file size) is one of the core findings of our work.

Credits & license

• Z.AI — GLM-5.3-Flash base model (MIT license; included in repo)
• Unsloth — BF16 source + imatrix
• kingjones30 / charlie12345 — ROCmFPX format & recipes
• ggml-org — llama.cpp
• PaoAI — glm5next port, FP4 recipe, fleet benchmarking

MIT, same as the base model. Not affiliated with Z.AI.