---

license: apache-2.0

base_model: Qwen/Qwen3-Next-80B-A3B-Instruct

base_model_relation: quantized

tags:

- gguf

- rocm

- rocmfp4

- strix-halo

- ryzen-ai-max

- qwen3-next

---

Qwen3-Next-80B-A3B-Instruct Q4_0_ROCMFP4_STRIX (GGUF)

First public ROCmFP4 quant of Qwen3-Next-80B-A3B-Instruct for AMD Ryzen AI Max+ 395

(gfx1151 / Radeon 8060S).

> ⚠️ Not compatible with upstream llama.cpp. Requires the

> charlie12345/ROCmFPX fork built with HIP + ROCmFP4 kernels.

Files

| File | Size | Notes |

|------|------|-------|

| Qwen3-Next-80B-A3B-Instruct-Q4_0_ROCMFP4_STRIX.gguf | 39.69 GiB | 4.28 BPW (quantize report) |

Base model: Qwen/Qwen3-Next-80B-A3B-Instruct

BF16 source: unsloth/Qwen3-Next-80B-A3B-Instruct-GGUF

BF16/ 4 shards, 148.51 GiB total (16.01 BPW)

Hardware / stack (validated)

• Ryzen AI Max+ 395, gfx1151, 128 GB unified
• ROCm 7.2.4
• Fork: charlie12345/ROCmFPX @ b41ce12

Build recipe

llama-quantize \
  Qwen3-Next-80B-A3B-Instruct-BF16-00001-of-00004.gguf \
  Qwen3-Next-80B-A3B-Instruct-Q4_0_ROCMFP4_STRIX.gguf \
  Q4_0_ROCMFP4_STRIX

Q4_0_ROCMFP4_STRIX (type 105) is the Strix Halo attention-K/V quality recipe.

Dry-run predicted 40641.96 MiB @ 4.28 BPW and the output matched exactly.

Serving

llama-server --host 127.0.0.1 --port 8080 \
  --model Qwen3-Next-80B-A3B-Instruct-Q4_0_ROCMFP4_STRIX.gguf \
  -dev ROCm0 -ngl 999 -fa on --no-mmap \
  --ctx-size 65536 --parallel 1 -b 2048 -ub 1024 -t 16 --poll 50 --jinja

Use the chat endpoint (/v1/chat/completions). Raw /completion with a bare instruction makes this

instruct model degenerate into repetition loops.

Measured

Against the UD-Q4_K_XL GGUF of the same model on the same machine, same flags, chat endpoint:

| | this quant | UD-Q4_K_XL |

|---|---:|---:|

| size | 39.69 GiB | 42.90 GiB |

| tok/s (median) | 45.7 – 47.6 | 42.6 |

| quality battery | 24 / 24 | 24 / 24 |

The battery is 24 items: 8 code tasks graded by executing the generated function against

assertions, 8 long-tail factual questions, 4 multilingual, 4 maths. Both quants scored 24/24, so

quality is at parity and the size and throughput gains come for free.

Throughput caveat: the two arms were measured with different numbers of co-resident models, so treat

the speed delta as directionally real but not precisely quantified.

Speculative decoding

Not available. The published MTP head

(yomaytk/Qwen3-Next-80B-A3B-Instruct-MTP-HEAD-GGUF)

cannot currently be used with llama.cpp:

• It is an MTP-only GGUF, so it will not load via -md (that expects a complete draft model) —

it fails with missing tensor 'blk.0.attn_norm.weight'.

• Grafting its 20 blk.48.* tensors into the target (block_count 48 → 49, plus

qwen3next.nextn_predict_layers = 1) produces a structurally correct GGUF that still will not

load: missing tensor 'blk.48.ssm_conv1d.weight'.

The reason is architectural. Qwen3-Next is a hybrid — every 4th layer is full attention

(indices 3, 7, 11 … 47) and the other 36 are Gated DeltaNet / SSM. Layer 48 lands on

48 % 4 == 0, so the loader types it SSM and demands ssm_conv1d, while an MTP layer is

attention-shaped. llama.cpp models nextn generally (n_layer_all - n_layer_nextn exists) but does

not exempt the nextn layer from this hybrid pattern.

ngram-map-k was also measured and came out at 1.065× — below a 1.15× ship gate.

License

Follow the base model (Qwen/Qwen3-Next-80B-A3B-Instruct) license terms.