---

license: apache-2.0

base_model: Zyphra/ZAYA1-8B

base_model_relation: quantized

tags:

- gguf

- llama.cpp

- rocm

- gfx1151

- strix-halo

- amd

- ryzen-ai-max-395

- ai-max-395

- rocmfpx

- zaya

- moe

language:

- en

pipeline_tag: text-generation

---

> ### ⚠️ STOCK llama.cpp WILL NOT LOAD THIS MODEL

> The zaya architecture is not merged upstream (draft PR

> #23112). Ignore the auto-generated

> "Use this model" commands above — use the ROCmFPX patch in patches/.

>

> 📦 4.86 GiB, smaller than Q4_K_M (5.19 GiB), from a 16.52 GiB BF16 source.

> ⚠️ Speed is parity, not a win — see the honest benchmark below.

ZAYA1-8B — ROCmFP4 (tier 102 COHERENT) GGUF

A 4-bit ROCmFP4 quantization of Zyphra/ZAYA1-8B

for AMD gfx1151 (Ryzen AI MAX+ 395 / Strix Halo).

| | |

|---|---|

| File | ZAYA1-8B-Q4_0_ROCMFP4_COHERENT.gguf |

| Size | 4.8629 GiB (5,221,551,424 bytes) |

| BPW | 4.71 |

| ftype | Q4_0_ROCMFP4_COHERENT (102) |

| Source | BF16 GGUF (16.52 GiB) — lossless source, not a requantization |

| sha256 | e663e10d86bd63b1d4cccb5cd3bde5eb0d7d7442f9cd0f5e3857d0fe2739357c |

---

⚠️ Read this before choosing 4-bit for ZAYA1

ZAYA1 is not weight-bandwidth-bound, so 4-bit buys size — not speed. We measured all three

builds on the same machine, median of 3, warm-up discarded, on an otherwise idle box:

| build | size | median decode | range |

|---|---|---|---|

| ROCmFP4 (this) | 4.86 GiB | 15.8 tok/s | [14.1 – 20.49] |

| Q4_K_M | 5.19 GiB | 19.26 tok/s | [15.26 – 19.88] |

| BF16 | 16.52 GiB | 17.0 tok/s | [16.38 – 17.41] |

BF16 reads 3.4× more bytes per token and decodes at the same rate. A bandwidth-bound model

would show roughly 3× separation here; ZAYA1 shows none. The bottleneck is compute — the

Compressed Convolutional Attention grouped-conv path (cca_conv_grp, 10 groups per even layer).

All three ranges overlap, so we make no speed claim in either direction — this build is

neither meaningfully faster nor slower than Q4_K_M. Take it for the 0.32 GiB size saving and

the fact that it fits alongside more models, not for throughput.

Where the time actually goes

We localized decode cost by structurally skipping the CCA grouped convolution and

re-measuring (rocprof is unavailable on this hardware):

| build | baseline | conv skipped | conv share |

|---|---|---|---|

| ROCmFP4 | 25.0 / 24.5 tok/s | 56.7 / 52.7 | ~54% |

| Q4_K_M | 24.9 / 24.5 tok/s | 55.5 | ~56% |

The CCA grouped convolution is roughly 55% of decode time, and it is identical across

quantizations — the conv weights are BF16 in both. Fitting t = fixed + bytes/BW to the BF16 and

4-bit points puts ~37 ms of ~42 ms per token in weight-independent fixed cost, i.e. decode is

~88% independent of the weight format.

⇒ No 4-bit format can make this model faster. A proper grouped-conv kernel could be worth up

to ~2× end-to-end, but that is a llama.cpp kernel/conversion fix, not a quantization one. We

prototyped a single-GEMM rewrite and did not ship it — it was not numerics-identical, and a

speedup that changes output is not a speedup.

Correctness

Official-sampling checks, reading both reasoning_content and content with a 2500-token budget

(ZAYA1 reasons at length — a small cap will make it look broken when it is not):

| check | result |

|---|---|

| 17 × 23 | ✅ 391 |

| capital of Japan | ✅ Tokyo |

| days in 2024 | ✅ 366 |

Per-tensor types (audited in the finished file, 1283 tensors)

| tensor class | type |

|---|---|

| token_embd.weight | Q6_K |

| cca_conv_grp.weight | BF16 — never quantized |

| cca_conv_grp.bias, cca_k_scale | F32 |

| ssm_conv1d.{weight,bias} (80) | F32 |

| ffn_gate_inp router (80) | F32 |

| norms (121) | F32 |

| routed experts, cca_val_proj1/2 | 4-bit |

⚠️ tie_word_embeddings is true on this model, so there is no separate output.weight and

--output-tensor-type is a silent no-op. --token-embedding-type q6_K is the flag that

actually lands — and on a 262,272-token vocabulary that matters a great deal.

cca_conv_grp is excluded from quantization in the patch. Those tensors drive the compressed

convolutional attention and are highly sensitive; other ZAYA1 quantizers exclude them too.

---

What was NOT measured

• No perplexity run, and no quality A/B against Q4_K_M or BF16. The checks above are

memorized-fact prompts — necessary but not sufficient.

• No long-context testing (the model supports 131,072).
• No tool-calling evaluation.
• No coding or reasoning benchmark.

Model

ZayaForCausalLM / zaya. 40 layers, all hybrid · hidden 2048 · vocab 262,272 ·

16 experts, 1 active · ssm_d_conv = 2 · 1283 tensors · context 131,072 ·

tie_word_embeddings: true. ~8.4B total / ~760M active.

Base model licence: Apache-2.0 (inherited). Credit for the model itself goes to

Zyphra.

All quant variants

Three builds of this model, all measured in one session on one box with one binary

(Ryzen AI MAX+ 395, gfx1151, ROCm 7.2.4, ROCmFPX-2809dc5) — so these rows are directly

comparable. Median of 3, warm-up discarded, otherwise-idle box.

| variant | ftype | size | bpw | decode (median) | range | repo |

|---|---|---|---|---|---|---|

| 4-bit COHERENT | 102 | 4.86 GiB | 4.71 | 23.04 | 22.83 – 23.70 | ZAYA1-8B-ROCmFP4-GGUF |

| 8-bit AGENT | 115 | 8.72 GiB | 8.45 | 21.02 | 20.95 – 21.47 | ZAYA1-8B-ROCmFPX-Q8_0-AGENT-GGUF |

| 8-bit plain | 111 | 8.59 GiB | 8.32 | 21.08 | 20.99 – 21.20 | ZAYA1-8B-ROCmFPX-Q8_0-GGUF |

⚠️ Decode is ~88% weight-independent on this architecture (the CCA grouped conv is ~55% of decode). All three builds land within ~10% of each other; the 4-bit is smallest and marginally fastest. No 8-bit or 4-bit format will make this model meaningfully faster.

What AGENT actually changes: it keeps far more tensors at true Q8_0 instead of the

packed 8-bit type — measured in these files, 154 tensors vs 1 tensor. On models with an

MTP draft head that raises draft acceptance and wins ~6%; these two models have no MTP head,

and here the two 8-bit builds are within noise of each other.