---

license: apache-2.0

base_model: ibm-granite/granite-4.1-8b

base_model_relation: quantized

tags: [gguf, llama.cpp, rocm, gfx1151, strix-halo, amd, ryzen-ai-max-395, ai-max-395, rocmfpx, granite, ibm]

language: [en]

pipeline_tag: text-generation

---

> ### ⚠️ STOCK llama.cpp WILL NOT LOAD THIS MODEL

> 🚀 39.90 tok/s — ~13% faster than Q4_K_M, ranges disjoint by a wide margin.

Granite-4.1-8B — ROCmFP4 (tier 102 COHERENT) GGUF

A 4-bit ROCmFP4 quantization for AMD gfx1151 (Ryzen AI MAX+ 395 / Strix Halo), quantized

from BF16 GGUF — a lossless source, not a requantization of a lower-bit build.

| | |

|---|---|

| File | granite-4.1-8b-Q4_0_ROCMFP4_COHERENT.gguf |

| Size | 5.162 GB |

| BPW | 4.69 |

| ftype | Q4_0_ROCMFP4_COHERENT (102) |

⛔ Requires a llama.cpp with the ROCmFP4 quant types

Q4_0_ROCMFP4_COHERENT (ftype 102) exists only in

charlie12345/ROCmFPX, not upstream llama.cpp.

Ignore the auto-generated "Use this model" commands above.

---

Measured

Ryzen AI MAX+ 395 (gfx1151, 128 GB unified, ROCm 7.2.4). **Median of 3+, warm-up discarded,

otherwise-idle box.** Correctness at the model's official sampling.

| build | size | decode (median) | range |

|---|---|---|---|

| this build | 5.162 GB | 39.90 | [39.87 – 39.93] |

| Q4_K_M | 5.348 GB | 35.27 | [35.22 – 35.28] |

~13% faster, ranges disjoint (39.87 min vs 35.28 max). Also 3.5% smaller.

Correctness: 17×23 ⇒ ✅ 391 · capital of Japan ⇒ ✅ Tokyo · days in 2024 ⇒ ✅ 366

Per-tensor types (audited in the finished file)

token_embd Q6_K · norms F32 · bulk Q4_0_ROCMFP4 (type 100)

granite is dense — no SSM/MoE state to protect.

---

What was NOT measured

• No perplexity run, and no quality A/B against the baseline or the source. The checks

above are memorized-fact prompts — necessary but not sufficient; a damaged model can pass them.

• No long-context testing.
• No tool-calling evaluation.

Base model licence inherited; credit for the model goes to its authors.