kingjones777/Granite-4.1-3B-ROCmFP4-GGUF overview
⚠️ STOCK llama.cpp WILL NOT LOAD THIS MODEL 📦 1.985 GB — 5.5% smaller than Q4 K M , decode within noise. Granite 4.1 3B — ROCmFP4 tier 102 COHERENT GGUF A 4 b…
Runs locally from ~1.85 GB disk (4 GB VRAM class GPUs with llama.cpp / guIDE).
Repository Files & Downloads
| File | Type | Quantization | Size | Link |
|---|---|---|---|---|
| granite-4.1-3b-Q4_0_ROCMFP4_COHERENT.gguf | GGUF | Q4_0_ROCMFP4_COHERENT | 1.85 GB | Download |
Model Details
| Model ID | kingjones777/Granite-4.1-3B-ROCmFP4-GGUF |
|---|---|
| Author | kingjones777 |
| Pipeline | text-generation |
| License | apache-2.0 |
| Base model | ibm-granite/granite-4.1-3b |
| Last modified | 2026-08-16T14:34:22.000Z |
Model README
---
license: apache-2.0
base_model: ibm-granite/granite-4.1-3b
base_model_relation: quantized
tags: [gguf, llama.cpp, rocm, gfx1151, strix-halo, amd, ryzen-ai-max-395, ai-max-395, rocmfpx, granite, ibm]
language: [en]
pipeline_tag: text-generation
---
> ### ⚠️ STOCK llama.cpp WILL NOT LOAD THIS MODEL
> 📦 1.985 GB — 5.5% smaller than Q4_K_M, decode within noise.
Granite-4.1-3B — ROCmFP4 (tier 102 COHERENT) GGUF
A 4-bit ROCmFP4 quantization for AMD gfx1151 (Ryzen AI MAX+ 395 / Strix Halo), quantized
from BF16 GGUF (6.81 GB) — a lossless source, not a requantization of a lower-bit build.
| | |
|---|---|
| File | granite-4.1-3b-Q4_0_ROCMFP4_COHERENT.gguf |
| Size | 1.985 GB |
| BPW | 4.66 |
| ftype | Q4_0_ROCMFP4_COHERENT (102) |
⛔ Requires a llama.cpp with the ROCmFP4 quant types
Q4_0_ROCMFP4_COHERENT (ftype 102) exists only in
charlie12345/ROCmFPX, not upstream llama.cpp.
Ignore the auto-generated "Use this model" commands above.
---
Measured
Ryzen AI MAX+ 395 (gfx1151, 128 GB unified, ROCm 7.2.4). **Median of 3+, warm-up discarded,
otherwise-idle box.** Correctness at the model's official sampling.
| build | size | decode (median) | range |
|---|---|---|---|
| this build | 1.985 GB | 62.08 | [61.17 – 63.84] |
| Q4_K_M | 2.100 GB | 59.37 | [58.18 – 61.70] |
⚠️ Decode ranges OVERLAP — we make no speed claim. The medians differ but the intervals cross, so this is within noise. The size saving is the real result.
Correctness: 17×23 ⇒ ✅ 391 · capital of Japan ⇒ ✅ Tokyo · days in 2024 ⇒ ✅ 366
Per-tensor types (audited in the finished file)
280× Q4_0_ROCMFP4 (type 100) · token_embd Q6_K · 81 norms F32
granite is a dense architecture here — no SSM/MoE state to protect.
---
What was NOT measured
- No perplexity run, and no quality A/B against the baseline or the source. The checks
above are memorized-fact prompts — necessary but not sufficient; a damaged model can pass them.
- No long-context testing.
- No tool-calling evaluation.
Base model licence inherited; credit for the model goes to its authors.
Run kingjones777/Granite-4.1-3B-ROCmFP4-GGUF with guIDE
Download guIDE — the AI-native code editor with local LLM inference and 69 built-in tools.
Source: Hugging Face · Compare models