---

license: other

license_name: nvidia-open-model-license

license_link: https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-open-model-license/

language:

- en

- es

- fr

- de

- ja

- it

base_model: nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16

base_model_relation: quantized

pipeline_tag: text-generation

library_name: gguf

tags:

- gguf

- rocmfp4

- quantized

- imatrix

- amd

- rocm

- strix-halo

- moe

- mamba

- nemotron

- nvidia

---

Nemotron-3 Nano 30B-A3B — ROCmFP4 STRIX_LEAN for AMD Strix Halo

ROCmFP4 GGUF quantization of nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B (hybrid Mamba-MoE, nemotron_h_moe, 31.6B params / ~3.5B active), produced for AMD Strix Halo (Ryzen AI Max, gfx1151).

To our knowledge this is the first ROCmFP4-family quant of a hybrid Mamba-MoE architecture — the recipe carries over cleanly (1 of 401 tensors required fallback quantization).

> ⚠️ Stock llama.cpp will reject this file (invalid ggml type 106). It runs on:

> - the kyuz0/amd-strix-halo-toolboxes rocm-7.2.4-rocmfp4 container image (used to make and bench it), or

> - the Hal0ai/Hal0_ROCmFPX llama.cpp fork.

Companion repos: Ornith-1.0-35B · Ornith-1.0-9B · Qwen3.6-35B-A3B

Files

| File | Quant | BPW | Size | Notes |

|---|---|---|---|---|

| Nemotron-3-Nano-30B-A3B-Q4_0_ROCMFP4_STRIX_LEAN.gguf | Q4_0_ROCMFP4_STRIX_LEAN | 4.28 | 16.9 GB | Size-biased Strix recipe (FP4 + Q5_K token embeddings) |

| imatrix_unsloth.gguf_file | — | — | 53 MB | Importance matrix used for calibration (from unsloth, included for reproducibility) |

Measured performance

On AMD Ryzen AI Max+ 395 (Strix Halo, 128 GB unified LPDDR5X, ROCm backend, llama-bench -fa 1 --mmap 0, same box and build for both rows):

| Quant | Size | pp512 | tg128 |

|---|---|---|---|

| Q4_0_ROCMFP4_STRIX_LEAN | 15.7 GiB | 1044.6 ± 20.8 | 82.5 ± 0.1 |

| Q4_K_M (ggml-org) | 22.8 GiB | 1022.5 ± 26.5 | 64.9 ± 0.1 |

+27% decode at 31% smaller. Decode on Strix Halo is bandwidth-bound; the FP4 kernels stream fewer bytes per token and sustain near-peak effective bandwidth on gfx1151.

How it was made

BF16 GGUF source and imatrix from unsloth/Nemotron-3-Nano-30B-A3B-GGUF, quantized with the toolbox image's llama-quantize (build 9219):

llama-quantize --imatrix imatrix_unsloth.gguf_file \
  Nemotron-3-Nano-30B-A3B-BF16-00001-of-00002.gguf \
  Nemotron-3-Nano-30B-A3B-Q4_0_ROCMFP4_STRIX_LEAN.gguf \
  Q4_0_ROCMFP4_STRIX_LEAN

Serving

llama-server -m Nemotron-3-Nano-30B-A3B-Q4_0_ROCMFP4_STRIX_LEAN.gguf \
  -ngl 999 -fa on --jinja -c 65536

Credits

• Base model: NVIDIA — Nemotron-3 Nano 30B-A3B (NVIDIA Open Model License)
• BF16 GGUF conversion + imatrix: unsloth
• ROCmFP4 quant formats: Hal0ai; GPU quantizer build: kyuz0