---

license: apache-2.0

language:

- en

- zh

base_model: Qwen/Qwen3.6-35B-A3B

base_model_relation: quantized

pipeline_tag: image-text-to-text

library_name: gguf

tags:

- gguf

- rocmfp4

- rocmfpx

- quantized

- imatrix

- amd

- rocm

- strix-halo

- gfx1151

- moe

- multimodal

---

Qwen3.6-35B-A3B — ROCmFP4 STRIX_LEAN quant for AMD Strix Halo

Q4_0_ROCMFP4_STRIX_LEAN GGUF quantization of

Qwen/Qwen3.6-35B-A3B

(Qwen3.5-MoE architecture, ~3B active, multimodal), produced for AMD Strix

Halo (Ryzen AI Max, gfx1151).

> ⚠️ These files use ROCmFP4 tensor types (ggml type 106) — stock

> llama.cpp will reject them (invalid ggml type). Use either:

> - kyuz0/amd-strix-halo-toolboxes

> rocm-7.2.4-rocmfp4 image (public, maintained), or

> - the Hal0ai/Hal0_ROCmFPX

> llama.cpp fork.

First published STRIX_LEAN of this model as far as I can tell (a

ROCMFP4_FAST and an MTP-focused ROCmFP4 exist elsewhere — the STRIX_LEAN

recipe trades a hair of file size for the Strix attn-K/V quality layout +

Q5_K token embeddings).

Files

| File | Quant | BPW | Size | Notes |

|---|---|---|---|---|

| Qwen3.6-35B-A3B-Q4_0_ROCMFP4_STRIX_LEAN.gguf | Q4_0_ROCMFP4_STRIX_LEAN | 4.29 | 19.0 GB | Fits a 64 GB carve-out with long context to spare |

| mmproj-BF16.gguf | BF16 | — | 0.90 GB | Vision projector — load alongside the quant |

| imatrix_unsloth.gguf_file | — | — | 184 MB | Importance matrix used for calibration (from unsloth) |

Measured performance

AMD Ryzen AI Max+ 395 (Strix Halo, 128 GB unified LPDDR5X), ROCm backend,

kyuz0 rocm-7.2.4-rocmfp4 toolbox, llama-bench -fa 1 --mmap 0:

| test | tok/s |

|---|---|

| pp512 | 1102 |

| tg128 | 63.1 |

For reference, the same model as UD-Q4_K_XL runs 48 tok/s on identical

hardware — the ROCmFP4 kernels are worth ~+31% decode on Strix Halo.

How it was made

BF16 GGUF source and imatrix from

unsloth/Qwen3.6-35B-A3B-MTP-GGUF:

llama-quantize --imatrix imatrix_unsloth.gguf_file \
  Qwen3.6-35B-A3B-BF16-00001-of-00002.gguf \
  Qwen3.6-35B-A3B-Q4_0_ROCMFP4_STRIX_LEAN.gguf \
  Q4_0_ROCMFP4_STRIX_LEAN

(nextn/MTP metadata from the source survives conversion, though speculative

decoding showed no speedup on Strix Halo in my testing — plain decode is

recommended.)

Serving

llama-server -m Qwen3.6-35B-A3B-Q4_0_ROCMFP4_STRIX_LEAN.gguf \
  --mmproj mmproj-BF16.gguf -ngl 999 -fa on --jinja -c 65536

Companion repos

• gsrunion/Ornith-1.0-35B-ROCmFPX-GGUF
• gsrunion/Ornith-1.0-9B-ROCmFPX-GGUF

Credits

• Base model: Qwen — Qwen3.6-35B-A3B (Apache-2.0)
• BF16 GGUF conversion + imatrix: unsloth
• ROCmFP4 quant formats: Hal0ai/Hal0_ROCmFPX;

maintained builds: kyuz0