---

license: apache-2.0

base_model: ornith-ai/Ornith-1.5-35B-A3B-GGUF

tags:

- rocmfp4

- strix-halo

- gguf

- moe

- hybrid-attention

- multimodal

- vision

---

Ornith-1.5-35B-A3B ROCmFP4

ROCmFP4 quantization of Ornith 1.5 35B-A3B for AMD Strix Halo (gfx1151) and RDNA 3.5 GPUs, produced with the Q4_0_ROCMFP4_STRIX_LEAN preset (FP16 embedding/norm preservation, nextn MTP head kept at q8_0).

> Revision 2026-08-28: rebuilt from the ornith-ai aug-24 MTP refresh. the retrained MTP head now verifies at 87.98% draft acceptance (mean 3.70/4 positions) for 105.6 tok/s effective decode on strix halo — speculative decoding is now recommended. note: the new MTP head requires this base revision; do not graft it onto older quants.

Quantization Details

| Property | Value |

|---|---|

| Quant format | Q4_0_ROCMFP4_STRIX_LEAN (ROCmFP4) |

| Bits per weight | 4.29 |

| File size | 18.16 GiB |

| SHA256 | 0f907917a1bfe4e0ca0d281e5709dcf34b6277063e94fab29491bb5c80fda696 |

| Vision projector | mmproj-Ornith-1.5-35B-BF16.gguf (0.84 GiB) |

| Projector SHA256 | d9ce31026d1cb1f3f8d5152e2e2a014d9d2b302b6c93a7dc07bb0a0487f52837 |

| Max Context | 262,144 tokens (256K) |

| Source | Ornith-1.5-35B-A3B-GGUF Q8_0 aug-24 MTP refresh (allow-requantize) |

| Notes | MTP head tensors preserved at q8_0 |

Measured Performance (AMD Ryzen AI Max+ 395, Radeon 8060S, Mesa RADV Wave64)

| Configuration | Decode |

|---|---|

| ROCmFP4 bare decode | 76.9 tok/s |

| Q4_K_M baseline | 71.5–71.7 tok/s |

| ROCmFP4 MTP (n4 p0.6, aug-24 refresh) | 105.6 tok/s (87.98% draft acceptance, mean 3.70/4) |

ROCmFP4 is ~7.5% faster and 16.7% smaller than stock Q4_K_M; with MTP speculative decoding it reaches ~137% of bare decode.

Serving

HaloFPX (Text + Vision)

halofpx pull downloads and verifies both the ROCmFP4 weights and BF16 vision projector:

halofpx pull ornith-1.5-35b
halofpx serve
halofpx load ornith-1.5-35b

halofpx list reports model-weight and vision-projector readiness separately.

Direct llama-server Vision

llama-server \
  -m Ornith-1.5-35B-A3B-ROCmFP4.gguf \
  -mm mmproj-Ornith-1.5-35B-BF16.gguf \
  -ngl 99 -c 131072 -fa on --no-mmap

Full 256K Max Context (Strix Halo 128GB UMA):

llama-server \
  -m Ornith-1.5-35B-A3B-ROCmFP4.gguf \
  -ngl 99 -c 262144 -fa on --no-mmap

Multi-Slot / Conservative Memory (131K Context):

llama-server \
  -m Ornith-1.5-35B-A3B-ROCmFP4.gguf \
  -ngl 99 -c 131072 -fa on --no-mmap

> Note: speculative decoding (MTP, --spec-type draft-mtp) is recommended on this model with the aug-24 refresh: measured 87.98% draft acceptance (mean 3.70/4 positions) for 105.6 tok/s effective decode vs 76.9 bare. suggested profile: --spec-type draft-mtp --spec-draft-n-max 4 --spec-draft-p-min 0.6 -fa on.