---

license: mit

base_model: inclusionAI/Ling-3.0-flash

base_model_relation: quantized

pipeline_tag: text-generation

tags:

- gguf

- rocmfp4

- strix-halo

- gfx1151

- ryzen-ai-max

- moe

- bailing-hybrid

- llama-cpp

language:

- en

- zh

---

Ling-3.0-flash — Q4_0_ROCMFP4_STRIX GGUF

**First ROCmFP4 build of Ling-3.0-flash, and the first GGUF quant of any bailing-hybrid model

tuned for Ryzen AI Max+ 395 (Strix Halo, gfx1151).**

I run a rack of Strix Halo boxes and I build ROCmFP4 quants for them because the stock Q4_K_M

leaves real speed on the table. This one is +23.3% faster decode and 9% smaller than Q4_K_M,

measured on my own hardware with disjoint ranges — not a guess.

Ling-3.0-flash is InclusionAI's 127.5B-total / 5.1B-active MoE (512 experts, 8 active, MIT).

That tiny active-parameter count is why it moves so well on a bandwidth-bound iGPU.

Measured on Ryzen AI Max+ 395 (128GB, gfx1151, ROCm 7.2.4)

7 runs each, same binary, same flags, spec decoding OFF (a stable instrument — 0.5% spread):

| | Q4_K_M | this build |

|---|---:|---:|

| decode | 29.34 t/s | 36.18 t/s |

| gain | — | +23.3%, ranges DISJOINT |

| size | 69.70 GiB | 63.40 GiB (−9%) |

| BPW | ~4.8 | 4.27 |

| headroom on 128GB | — | ~57 GiB free with model resident |

-c 32768 is free. 32K context measured the same as 8K. Use it.

⚠️ Read this before you run it: use greedy or low temperature

This model injects junk tokens into output when you sample too loosely. I measured it, 30 responses

per config, executed-code battery:

| sampling | code pass | responses with junk tokens |

|---|---:|---:|

| greedy (temp 0) | 21/30 | 0/30 ✅ |

| temp 0.3, no top_k | 20/30 | 6/30 (20%) ❌ |

| temp 0.6 + top_k 20 (the official recipe) | 16/30 | 4/30 (13%) ❌ |

| temp 0.3 + top_k 20 | 6/10 | 4/10 ❌ |

top_k does not save you — low temperature does. At temp 0.3 you get Chinese characters and

phantom identifiers dropped into ASCII code (潜水, oly, ulp, eville). At greedy: zero.

Recommended: --temp 0.0 --top-k 1. And give it generous max_tokens (1500+) — it is verbose

and will get truncated mid-function otherwise.

Honest note on quality

There is a residual defect in this model that is not specific to my build: roughly 1–2 times per

250 generated tokens the output distribution flattens for a single step and a junk token can win,

then it recovers. I chased this properly:

• it reproduces bit-for-bit on CPU (-ngl 0), so it is not a GPU kernel issue
• another model on the identical stack (Hunyuan-3 295B) is completely clean, 0/1040 tokens
• I converted the official BF16 myself and it still happens — so it is not the upstream

quantizer's fault either

It lives somewhere in the shared bailing-hybrid GGUF conversion/graph path and affects every

Ling GGUF I have tested, including the standard Q4_K_M. At greedy it is mostly invisible; code

quality here measures the same as Q4_K_M. I would rather tell you it exists than let you find it.

Requirements

This is not a stock llama.cpp quant. You need the ROCmFPX fork

(github.com/charlie12345/ROCmFPX) plus a bailing-hybrid arch port — the architecture is not in

upstream llama.cpp yet (PR #26608 is still open).

env LD_LIBRARY_PATH=<rocmfpx>/build/bin:/opt/rocm/lib \
    HSA_OVERRIDE_GFX_VERSION=11.5.1 \
    GGML_HIP_ENABLE_UNIFIED_MEMORY=1 \
  llama-server -m Ling-3.0-flash-Q4_0_ROCMFP4_STRIX.gguf \
    -ngl 999 -fa on -dio --no-warmup --jinja \
    -c 32768 --parallel 1 \
    --temp 0.0 --top-k 1

-dio is mandatory at this size — without it the mmap path stalls and never becomes healthy.

Things I tested so you don't have to

• MTP / speculative decoding: leave it off. The head is near-perfect (100% draft acceptance) but

exact verification on this stack forces single-row decode, which costs more than the drafts save.

Honest measurement: 0.83× — slower. The lenient mode looks faster but silently changes your

output; I verified that with a byte-identical check against no-spec decoding.

• Q4_0_ROCMFP4_STRIX (type 105), not _STRIX_LEAN (106). LEAN's cheaper token embeddings cost

real accuracy elsewhere in my testing for ~0.5 GiB of disk.

• Quantized from the Q8_0 source, never from a 4-bit intermediate.

License

MIT, inherited from inclusionAI/Ling-3.0-flash. All credit to InclusionAI for the model.