---

license: mit

base_model: inclusionAI/Ling-3.0-tiny-base

base_model_relation: quantized

pipeline_tag: text-generation

library_name: gguf

tags:

- gguf

- llama.cpp

- rocm

- amd

- rocmfp4

- rocmfpx

- strix-halo

- gfx1151

- ryzen-ai-max-395

- radeon-8060s

- mtp

- speculative-decoding

- base-model

- pretrained

- ling

- bailing-hybrid

- quantized

---

> ### 🔧 Runtime: build the ROCmFPX fork below

> Stock llama.cpp will not load this file. You need both the bailing-hybrid architecture

> and the ROCmFP4 tensor types in one tree. Upstream

> charlie12345/ROCmFPX has the ROCmFP4 types but

> not bailing-hybrid. Our fork has both:

>

> kingjones30/ROCmFPX — a fork of charlie12345/ROCmFPX, branch main.

>

> ```bash

> git clone https://github.com/kingjones30/ROCmFPX.git

> cd ROCmFPX

> cmake -B build -DGGML_HIP=ON -DGPU_TARGETS=gfx1151 -DGGML_NATIVE=ON -DCMAKE_BUILD_TYPE=Release

> cmake --build build --target llama-server llama-quantize -j$(nproc)

> ```

>

> Verified 2026-08-27 on gfx1151: clean clone → 0 build errors → llama-server loads a

> bailing-hybrid ROCmFP4 GGUF from this family and generates coherent text.

Ling-3.0-tiny-base — ROCmFP4 for AMD Strix Halo (gfx1151)

4-bit ROCmFP4 quantisation of Ling-3.0-tiny-base, one of the six Ling-3.0 base/training

checkpoints inclusionAI released on 2026-08-20. The multi-token-prediction (MTP) head is preserved.

⚠️ This is a base checkpoint, not an instruct model

Released for continued pretraining, domain adaptation and fine-tuning. Not instruction-tuned.

It ships a chat_template.jinja, but that is a tokenizer asset and does not make the weights

conversational. Prompt it as a text continuation model. For chat, use inclusionAI/Ling-3.0-tiny.

The file

| | |

|---|---|

| ftype | 102 — Q4_0_ROCMFP4_COHERENT |

| size | 4,769,679,040 bytes |

| architecture | bailing-hybrid — hybrid KDA linear attention + MLA |

| tensors | 549 · block_count 25 (24 layers + 1 MTP layer) |

| q_lora_rank | 256 — low-rank compressed queries |

| context | 262,144 |

Head protection, verified in the finished file:

output.weight        Q6_K     1536 x 157184
token_embd.weight    Q6_K     1536 x 157184

tie_word_embeddings is false, so --output-tensor-type does real work here — the COHERENT tier

alone leaves output.weight at 4-bit. Both heads were forced to Q6_K and audited on exact tensor

names after the build.

⚙️ Requires a patched llama.cpp — details

Ling-3.0-tiny sets q_lora_rank: 256, so its MLA layers use a two-stage compressed query

(q_a_proj → RMS norm → q_b_proj). Ling-3.0-flash sets q_lora_rank: null and uses a single

wide q_proj. A bailing-hybrid implementation written against flash therefore cannot load tiny.

The patch, against ROCmFPX, mirrors the existing

DeepSeek-V2 low-rank query path:

| file | change |

|---|---|

| gguf-py/gguf/tensor_mapping.py | map attention.q_a_proj / q_b_proj / q_a_layernorm → ATTN_Q_A / ATTN_Q_B / ATTN_Q_A_NORM |

| gguf-py/gguf/constants.py | add those three tensors to MODEL_ARCH.BAILING_HYBRID (TensorNameMap skips anything not in the arch list) |

| convert_hf_to_gguf.py | emit add_q_lora_rank() when set; the null path is unchanged |

| src/models/bailing-hybrid.cpp | read Q_LORA_RANK as optional; when n_lora_q > 0 create wq_a / wq_b / attn_q_a_norm and run q_a → RMS → q_b at both graph sites, else keep the wide wq |

Because the KV is read as optional, flash GGUFs (which lack it) keep n_lora_q = 0 and take the

original path unchanged. ~51 lines across 5 files.

Verify a build carries it with strings libllama.so | grep bailing-hybrid — the architecture table

lives in the shared library, not the thin CLI binary.

Measured throughput

AMD Ryzen AI Max+ 395, Radeon 8060S (gfx1151), ROCm 7.2.4, 128 GB unified memory.

llama-cli, -dio -ngl 999 -st -c 2048 -n 512 --temp 0 --seed 1234, 3 repetitions.

| config | flags | generation (median) | runs |

|---|---|---:|---|

| no drafter | --spec-type none | 107.2 t/s | 106.8 / 107.6 / 107.2 |

| MTP n-max 3 | --spec-type draft-mtp --spec-draft-ngl 999 --spec-draft-n-max 3 | 115.2 t/s | 114.7 / 115.3 / 115.2 |

+7.5% from MTP. Enable MTP. n-max 3 is worth +7.5% on this checkpoint.

The three Ling-3.0-tiny base checkpoints do not agree on this. Measured on identical hardware

with identical flags: tiny-base +7.5%, tiny-base-midtrain +5.1%, tiny-base-30T

−4.4%. Every comparison was range-disjoint against a 0.2–0.7% baseline spread. The MTP draft

head is trained alongside the model, so its quality is a property of the training checkpoint, not

of the architecture — and a checkpoint can be fully competitive on quality and speed while shipping

a draft head that is a net negative. Measure before enabling it.

Sample output

Continuation from "The history of mathematics begins in ancient times. One of the earliest known":

>  The history of mathematics begins in ancient times. One of the earliest known civilizations to develop a sophisticated mathematical system was the Sumerians of Mesopotamia, who lived in what is now southern Iraq around 3000 BCE. They used a base-60 (sexagesimal) number system, which is still evident in our modern

Not measured

Perplexity is not published for this build. No perplexity figure is quoted because none was

completed. Quality evidence is the coherence check above plus the tensor-level audit.

Provenance

Converted from inclusionAI/Ling-3.0-tiny-base at revision bab7297fa02713af237e378bf21107718b8e0e1a to BF16 GGUF (549 tensors),

then quantised to ftype 102 with --output-tensor-type q6_K. Licence MIT, inherited from the base model.