---

base_model: inclusionAI/Ling-3.0-flash

base_model_relation: quantized

license: mit

license_link: https://huggingface.co/inclusionAI/Ling-3.0-flash/blob/main/LICENSE

language:

- en

- zh

pipeline_tag: text-generation

library_name: llama.cpp

tags:

- gguf

- quantized

- imatrix

- bailingmoe3

- moe

---

Ling-3.0-flash: Atomic Dynamic Q4_K_S and Q5_K_M, rebuilt for upstream llama.cpp

A GGUF quantization of inclusionAI/Ling-3.0-flash

that uses **AtomicChat's "Atomic Dynamic" (AD) bit-placement

recipe and AtomicChat's published importance matrix**, but is built from an

upstream-format source so that it loads on stock, unmodified llama.cpp.

| | AD-Q4_K_S | AD-Q5_K_M |

|---|---|---|

| Folder | Ling-3.0-flash-AD-Q4_K_S/ | Ling-3.0-flash-AD-Q5_K_M/ |

| Shards | 2 | 3 |

| Unsharded size | 75,981,701,568 bytes (70.76 GiB) | 91,576,648,128 bytes (85.29 GiB) |

| Bits per weight | 4.77 | 5.75 |

| Architecture | bailingmoe3, 938 tensors, 43 blocks | bailingmoe3, 938 tensors, 43 blocks |

| Base ftype | Q4_K_S, with 8 per-tensor rules | Q5_K_M, with 8 per-tensor rules |

| Rule file | tensor-types-q4ks.txt | tensor-types-q5km.txt |

| Loads on | stock llama.cpp v0.5.0 (build b11146) and later, with no fork required | same |

---

Credit where it is due

The recipe in these files is not mine. It is AtomicChat's.

The bit-placement design (routers and expert bias kept at F32, attention and KDA gates at

Q8_0, output at F16, token_embd at Q8_0, extra bits for the edge MoE blocks, and only the

routed experts squeezed) is AtomicChat's "Atomic Dynamic" line,

published at:

> ### → AtomicChat/Ling-3.0-flash-GGUF

The importance matrix used to quantize these files is also theirs, and used unmodified. If you

find this quant useful, the design credit belongs to them, and their repo is the one to follow.

Why these files exist

AtomicChat's own AD- files were converted about **two weeks (11 to 13 days) before upstream

llama.cpp gained bailingmoe3 support**. Their card says so plainly ("the files need a

TurboQuant build"). That was entirely correct when published, as their fork shipped

BailingMoeV3 around 2026-08-04, and upstream's support PR did not close until 2026-08-17.

The practical consequence today is that their files do not load on an upstream build. If you

try, you will see this:

check_tensor_dims: tensor 'blk.0.ssm_f_a.weight' not found

This is a structural rejection by the upstream bailingmoe3 loader, and it is

backend-independent. If you have hit that error, this repo is what you want.

This repo is the same bit placement, rebuilt against an upstream-compatible source, for people

who want the AD bit placement without running a fork. It was checked tensor by tensor against

AtomicChat's published AD-Q4_K_S and AD-Q5_K_M files: all 917 tensors they carry have the same

type and shape here (see "How the match was checked"). Nothing here is a criticism of

AtomicChat's work. It is a repackaging of it, for a llama.cpp that did not exist yet when

they shipped.

---

Provenance

Everything below is recorded so the file can be rebuilt bit-for-bit.

Base model

inclusionAI/Ling-3.0-flash, MIT

licence. 124B total parameters, 5.1B active. Hybrid linear attention: 35 KDA blocks

interleaved 5:1 with 7 gated MLA blocks, over a 512-expert MoE.

Source weights

| | |

|---|---|

| Repo | bartowski/Ling-3.0-flash-GGUF |

| Revision | 7e1a1243566501d52af98e9aa7a7a61986279c6d |

| Directory | Ling-3.0-flash-bf16/ |

| Shards | 7 |

| Bytes | 255,091,083,264 (237.57 GiB) |

Already bf16 GGUF in upstream format, so no HF→GGUF conversion step was needed, and

this is the whole point: the conversion is where AtomicChat's files and upstream diverge.

Importance matrix

| | |

|---|---|

| Repo | AtomicChat/Ling-3.0-flash-GGUF |

| Revision | 253738fe190c15f329001f263f355fc1562bbe7c |

| Path | imatrix/Ling-3.0-flash.imatrix.gguf |

| Bytes | 487,748,480 (465.15 MiB) |

| sha256 | 7d3c0ebe9eb235cc08e0b7c91886f5c422772ea95eebbf0f53b0974a1c040991 |

Quantizer

| | |

|---|---|

| Image | ghcr.io/ggml-org/llama.cpp:full-v0.5.0 |

| Digest | sha256:386ead2afc762e8ca7d2e65d6cd8b031d9a64ffcf989bce815c1614d7ef2afeb |

| Build | b11146, commit 7fe450e19 |

| Wall clock | AD-Q4_K_S 4,759.0 s (79.3 min); AD-Q5_K_M 4,691.3 s (78.2 min); 6 threads each, two builds running side by side on one node with 12 threads (6 cores, SMT) |

---

Reproducing these files

1. The per-tensor rule files

Both are included in this repo. The order of the rules is load-bearing: llama.cpp uses

std::regex_search and the first match wins, and any manual match bypasses the built-in k-quant

mixture for that tensor entirely. The placement was read from the tensor tables of AtomicChat's own

GGUF files (HTTP Range reads of the headers, revision 253738fe190c15f329001f263f355fc1562bbe7c);

AtomicChat publish no recipe file.

tensor-types-q4ks.txt (base Q4_K_S):

^token_embd\.weight$=q8_0
^blk\.(2|3|39|40|41)\.ffn_(down|gate|up)_exps\.weight$=q5_k
^blk\.\d+\.ffn_(down|gate|up)_exps\.weight$=q4_k
^blk\.\d+\.attn_(q|k|v|output|gate|k_b|v_b|kv_a_mqa)\.weight$=q8_0
^blk\.\d+\.ssm_(f_a|g_a|beta)\.weight$=q8_0
^blk\.\d+\.ffn_(down|gate|up)_shexp\.weight$=q8_0
^blk\.\d+\.ffn_(down|gate|up)\.weight$=q8_0
^blk\.\d+\.nextn\.eh_proj\.weight$=q8_0

tensor-types-q5km.txt (base Q5_K_M) differs only in the two expert rules, which move up one step:

^token_embd\.weight$=q8_0
^blk\.(2|3|39|40|41)\.ffn_(down|gate|up)_exps\.weight$=q6_k
^blk\.\d+\.ffn_(down|gate|up)_exps\.weight$=q5_k
(the remaining five rules are identical)

In both files, edge blocks 2, 3, 39, 40 and 41 get one step more than the other routed-expert blocks,

and down, gate and up get the same type. Norms, routers (ffn_gate_inp), exp_probs_b and

the ssm_* tensors (small, norm or conv tensors that llama.cpp never quantizes) come out as F32 with no rule, and output.weight is F16 by the flag below.

sha256 of the rule files: tensor-types-q4ks.txt d854d6f53dd34d0e96060e4cd083604840474ad04dbfdc55c5eacb7b2c6d4711,

tensor-types-q5km.txt a0d82ffdd269bcd000b15bb34d330ac3ba165c9b5467aac7d0a59305d7129434.

2. The quantize invocation

Verbatim, as run (Q4_K_S shown; for the other file use tensor-types-q5km.txt, an output name

with AD-Q5_K_M, and Q5_K_M):

llama-quantize \
  --imatrix            Ling-3.0-flash.imatrix.gguf \
  --output-tensor-type f16 \
  --tensor-type-file   tensor-types-q4ks.txt \
  --max-buffer-size    2048 \
  Ling-3.0-flash-bf16-00001-of-00007.gguf \
  Ling-3.0-flash-AD-Q4_K_S-upstream.gguf \
  Q4_K_S \
  6

6 is the thread count. token_embd.weight is set to Q8_0 by the first rule (the --token-embedding-type q8_0

flag is equivalent).

3. The split

llama-gguf-split --split --split-max-size 45G \
  Ling-3.0-flash-AD-Q4_K_S-upstream.gguf \
  Ling-3.0-flash-AD-Q4_K_S

llama-gguf-split --split --split-max-size 32G \
  Ling-3.0-flash-AD-Q5_K_M-upstream.gguf \
  Ling-3.0-flash-AD-Q5_K_M

--split-max-size units are decimal (45G = 45,000,000,000 bytes). This keeps every shard

under Hugging Face's 50 GB per-file limit.

How the match was checked

The tensor tables (name, type, shape) of the shards in this repo were read with the same HTTP Range

method as AtomicChat's, and compared one tensor at a time. Result for both files: all 917

tensors that AtomicChat publish are present here with the same type and the same shape (0 type

mismatches, 0 shape mismatches). The only name difference is that their fork calls the KDA tensors

ssm_f.weight/ssm_g.weight, where upstream says ssm_f_a/ssm_g_a; those were compared as the same tensor.

The 21 extra tensors in these files are the NextN block (below).

---

Composition

AD-Q4_K_S (measured from the tensor table):

| type | tensors | GiB | share |

|---|---:|---:|---:|

| Q4_K | 108 | 56.95 | 80.5% |

| Q5_K | 15 | 9.67 | 13.7% |

| Q8_0 | 424 | 3.18 | 4.5% |

| F16 | 1 | 0.75 | 1.1% |

| F32 | 390 | 0.21 | 0.3% |

AD-Q5_K_M (measured from the tensor table):

| type | tensors | GiB | share |

|---|---:|---:|---:|

| Q5_K | 108 | 69.61 | 81.6% |

| Q6_K | 15 | 11.54 | 13.5% |

| Q8_0 | 424 | 3.18 | 3.7% |

| F16 | 1 | 0.75 | 0.9% |

| F32 | 390 | 0.21 | 0.2% |

That table looks lopsided until you notice 424 Q8_0 tensors occupy about 3 GiB. All the

attention projections, the KDA gates, every shared expert and token_embd together are only about 3% of

the weights, so keeping them at 8 bits is nearly free. The routed experts are

97.12% of the parameters (measured: 123.816B of 127.486B; AtomicChat's card states

97.1%), and they are the only thing this recipe actually squeezes.

For scale against neighbouring builds of the same model:

| build | bytes | GiB | tensors | vs AD-Q4_K_S here |

|---|---:|---:|---:|---:|

| AD-Q4_K_S here | 75,981,701,568 | 70.76 | 938 | n/a |

| AD-Q5_K_M here | 91,576,648,128 | 85.29 | 938 | +20.5% |

| AtomicChat AD-Q4_K_S | 74,223,552,640 | 69.13 | 917 (no NextN) | -2.31% |

| AtomicChat AD-Q5_K_M | 89,441,011,840 | 83.30 | 917 (no NextN) | +17.7% |

| bartowski Q4_K_S | 75,049,688,768 | 69.90 | 938 | -1.23% |

| bartowski IQ4_XS | 68,739,201,216 | 64.02 | 938 | -9.53% |

Each file here is about 2.4% larger than AtomicChat's file of the same name. The whole difference is

the NextN block (21 tensors, about 1.76 GB in AD-Q4_K_S and about 2.14 GB in AD-Q5_K_M), which their converter drops and the upstream source keeps.

On the 917 tensors both files carry, the types and shapes are identical.

---

Files

Ling-3.0-flash-AD-Q4_K_S/

| file | bytes | GiB | tensors | sha256 |

|---|---:|---:|---:|---|

| Ling-3.0-flash-AD-Q4_K_S-00001-of-00002.gguf | 44,797,227,616 | 41.72 | 573 | 2be134529a49adde75f144164c2e1a57533b39f0bdf7beab5337461b30ae3c9f |

| Ling-3.0-flash-AD-Q4_K_S-00002-of-00002.gguf | 31,184,474,144 | 29.04 | 365 | ab85ee0989690904c47d7098a140524ed46653614c912dae0490c04c273a763d |

Ling-3.0-flash-AD-Q5_K_M/

| file | bytes | GiB | tensors | sha256 |

|---|---:|---:|---:|---|

| Ling-3.0-flash-AD-Q5_K_M-00001-of-00003.gguf | 31,660,549,472 | 29.49 | 338 | b16ca636ddbdc4d9c43b429e4989efbeb67cfc34567f995895278f8a5f580e23 |

| Ling-3.0-flash-AD-Q5_K_M-00002-of-00003.gguf | 31,523,973,888 | 29.36 | 323 | e851bf4907c987a5999adaa0f9d28ba6f28701f8404e3becdb545755cff05e04 |

| Ling-3.0-flash-AD-Q5_K_M-00003-of-00003.gguf | 28,392,125,088 | 26.44 | 277 | 63916691d10600f2986f7872e6b9cd4a777c220533bb728b29391194eb23c384 |

The shards of AD-Q4_K_S sum to 192 bytes more than the unsharded file, and those of AD-Q5_K_M to 320 bytes more,

because gguf-split adds its split metadata and a header to every shard. Their tensor names together are

exactly the original 938, in the original order, with no duplicates.

The unsharded files, for reference: AD-Q4_K_S is 75,981,701,568 bytes with sha256

5210843db2e5b15784d6379d6c9aed80b7db99fb7b43f4ef7723778dc99a2d8c; AD-Q5_K_M is

91,576,648,128 bytes with sha256 0f05755aa0141ba9b174d30e02e8a007d2c53b738d60453c8e762782ca9da047.

They are not uploaded here, because both exceed the Hugging Face 50 GB per-file limit.

You can recombine the shards with llama-gguf-split --merge if you need to verify them, or

just point llama-server at shard 1; it follows the split automatically.

---

Usage

llama-server \
  --model Ling-3.0-flash-AD-Q5_K_M-00001-of-00003.gguf \
  --ctx-size 131072 \
  --jinja

Use the first shard of whichever file you downloaded (...-AD-Q4_K_S-00001-of-00002.gguf or

...-AD-Q5_K_M-00001-of-00003.gguf). Pass only the first shard. llama.cpp reads split.count from its metadata and opens the

rest itself.

Requires llama.cpp v0.5.0 / b11146 or newer. v0.5.0 is also the first release with the

dedicated Ling 3.0 (Bailing V3) chat parser (upstream #28682, merged 2026-09-19), so tool

calling works correctly there.

---

Performance

Measured with llama-bench (build b11146, Vulkan/RADV) on an AMD Ryzen AI Max+ 395

(Strix Halo, Radeon 8060S, gfx1151, 128 GB unified memory). Both files ran on the same

machine, one at a time, with nothing else on the GPU:

llama-bench -m <shard 1> -p 2048 -n 128 -d 0,32768,65536,131072 -r 2 -fa 1 -ngl 999 -ctk <kv> -ctv <kv>

Prefill is pp2048 and decode is tg128, both in tokens/s, at the given depth of context

already in the cache. Each value is the mean of 2 repetitions.

KV cache f16:

| depth | AD-Q4_K_S prefill | AD-Q4_K_S decode | AD-Q5_K_M prefill | AD-Q5_K_M decode |

|---:|---:|---:|---:|---:|

| 0 | 499.6 | 34.11 | 450.7 | 32.95 |

| 32,768 | 279.3 | 28.28 | 264.5 | 27.44 |

| 65,536 | 194.1 | 24.18 | 186.4 | 23.54 |

| 131,072 | 120.8 | 18.77 | 118.3 | 18.41 |

KV cache q8_0:

| depth | AD-Q4_K_S prefill | AD-Q4_K_S decode | AD-Q5_K_M prefill | AD-Q5_K_M decode |

|---:|---:|---:|---:|---:|

| 0 | 482.2 | 33.99 | 436.9 | 32.80 |

| 32,768 | 167.3 | 25.00 | 161.8 | 24.36 |

| 65,536 | 74.5 | 19.64 | 75.0 | 19.25 |

| 131,072 | 40.4 | 13.66 | 40.2 | 13.48 |

Peak GPU memory (GTT) during each run, at 133,120 tokens of context: AD-Q4_K_S 70.2 GiB

(q8_0) and 70.6 GiB (f16); AD-Q5_K_M 84.4 GiB and 84.8 GiB.

On this hardware AD-Q5_K_M costs 1 to 4% of decode speed and up to 10% of prefill for

about 14 GiB more memory. A q8_0 KV cache saves little memory here and is much slower at long

context, so use f16.

What is not measured

There is no quality benchmark for these files. They have not been evaluated against

bartowski's plain IQ4_XS, and they have not been evaluated against AtomicChat's own AD-

files. Those cannot run on the stock build at all, which is why this repo exists and also

why a head-to-head is not available.

Nothing here should be read as a measured quality win over any other quant of

this model. AtomicChat publish a KL table for

their AD line, but their controls are their own stock builds, so that result does not

transfer to these files either.

What is verified is that both files load on stock llama.cpp (a CPU-only smoke test on build

b11146), and that their tensor types and shapes match AtomicChat's own files. AD-Q4_K_S answered both smoke

prompts correctly. AD-Q5_K_M hit the 120-token limit mid-answer on the first (391 appeared only in

reasoning_content) and answered the second (a Fibonacci function) correctly.

AtomicChat's KL numbers for AD-Q5_K_M versus AD-Q4_K_S were measured on their fork and do not

transfer to these files until measured here.

The NextN/MTP block

These files carry the NextN / multi-token-prediction block (blk.42, 21 tensors, about 1.76 GB in AD-Q4_K_S and about 2.14 GB in AD-Q5_K_M),

because the bartowski source does. AtomicChat's own files do not: they have 917 tensors, and their

card says the converter drops this block ("MTP / speculative decoding is not wired up"). The

block's expert tensors are quantized with the same rule as the other routed experts (Q4_K in

AD-Q4_K_S, Q5_K in AD-Q5_K_M). llama.cpp ignores the block unless you pass

--spec-type draft-mtp, and logs 21 unused tensor warnings on load. **Those warnings are

harmless**. Nothing is wrong with the file.

If you are tempted to turn MTP on: on the hardware above, a controlled single-variable test

measured decode −49% at 57.7k and −64% at 115.7k of context versus MTP off, despite

healthy draft acceptance rates (0.55 and 0.50). That is a property of this llama.cpp version

on this hardware, not a property of the quant, and it may well not reproduce on a

discrete GPU. But it is the reason MTP is left off where these files are deployed.

---

Licence

MIT, inherited from inclusionAI/Ling-3.0-flash.