---

library_name: gguf

license: apache-2.0

base_model: XingChen-AGI/Xing4.0-29B-A4B

base_model_relation: quantized

pipeline_tag: text-generation

tags:

- gguf

- rocm

- rocmfp4

- strix-halo

- gfx1151

- amd

- moe

- mla

- imatrix

---

Xing4.0-29B-A4B — ROCmFP4 FAST + imatrix (Strix Halo / gfx1151)

The single importance-matrix build of

XingChen-AGI/Xing4.0-29B-A4B, on the

FAST tier — the fastest-decoding and smallest of the three, which is also the tier an

imatrix has the most room to help.

Standard (non-imatrix) tiers:

Xing4.0-29B-A4B-ROCmFP4-GGUF.

⛔ Needs the ROCmFPX fork — stock llama.cpp cannot load this

ROCmFP4 tensor types are ≥ 100 (past stock GGML_TYPE_COUNT), and xing4 is a new

architecture (mHC hyper-connections + MLA with n_embd_head_k=192 / n_embd_head_v=128).

Build from the xing4-port branch of our fork (pinned at cbc2cd5) — main does

not carry xing4:

git clone -b xing4-port https://github.com/kingjones30/ROCmFPX.git

Files

| file | size | PPL | tg128 | pp512 |

|---|---|---|---|---|

| …-FAST-imatrix.gguf | 15.51 GiB | 9.1506 ± 0.280 | 29.69 t/s | 830.8 t/s |

| xing4.imatrix | 88 MB | — | — | — |

The .imatrix is shipped so you can reproduce or re-target the quantization yourself.

What the imatrix actually bought — read this before assuming

| build | PPL | Δ |

|---|---|---|

| FAST, no imatrix | 9.2258 ± 0.282 | — |

| FAST + imatrix | 9.1506 ± 0.280 | −0.8% |

| COHERENT, no imatrix | 9.0639 ± 0.274 | −1.8% vs FAST |

−0.8% is a modest gain, and it does not make FAST the best-quality file in the family.

COHERENT-noimatrix still measures better. Choose this build for speed and size; choose

COHERENT if quality is the priority.

For context, the same pipeline gained −5.9% on a different MoE. The likely reason it is small

here: calibration was English wikitext against a Chinese-first model. A Chinese or mixed

calibration set would probably do better, and the shipped .imatrix plus the recipe below

let you try exactly that.

The imatrix was genuinely applied, not silently ignored — llama-quantize reported

"loaded 552 importance matrix entries … computed on 598 chunks".

Usage

llama-server -m Xing4.0-29B-A4B-Q4_0-ROCmFP4-FAST-imatrix.gguf \
  -c 32768 -ngl 99 -fit off --jinja

⚠️ Reasoning model — budget the tokens

Thinking is on by default. A two-sentence answer took 569 completion tokens;

max_tokens: 200 returns empty content. Use ≥ 600. reasoning_content is separated

correctly by llama-server. The trace is frequently Chinese even for English answers —

expected.

Tool calling

Verified on this exact file:

finish_reason: tool_calls
get_weather({"city":"Tokyo","unit":"celsius"})

Verification

The port was validated against upstream transformers on identical tokens:

| | PPL |

|---|---|

| this port, BF16 | 8.3608 |

| HF transformers bf16, identical tokens | 8.4252 |

⚠️ llama-perplexity scores only the second half of each window; a naive script scoring

from position 1 gives 11.91 for the same model and data. Match the window before comparing.

Not tested

• Contexts beyond 32K (262144 cannot be allocated on 128 GiB hardware — see MTP), and the

Vulkan backend.

MTP: measured, and it is the biggest lever on this model

Layer 40 is the model's MTP block and its weights are in these files (block_count = 41,

nextn_predict_layers = 1). Measured 2026-09-19 on the STRIX_LEAN file, same flags, 589-token

prompt generating 160 tokens, 32K context, quiet box, ROCm0:

| --spec-type | prefill | decode | vs off | mean accepted length |

| --- | ---: | ---: | ---: | ---: |

| off | 454.8 t/s | 12.67 t/s | — | — |

| draft-mtp --spec-draft-n-max 1 | 388.9 t/s | 18.87 t/s | +48.9% | 1.65 |

| draft-mtp --spec-draft-n-max 2 | 411.9 t/s | 14.45 t/s | +14.0% | 1.83 |

Use --spec-draft-n-max 1. n-max 2 accepts slightly longer drafts (1.83 vs 1.65) and is still

much slower, because the extra draft work costs more than it returns. Prefill pays 14.5% for the

gain, so MTP favours generation-heavy traffic over long-prompt batch work.

⛔ MTP costs you half your context

The MTP draft context allocates its own full-size KV cache. Two consequences measured on a

128 GiB Strix Halo:

• The declared 262144 context is not reachable on this hardware at all: the KV allocation asks

for 209,920 MiB (205 GiB) and fails. That is ~0.80 MiB per token, uncompressed.

• With MTP enabled, 131072 still fails (failed to create MTP context) and 32768 is what fits.

⚠️ -ctk q8_0 -ctv q8_0 is silently ignored on this arch. At 131072 with both flags set the

allocation was 104,960 MiB — exactly half the 262144 figure, i.e. pure context scaling with no

quantisation benefit. Do not plan context around quantised KV here.

Reproduction

box     AMD Ryzen AI Max+ 395 "Strix Halo", gfx1151, ROCm 7.2.4, 123 GiB unified
build   github.com/kingjones30/ROCmFPX, arch xing4
source  XingChen-AGI/Xing4.0-29B-A4B -> 936-tensor BF16 GGUF (62.4 GB)

imatrix llama-imatrix -m <BF16>.gguf -f wiki.test.raw -c 512 -ub 64 -ngl 99 -fit off
        -> 552 entries over 598 chunks
quant   llama-quantize --imatrix xing4.imatrix --output-tensor-type q4_K <BF16> <out> 103 32

ppl     llama-perplexity -m <f> -f wiki.test.raw -c 512 --chunks 32 -ub 64 -ngl 99 -fit off
speed   llama-bench -m <f> -p 512 -n 128 -ngl 99 -r 3     (quiet box)

⛔ -ub 64 in the imatrix command is load-bearing, not a preference. On gfx1151 the ROCm

mul_mat_id sorted fallback computes wrong values for unquantized MoE, and the imatrix pass

runs on BF16. Built at the default -ub 512, this model's imatrix produced PPL 318,130;

at -ub 64 it produced a valid one. Reproducer: test-backend-ops -o MUL_MAT_ID →

1311/1320, all 9 failures f16/bf16, all 920 quantized cases pass (so the shipped file

here is unaffected).

Credits

Base model © China Telecom Artificial Intelligence Technology Co., Ltd.

(XingChen-AGI ·

GitHub), Apache-2.0. Quantized and

arch-ported by kingjones777; this GGUF inherits Apache-2.0.