---

license: apache-2.0

base_model: XHToken/Spark-X2.5-4B

base_model_relation: quantized

tags:

- gguf

- llama.cpp

- spark2_5

pipeline_tag: text-generation

---

Spark-X2.5-4B — GGUF

Full-precision GGUF conversion of XHToken/Spark-X2.5-4B.

⚠️ Requires a patched llama.cpp — upstream will NOT load this

spark2_5 is not in upstream llama.cpp. Loading this file with a stock build fails at the

architecture check. Build from the branch that adds it:

git clone -b model/Spark2_5 https://github.com/brywil/llama.cpp
cd llama.cpp && cmake -B build -DGGML_CUDA=ON && cmake --build build -j
./build/bin/llama-cli -m Spark-X2.5-4B-BF16.gguf --ctx-size 32768 -ngl 999 -p "..."

Based on upstream 6d0549831. The branch adds the arch, a converter, and a new BPE pre-tokenizer

type; nothing else is changed.

Always pass --ctx-size. The model declares max_position_embeddings: 1048576, and llama.cpp

defaults to the trained context — that will try to allocate the full 1 M window.

What's here

| file | size | bpw | tensor types |

|---|---|---|---|

| Spark-X2.5-4B-BF16.gguf | 7.67 GiB | 16.0 | reference |

| Spark-X2.5-4B-Q8_0.gguf | 4.08 GiB | 8.52 | F32 109 · Q8_0 253 |

| Spark-X2.5-4B-Q6_K.gguf | 3.15 GiB | 6.58 | F32 109 · Q6_K 253 |

| Spark-X2.5-4B-Q5_K_M.gguf | 2.75 GiB | 5.74 | F32 109 · Q6_K 37 · Q5_K 216 |

| Spark-X2.5-4B-Q4_K_M.gguf | 2.37 GiB | 4.95 | F32 109 · Q6_K 37 · Q4_K 216 |

| spark-x25-4b.imatrix | 4.1 MB | — | importance matrix used for the K-quants |

All 362 tensors present in every file, attn_gate F32 in all of them, and **zero block-32

substitutions** — verified by reading the type histogram out of the shipped bytes rather than

trusting the filename.

The bf16 is published deliberately: the arch needed a fork to run at all, so it is the reference

artifact to check a quant against.

Which one

Q5_K_M is the default recommendation — 2.75 GiB fits an 8 GB card with room for real context,

and 128k of KV costs only another 4.5 GiB. Q4_K_M if you are tight on VRAM. Q8_0 if you

want a near-reference artifact without carrying 7.67 GiB.

The imatrix was built from a code-inclusive mixed corpus, 1,000 chunks × 512 tokens = 512k tokens,

chosen over a prose-only set because the intended workload is agentic coding.

The model

4.11 B dense, 36 layers, apache-2.0. Interleaved sliding-window attention — 27 sliding (window

512) and 9 full, in a strict sssF pattern. 16 query heads, 4 KV heads, head_dim 256, hidden

2560, GELU FFN, 131,072 vocab, tied embeddings. It is a reasoning model and emits a thinking

block.

Long context is cheap here, by design

Only the 9 full-attention layers scale with context; the 27 sliding layers are bounded by their

512 window.

9 layers x 4 KV heads x 256 head_dim x 2 (K+V) x 2 bytes = 36 KiB/token

| context | KV (f16) |

|---|---|

| 32k | 1.1 GiB |

| 128k | 4.5 GiB |

| 256k | 9.0 GiB |

| 1M | 36 GiB |

So 128k of context costs less KV than the weights do. Note this is a memory calculation, not a

retrieval claim — long-context quality is unmeasured here.

Notes for anyone quantizing this

• Every dimension is divisible by 256 (hidden 2560, intermediate 10240, vocab 131072, KV dim

1024), so all QK_K types are legal. A file labelled Q4_K_M really will be Q4_K_M — unlike

archs with an awkward dimension, where llama-quantize silently substitutes block-32 types.

• Pin attn_gate to F32. g_proj is Linear(2560, 16) — 40,960 params per layer, 1.47 M

total, about 5.9 MB. Its sigmoid output multiplies the attention result, so error there

compounds along the sequence instead of adding to one layer's output, which is exactly the case

where per-tensor weight error under-reports the true cost. At this size the bits are free. The

converter on the branch above emits it as F32 — but **llama-quantize will still quantize it

unasked*, because ne0 is 2560 and therefore 256-divisible, so Q5_K is perfectly legal* on

[2560,16]. Legal is not correct here. Pass it explicitly:

```

./build/bin/llama-quantize --imatrix spark-x25-4b.imatrix --tensor-type attn_gate=f32 \

Spark-X2.5-4B-BF16.gguf out.gguf Q5_K_M

```

That is how every quant in this repo was built. Cost: 5.9 MB on a 2.7 GiB file.

Verification

Greedy generation from this file is byte-identical to the HF reference (transformers 4.57.1)

over 24 tokens on "The capital of France is Paris. The capital of Japan is" — a span crossing

six full-attention and eighteen sliding layers, so both rope configurations and the attention gate

are exercised. That check matters more than usual on this arch: it uses two different rope setups

selected per layer type (sliding layers rotate 256 of 256 dims at theta 10,000; full-attention

layers rotate only 64 of 256 at theta 5,000,000), and applying one rope to every layer yields a

model that loads and produces fluent, wrong output with no error anywhere.

Measured

Throughput (GB10 / DGX Spark, unified memory, ~123 GB/s achieved)

| arm | decode | prefill | conditions |

|---|---|---|---|

| Q5_K_M | 48.3 t/s (p50) | 4,482 t/s (p50) | single slot, n=970 decode samples |

| BF16 | 18.6 t/s (p50) | 2,498 t/s (p50) | 4 concurrent slots, per-slot, n=25,724 |

Do not compute a ratio from those two rows — they were served under different slot counts, and

the bf16 figure is per-slot under 4-way batching (~74 t/s aggregate). The clean statement is that

decode here is bandwidth-bound: 2.75 GiB of weights at ~123 GB/s predicts ~45 t/s, and Q5_K_M

measures 48.3.

Agentic coding, 42 tasks (bf16)

Driven through a real opencode agentic loop with tool calling, graded by a fresh pytest per

task — not a multiple-choice benchmark.

| suite | tasks | valid | passed | mean output tokens |

|---|---|---|---|---|

| general | 25 | 22 | 22/22 | 6,120 |

| hard | 5 | 3 | 3/3 | 6,638 |

| frontier | 9 | 4 | 4/4 | 22,340 |

| tier4a | 3 | 3 | 3/3 | 5,915 |

32 of 32 gradeable tasks passed. The 10 excluded tasks were harness limits, not wrong answers:

8 were cut off by a 32,768-token output cap and 2 by the wall clock, with no degeneration detected

in any of them. They are reported as void rather than failed because the model was still working

when the harness stopped it. Treat 32/32 as a floor on capability, not a measurement of it.

The output-token gradient (6,120 → 6,638 → 22,340) is the more informative number: cost stays flat

from general to hard, then rises 3.6× at frontier, which locates this model's ceiling between those

two tiers.

Do not use wikitext perplexity to pick a quant on this model

A full ladder on one binary, corpus and config (wiki.test.raw, 137 chunks) came out

non-monotonic, with three quants scoring "better" than bf16:

| arm | PPL |

|---|---|

| BF16 | 24.8258 |

| Q8_0 | 24.6496 |

| Q6_K | 24.1758 |

| Q5_K_M | 25.8216 |

| Q4_K_M | 24.3353 |

| BF16 (repeat) | 24.8258 |

The repeat pass is identical to four decimals, so the noise floor is zero and those differences

are real measurements — they simply are not measuring quantization damage. Two structural reasons:

the corpus is English prose against a model whose chat template and 131k vocab indicate

Chinese-primary training, and 27 of the 36 layers see only a 512-token window, so 2048-token chunks

exercise the long-range path in 9 layers and nothing else in the other 27. An English-prose PPL on

this arch measures roughly a quarter of it.

Use KL divergence against the bf16 logits instead — it asks "how far did quantization move the

model from its own full-precision self", which is the question that matters:

| arm | max KLD | RMS Δp | same top-1 |

|---|---|---|---|

| Q8_0 | 7.25 | 1.656% | 97.18% |

| Q6_K | 10.41 | 3.300% | 93.64% |

same top-1 % is the figure worth quoting. Q5_K_M and Q4_K_M are pending and will be added.

Provenance

Quantized with llama-quantize from the model/Spark2_5 branch at 498cbc7e0 (upstream base

6d0549831), x86_64 CUDA build verified to contain the arch before use. Every file was loaded and

generated from before publication — the PPL ladder above is that evidence — and Q5_K_M has

additionally been driven through the full 42-task agentic suite.

Credit

Model by XHToken. This repo is a format conversion only — no

weights were retrained or altered.