---

license: apache-2.0

language:

• en
• zh

base_model: XHToken/Spark-X2.5-4B

base_model_relation: quantized

library_name: llama.cpp

pipeline_tag: text-generation

tags:

• gguf
• llama.cpp
• spark2_5
• quantized
• text-generation

---

Spark-X2.5-4B GGUF

Community GGUF quantizations of XHToken/Spark-X2.5-4B.

<div align="center" style="background-color:#f59e0b;color:#ffffff;padding:16px 20px;border-radius:10px;line-height:1.7;">

☕ If this GGUF made your day easier, a coffee would make mine.<br>

<a href="https://ko-fi.com/ngquocvinh" style="color:#ffffff;"><strong style="color:#ffffff;">Send a coffee ☕</strong></a><br>

I build and test these releases myself. Your coffee helps keep me going.<br>

Thank you for supporting this work.

</div>

About Spark-X2.5-4B

Spark-X2.5-4B is a compact, general-purpose language model developed by SparkLLM. According to the upstream authors, the model is designed for:

• General-purpose capabilities: conversation, writing, translation, reasoning, and coding
• Tool use and agentic workflows
• Native context window of up to 1M tokens
• More than 200 supported languages
• Efficient hybrid attention: one full-attention layer combined with three sliding-window attention layers
• Broad ecosystem support: llama.cpp, vLLM, SGLang, MLX, Ollama, and LM Studio

The upstream model is also designed for broad hardware compatibility and efficient long-context inference.

For the original model architecture, training details, benchmarks, and official usage instructions, see the official model card.

This repository is a quantization-only release for local inference. No model training or fine-tuning was performed.

Recommended quantized profiles

These are three practical starting points, all excluding the BF16 reference:

| Profile | Choose it when |

|---|---|

| Spark-X2.5-4B-Q8_0.gguf | You want the closest practical profile to the BF16 reference. |

| Spark-X2.5-4B-Q4_K_M.gguf | You want the main size/quality balance. |

| Spark-X2.5-4B-Q2_K.gguf | You need a smaller profile and accept a larger fidelity trade-off. |

Fidelity measurements

The table below compares every published GGUF file with the BF16 reference on

a held-out WikiText-2 evaluation: eight chunks from wiki.test.raw and eight

chunks from wiki.valid.raw, using a 4,096-token context and the same Spark

llama.cpp runtime. Values are averaged across the two splits. The BF16

reference mean PPL was 20.718057 in this evaluation. Lower Mean KLD, ΔPPL, and

RMS Δp, and higher Top-1 agreement, indicate closer next-token behavior to

BF16. These are next-token fidelity measurements, not a direct percentage of

capabilities retained and not a substitute for task-specific evaluation.

| File | Mean KLD ↓ | Top-1 vs BF16 ↑ | ΔPPL | RMS Δp |

|---|---:|---:|---:|---:|

| Spark-X2.5-4B-Q8_0.gguf | 0.005110 | 97.219% | −0.770% | 1.803% |

| Spark-X2.5-4B-Q6_K.gguf | 0.020056 | 93.689% | −2.333% | 3.402% |

| Spark-X2.5-4B-Q5_K_M.gguf | 0.057362 | 89.332% | +1.693% | 5.748% |

| Spark-X2.5-4B-Q5_K_S.gguf | 0.066414 | 88.541% | −0.175% | 6.175% |

| Spark-X2.5-4B-Q4_K_M.gguf | 0.151599 | 83.009% | +4.698% | 8.766% |

| Spark-X2.5-4B-Q3_K_M.gguf | 0.303065 | 76.429% | +22.339% | 12.197% |

| Spark-X2.5-4B-Q2_K.gguf | 0.904467 | 60.469% | +89.737% | 21.993% |

| Spark-X2.5-4B-IQ4_XS.gguf | 0.166476 | 82.248% | +8.917% | 9.248% |

| Spark-X2.5-4B-IQ4_NL.gguf | 0.160332 | 82.514% | +7.958% | 9.113% |

| Spark-X2.5-4B-IQ3_M.gguf | 0.363476 | 73.510% | +17.084% | 14.386% |

| Spark-X2.5-4B-IQ3_S.gguf | 0.424754 | 71.733% | +12.067% | 15.217% |

| Spark-X2.5-4B-Q3_K_L.gguf | 0.258238 | 78.081% | +24.321% | 11.409% |

| Spark-X2.5-4B-Q2_K_S.gguf | 1.582512 | 51.261% | +202.887% | 29.706% |

| Spark-X2.5-4B-IQ2_XS.gguf | 2.931449 | 30.453% | +895.742% | 42.489% |

| Spark-X2.5-4B-IQ1_M.gguf | 6.530486 | 7.194% | +31,288.455% | 52.605% |

| Spark-X2.5-4B-Q1_0.gguf | 12.363455 | 0.000% | +5,913,223.270% | 55.296% |

The machine-readable results are available in

reproducibility/quality-summary.tsv,

with corpus hashes, evaluation settings, calibration provenance, and runtime

details in reproducibility/manifest.md.

Quick start

./llama-cli \
  -m Spark-X2.5-4B-Q4_K_M.gguf \
  --chat-template-file chat_template.jinja \
  --jinja \
  -p 'Answer briefly in English: What is GGUF and why is it useful for running language models locally?' \
  -n 128 -c 4096 -ngl 99

Reproducibility and validation

This is a quantization-only release: no training or fine-tuning was performed.

The BF16 source, runtime revision, clean calibration split, importance matrix,

quantization commands, checksums, and validation profile are recorded in the

reproducibility/ package. All 16 published files passed

load/generate validation on an NVIDIA A10M.

License and attribution

The upstream model is released under Apache License 2.0. Preserve the upstream attribution and license when redistributing these derivative files. This is a community GGUF quantization, not an official XHToken/SparkLLM release or endorsement.

Checksums are available in SHA256SUMS.txt.