---

base_model: RedHatAI/GLM-5.2-speculator.dspark

tags:

• safetensors
• dspark
• speculators
• glm
• custom_code
• license:apache-2.0
• region:us

---

GLM-5.2 DSpark GGUF

GGUF quantizations of RedHatAI DSpark draft model for GLM-5.2.

Use with BeeLlama.cpp, a llama.cpp fork with advanced quantization features.

---

RedHatAI/GLM-5.2-speculator.dspark

This is a preliminary (and subject to change) DSpark speculator model for RedHatAI/GLM-5.2-NVFP4-FP8.

It was trained using the Speculators library on the mgoin/open-perfectblend-glm5.2-regen dataset.

Note:

It was validated on Nvidia B200, other hardware validation pending.

Training Details

Launch training

scripts/train.py \
  --verifier-name-or-path RedHatAI/GLM-5.2-NVFP4-FP8 \
  --data-path ./output/dspark_glm52 \
  --vllm-endpoint http://localhost:8000/v1 \
  --save-path ./output/dspark_glm52/checkpoints \
  --epochs 1 \
  --lr 3e-4 \
  --total-seq-len 8192 \
  --speculator-type dspark \
  --block-size 8 \
  --max-anchors 1024 \
  --num-layers 3 \
  --target-layer-ids 2 20 39 58 75 \
  --markov-rank 256 \
  --markov-head-type vanilla \
  --enable-confidence-head \
  --confidence-head-with-markov \
  --loss-fn '{"ce": 0.1, "tv": 0.9}' \
  --confidence-head-alpha 1.0 \
  --fsdp-shard \
  --logger tensorboard \
  --log-dir ./output/dspark_glm52/logs \
  --on-missing generate \
  --on-generate delete \
  --checkpoint-freq 0.01 \
  --request-timeout 60 \
  --max-retries 2

Deployment

Deploy with vLLM using the speculator as a draft model.

vllm serve RedHatAI/GLM-5.2-NVFP4-FP8  \
  --max-model-len 20480 \
  --spec-model RedHatAI/GLM-5.2-speculator.dspark \
  --spec-method dspark \
  --spec-tokens 7 \
  -tp 8

Preliminary Evaluations

Per-Position Acceptance Rate

| Dataset | Pos 0 | Pos 1 | Pos 2 | Pos 3 | Pos 4 | Pos 5 | Pos 6 | Avg. Length |

|:-------|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|:----------:|

| HumanEval | 77.8% | 59.1% | 44.6% | 33.9% | 26.1% | 20.3% | 15.8% | 3.78 |

| math_reasoning | 87.5% | 76.6% | 67.6% | 59.2% | 51.8% | 45.1% | 39.5% | 5.27 |

| qa | 70.7% | 48.5% | 33.7% | 23.7% | 17.1% | 11.9% | 8.4% | 3.14 |

| question | 73.7% | 52.6% | 38.1% | 28.5% | 21.6% | 16.7% | 13.2% | 3.44 |

| rag | 78.5% | 61.3% | 48.7% | 38.9% | 30.7% | 24.2% | 19.2% | 4.01 |

| summarization | 77.9% | 58.0% | 43.0% | 31.7% | 22.8% | 16.2% | 11.4% | 3.61 |

| tool_call | 71.9% | 51.1% | 37.1% | 27.6% | 21.4% | 16.8% | 13.2% | 3.39 |

| translation | 76.8% | 57.4% | 43.5% | 33.6% | 26.1% | 20.0% | 15.1% | 3.72 |

| writing | 74.0% | 52.2% | 38.0% | 28.1% | 21.3% | 16.7% | 13.1% | 3.44 |