---

license: mit

base_model: deepseek-ai/DeepSeek-V4-Flash

tags:

• gguf
• speculative-decoding
• ds4
• dgx-spark

---

DeepSeek-V4-Flash DSpark drafter (Q2K-Q8 GGUF)

The DSpark block drafter for

DeepSeek-V4-Flash,

packaged for the ds4 C+CUDA inference

engine — specifically the Entrpi/ds4

batched-serving fork (release v0.1.1+), where it drives **lossless

speculative decode at a 9-workload suite mean of 1.38x plain decode on a DGX

Spark (GB10)**, peaking at 1.71x on structured content. Output is lossless by

construction: the target model's verify forward is the sole token source.

What's in the file

The drafter's 3 full DeepSeek-V4 transformer layers (attention + 256-expert

MoE), the target-fusing projection (main_proj/main_norm over target layers

40/41/42), the rank-256 Markov head, the confidence head, and the output head.

embed_tokens/lm_head are shared from the target model at load time and are

not duplicated here.

Quant policy: Q2_K routed experts (the ds4 v0.1.1 ship default — measured

equal to Q4_K in acceptance and throughput, 4.2 GiB smaller), Q8_0

attention/projections/shared experts, F32 norms/gates, F16 Markov/confidence

heads. ~6.5 GiB.

Extracted from the official checkpoint's drafter shards with

gguf-tools/dspark_extract.py

(FP8 e4m3/ue8m0 dequant identical to the engine's C implementation), routed

experts requantized to Q2_K.

Usage

One-command serving on a DGX Spark via

ds4-on-spark:

curl -sSL https://raw.githubusercontent.com/entrpi/ds4-on-spark/main/install.sh | bash -s -- --start

Or by hand, alongside the base + MTP GGUFs from

antirez/deepseek-v4-gguf:

env DS4_CONT_MTP_MODE=2 DS4_CONT_DSPARK=1 \
    DS4_DSPARK_MODEL=DSpark-drafter-Q2K-Q8.gguf \
  ./ds4-server --cuda -m <base.gguf> --mtp <mtp.gguf> -c 32768 --port 8000

Speculation is content-dependent (52-91% draft acceptance across workloads).

Since ds4 fork v0.1.1 that variance is bounded by default: a terminal

yield-quench controller turns speculation off per request when acceptance

can't pay the verify cost (worst case ~0.96x plain), and a kv-depth gate hands

>64k-context requests to plain decode. Benchmarks, the two-corpus context

frontier, and the break-even analysis: see the

ds4-on-spark README.