---

license: mit

base_model: deepseek-ai/DeepSeek-V4-Flash

tags:

• gguf
• speculative-decoding
• ds4
• dgx-spark

---

DeepSeek-V4-Flash DSpark drafter (Q2K-Q8 GGUF)

The DSpark block drafter for

DeepSeek-V4-Flash,

packaged for the ds4 C+CUDA inference

engine — specifically the Entrpi/ds4

batched-serving fork (release v0.1.1+), where it drives **lossless

speculative decode at a 9-workload suite mean of 1.38x plain decode on a DGX

Spark (GB10)**, peaking at 1.71x on structured content. Output is lossless by

construction: the target model's verify forward is the sole token source.

What's in the file

The drafter's 3 full DeepSeek-V4 transformer layers (attention + 256-expert

MoE), the target-fusing projection (main_proj/main_norm over target layers

40/41/42), the rank-256 Markov head, the confidence head, and the output head.

embed_tokens/lm_head are shared from the target model at load time and are

not duplicated here.

Quant policy: Q2_K routed experts (the ds4 v0.1.1 ship default — measured

equal to Q4_K in acceptance and throughput, 4.2 GiB smaller), Q8_0

attention/projections/shared experts, F32 norms/gates, F16 Markov/confidence

heads. ~6.5 GiB.

Extracted from the official checkpoint's drafter shards with

gguf-tools/dspark_extract.py

(FP8 e4m3/ue8m0 dequant identical to the engine's C implementation), routed

experts requantized to Q2_K.

Files

| file | pairs with | Markov table | size |

|---|---|---|---|

| DSpark-drafter-Q2K-Q8-MarkovQ8-0731.gguf | DeepSeek-V4-Flash-0731 | Q8_0 | 6.94 GB |

| DSpark-drafter-Q2K-Q8-MarkovQ8-vision-exp.gguf | DeepSeek-V4-Flash-Vision-Exp (source revision e46e16bf) | Q8_0 | 6.94 GB |

| DSpark-drafter-Q2K-Q8-0731.gguf | DeepSeek-V4-Flash-0731 | F16 | 6.97 GB |

| DSpark-drafter-Q2K-Q8.gguf | the pre-0731 Flash checkpoint | F16 | 6.97 GB |

The two MarkovQ8 files (added 2026-09-03) store the Markov head's

markov_w2 table as Q8_0 instead of F16; every other tensor is

byte-identical to the F16 file of the same checkpoint. The engine reads

that table in full four times per draft block, and in Q8_0 it takes the

existing Q8_0 mat-vec path: 0.18 ms per call instead of 0.63 on a GB10,

about 1.8 ms per draft block, with the file 31 MB smaller. Acceptance is

unchanged: identical per-record accept and the same mean commit (3.70)

replaying a recorded trace through both forms, and within run-to-run noise

on a live A/B. Decoding stays lossless in either form because the target

verifies every draft.

A ds4 build from 2026-09-03 or later is needed to load the MarkovQ8

files (its loader accepts F16 or Q8_0 for that tensor); its launch defaults

prefer the MarkovQ8 file beside the base GGUF and fall back to the F16

one. Older builds keep using the F16 files, which stay here unchanged.

Generations never mix: the engine refuses a Vision-Exp drafter beside a

0731 base and the reverse.

Usage

One-command serving on a DGX Spark via

ds4-on-spark:

curl -sSL https://raw.githubusercontent.com/entrpi/ds4-on-spark/main/install.sh | bash -s -- --start

Or by hand, alongside the base + MTP GGUFs from

antirez/deepseek-v4-gguf:

env DS4_CONT_MTP_MODE=2 DS4_CONT_DSPARK=1 \
    DS4_DSPARK_MODEL=DSpark-drafter-Q2K-Q8-MarkovQ8-0731.gguf \
  ./ds4-server --cuda -m <base.gguf> --mtp <mtp.gguf> -c 32768 --port 8000

Speculation is content-dependent (52-91% draft acceptance across workloads).

Since ds4 fork v0.1.1 that variance is bounded by default: a terminal

yield-quench controller turns speculation off per request when acceptance

can't pay the verify cost (worst case ~0.96x plain), and a kv-depth gate hands

>64k-context requests to plain decode. Benchmarks, the two-corpus context

frontier, and the break-even analysis: see the

ds4-on-spark README.