---

license: mit

base_model: deepseek-ai/DeepSeek-V4-Flash-DSpark

pipeline_tag: text-generation

tags:

- gguf

- deepseek

- moe

- quantized

- 2-bit

- mxfp4

- speculative-decoding

- expert-pruning

- gb10

---

DeepSeek-V4-Flash — REAP-25 · type-40 W4A8 / MXFP8_LT · DSpark (ds4 GGUF)

A 91 GB single-file build of DeepSeek-V4-Flash

(284B MoE, 13B active, 1M context) sized to run **fully resident on one NVIDIA

GB10 (128 GB unified memory)** with room for a 1M-token context — the DSpark

speculative-decoding drafter is merged into the same file and auto-enables on

load.

Runs only on the ds4 engine. The file

uses custom tensor formats (MXFP4/MXFP8 microscaling, REAP-compacted expert

layout, embedded drafter) that llama.cpp and other GGUF loaders will not accept.

What's in the file

The routed-expert formats were chosen by a measured-KL allocation — not a

hand rule. Each candidate format's reconstruction error was measured per tensor

against the FP8/FP4 QAT source, weighted by an empirical Fisher sensitivity, and

allocated under the byte budget by an exact knapsack. The result is a

per-layer, per-role mix rather than a uniform bpw.

| Component | Treatment |

|---|---|

| Routed experts | IQ2_XXS floor (2.06 bpw) with MXFP4 (4.25 bpw, byte-lossless) promoted on the quality-sensitive layers, split by projection role. As of v0.2.3 the MXFP4 layers run the CUTLASS tensor-core type-40 W4A8 grouped GEMM (4-bit weights, E4M3 activations) |

| — early layers (0–12) | MXFP4 on gate/up, floor on down — early layers shape routing |

| — late layers (26–42) | MXFP4 on down, floor on gate/up — late layers write results |

| — a few mid layers | fully MXFP4 |

| Expert count | REAP-25: 25% of routed experts pruned (saliency-based), dense-compacted, padded router |

| Attention, shared experts, dense | MXFP8 (E4M3) — byte-lossless re-encode of the FP8 QAT source, served on the tensor-core FP8 path. As of v0.2.3 stored in the type-41 MXFP8_LT swizzle and loaded zero-copy (no runtime repack) |

| LM head | MXFP8 (E4M3), MXFP8_LT swizzle |

| DSpark drafter | Embedded in the same GGUF; auto-detected and enabled at load |

| KV cache (runtime) | Packed MXFP8 attention KV + MXFP4 indexer cache — bit-exact with the QAT cache format |

Why MXFP4 for the promoted layers instead of a k-quant: MXFP4 is the

checkpoint's source encoding, so it is a byte-lossless re-encode (zero

reconstruction error) at 4.25 bpw. No k-quant can be both lossless and this

small — Q6_K gets close on error but costs 6.56 bpw. This is the edge the ds4

engine's tensor-core paths provide and a portable k-quant GGUF cannot.

Requirements

• NVIDIA GB10 (SM 12.1, 128 GB unified memory)
• ds4 engine built with CUDA_ARCH=sm_120f
• Release scope: a single session up to the full 1M context, **or up to

three concurrent sessions** at moderate context (64k default per slot) — the

sum of context sizes is what's bounded (~14 GiB session budget on top of the

weights), not session-count × 1M. KV is cheap (packed MXFP8/MXFP4, ~0.5 GiB at

64k); per-session cost is dominated by prefill scratch. (v0.2.3 lifted the

concurrent cap from two.)

Quickstart

hf download twaggs88/DeepSeek-V4-Flash-REAP25-DSpark-ds4-GGUF \
  ds4flash-v5mx-reap25-type40-mxfp8lt-dspark-v1.gguf --local-dir ./gguf

# OpenAI- and Anthropic-compatible server
./ds4-server -m gguf/ds4flash-v5mx-reap25-type40-mxfp8lt-dspark-v1.gguf --ctx 1048576

# or the CLI
./ds4 -m gguf/ds4flash-v5mx-reap25-type40-mxfp8lt-dspark-v1.gguf

Checksum:

sha256: 000974720296f2cad17ac0525796f4bb9ceaac9f4015ed61af3fba445dfb1039
size:   91,321,404,640 bytes

Speculative decoding needs no flags: the drafter rides in the file and is used

at every temperature with exact sampled acceptance — the output

distribution is identical to plain sampling (verified against a χ² oracle).

Recommended sampling (best agentic/tool-use results in our testing):

temperature 0.95, top-p 0.38

Speed (measured on GB10)

Measured on a DGX Spark GB10, 2026-07-18, v0.2.3 (type-40 W4A8 experts +

type-41 MXFP8_LT). Prefill is cold (cache_prompt off, unique-prefix probe);

decode is speculative (merged DSpark drafter), single stream, --temp 0;

median of three runs.

| Context (actual) | Prefill cold (t/s) | TTFT cold | Decode structured (t/s) | Decode prose (t/s) |

|---:|---:|---:|---:|---:|

| ~0.3k | — | ~1.3 s | ~27.0 | ~18.1 |

| ~2.3k | ~390 | ~5.9 s | ~24.0 | ~20.7 |

| ~9.3k | ~413 | ~22.6 s | ~25.6 | ~19.6 |

Draft acceptance (α): ~81-91% on structured/tool output, ~60-68% on general

prose. Speculative throughput is stochastic run-to-run under sampling — the

figures above are greedy (--temp 0). Versus the prior v0.2.2 build, decode is

neutral (within run-to-run noise) and prefill improved (+17.5% at the

pre-release single-shape gate). The type-40/MXFP8_LT re-encode is

fidelity-neutral: measured KL 0.007, top-1 agreement 100%. A uniform 2-bit

build prefills faster but scores lower on quality; this mixed-quant allocation

trades some prefill for the tool-use win.

Quality

A fresh 4-seed tool-eval-bench

run of the shipping build (hardmode, temperature 0.95, top-p 0.38) averages

~90 / 100 (range 87–92 across seeds; even a fixed seed varies ±2 from

KV/CUDA nondeterminism). In the hardmode bake-off that selected this

allocation, it scored 92 / 100 against 84 / 100 for a uniform

all-Q2_K build:

| Build | tool-eval-bench (hardmode bake-off) |

|---|---|

| This build (measured MXFP4/IQ2) | 92 / 100 ★★★★★ |

| all-Q2_K (uniform 2-bit experts) | 84 / 100 ★★★★ |

The measured allocation recovers most of what uniform 2-bit quantization costs

on tool-use. Measured against the unquantized DeepSeek-V4-Flash reference,

the total quantization + REAP-pruning quality tax is only ~1–2 points: the

remaining failures are model-inherent — long-horizon multi-turn retention and a

cross-turn prompt-injection class that current models broadly fail — not

quantization artifacts.

Provenance & method

1. Base: deepseek-ai/DeepSeek-V4-Flash-DSpark (MIT) — DeepSeek-V4-Flash

weights (FP8/FP4 QAT) plus the DSpark speculative-decoding drafter.

1. Expert pruning: REAP at 25% ([Router-weighted Expert Activation

Pruning](https://arxiv.org/abs/2510.13999)), dense-compacted, router padded.

The survivor set is the LiveCodeBench-50-calibrated prune published by

eouya2.

1. Allocation: per-tensor KL measured against the QAT source, Fisher-weighted,

allocated by exact knapsack over the ds4-servable format menu (IQ2_XXS / Q2_K

/ MXFP4 experts; MXFP8 non-experts), using

PrismaQuant (Rob Tand).

1. Quantization: ds4's deepseek4-quantize --format-map (MXFP4/MXFP8 packed

losslessly straight from the QAT safetensors).

1. Drafter: confidence-scheduled DSpark, merged in-file.

Limitations

• Not loadable by llama.cpp / ollama / vLLM — ds4 engine only.
• 2-bit floor on the majority of experts: measurable but small quality loss vs

the source (tables above).

• Single-GPU, single-session design point; no tensor parallelism.
• Same-seed runs are reproducible on identical hardware/build; reproducibility

across driver/CUDA versions is not guaranteed.

Acknowledgements

• DeepSeek — the DeepSeek-V4-Flash / DSpark open weights and their

compressed-KV, FP4/FP8-QAT architecture.

• Rob Tand / PrismaQuant — the

measured-KL mixed-precision allocation method that chose this build's

per-layer expert formats.

• REAP — the Router-weighted Expert

Activation Pruning method behind the 25% expert prune.

• eouya2

— the REAP-25 (LiveCodeBench-50-calibrated) DeepSeek-V4-Flash expert-survivor

set this build's REAP transplant uses.

• llama.cpp / GGML — the GGUF

ecosystem and quantization groundwork the engine builds on.

• antirez/ds4 — the upstream engine this

fork is based on.

License

MIT, inherited from the base model. Quantized weights are a derivative work of

DeepSeek's checkpoint; this repository adds no additional restrictions.