---

license: mit

base_model: deepseek-ai/DeepSeek-V4-Flash-0731

base_model_relation: quantized

tags:

- gguf

- deepseek

- deepseek-v4

- deepseek-v4-flash

- ds4

- dspark

- iq2_xxs

- q2_k

- imatrix

- 2-bit

- dgx-spark

- gb10

pipeline_tag: text-generation

---

DeepSeek-V4-Flash-0731 — IQ2_XXS/Q2_K GGUF for ds4

A 2-bit GGUF of the official DeepSeek-V4-Flash-0731 release, built for the

ds4 engine so the model fits on a **single 128 GB

GB10 box** (DGX Spark / ASUS Ascent GX10).

86.7 GB. Every other public GGUF of -0731 at the time of writing was 155–165 GB

(MXFP4 / Q4_K / Q8), which does not fit in 128 GB of unified memory. This one does, with

room for the KV cache and the DSpark drafter.

Verified working: it is currently serving on a 121 GB GB10.

Files

| file | size | what |

|---|---|---|

| DeepSeek-V4-Flash-0731-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-imatrix.gguf | 86,720,111,488 B | the model |

You also need a drafter for speculative decoding — not included here, see below.

Quantization recipe

Same asymmetric recipe antirez uses for the 2-bit Flash builds: **only the routed MoE experts

are quantized** — up/gate at IQ2_XXS, down at Q2_K. Attention projections, shared

experts and the output head stay at Q8_0.

Built with gguf-tools/deepseek4-quantize from the ds4 repo:

deepseek4-quantize \
  --hf      DeepSeek-V4-Flash-0731/            # official FP8 safetensors, 166.9 GB
  --template DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2.gguf \
  --out     DeepSeek-V4-Flash-0731-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-imatrix.gguf \
  --imatrix DeepSeek-V4-Flash-chat-v2-routed-moe-ds4-1p5m.dat \
  --threads 18

Both the template and the imatrix come from

antirez/deepseek-v4-gguf and are from the

Preview checkpoint. That is safe, and it was checked rather than assumed:

• Diffing model.safetensors.index.json of -0731 against the Preview shows the model **body is

identical**: 67,612 non-MTP tensors, name for name, zero differences. Only the MTP/DSpark module

changed (1 layer → 3 layers), and that module is not part of this file.

• --dry-run reported type_changes: 0 and predicted approx_file_bytes: 86720111488; the

finished file is exactly that — 288 bytes from the template.

⚠️ Do not pass --imatrix-strict: that imatrix covers only the routed-MoE tensors (129

entries = 43 layers × 3), so token_embd.weight has no vector and strict mode aborts the build

after ~2 hours. The non-routed tensors going through plain RTN is the intended recipe.

Running it

Needs ds4 v0.5.0 or newer — the Entrpi fork for

Blackwell/GB10. v0.5.0 is a large speed jump over v0.4.2, see the numbers below.

Build it with make cuda-spark, which selects sm_121 plus the Spark HBM weight

cache — not sm_120, which falls back to slower Turing-era code paths.

ds4-server --cuda \
  -m DeepSeek-V4-Flash-0731-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-imatrix.gguf \
  --dspark DSpark-drafter-Q2K-Q8-0731.gguf \
  -c 131072 --host 0.0.0.0 --port 30001

The drafter matters — get the right one

Use bleysg/DeepSeek-V4-Flash-DSpark-drafter-GGUF

→ DSpark-drafter-Q2K-Q8-0731.gguf.

Older-format DSpark files fail at load with

ds4: required tensor is missing: dspark.main_proj.weight. v0.4.x expects tensors prefixed

dspark., metadata keys prefixed deepseek4.dspark., and markov_w1/markov_w2 in F16;

the pre-0731 files use mtp., bare dspark. keys and Q8_0. Same module, different

serialization — re-extract with gguf-tools/dspark_extract.py if you need to build your own.

Memory notes on a 128 GB box

• -c 131072 uses ~3.1 GB of context buffers. -c 262144 uses ~5.9 GB and, together with the

continuous batch, was enough to drive MemAvailable to zero on a 121 GB box — leave headroom.

• Speculation lives on the continuous-batch path. DS4_SERVER_COALESCE_MAX=1 skips creating that

batch entirely, and DSpark silently never runs (ds4_spec_drafts_total 0). Minimum is 2.

• Check it is actually speculating: curl -s :30001/metrics | grep ds4_spec.

Measured (be skeptical)

On one GB10, short context, -c 131072, drafter armed, engine v0.5.0:

| workload | decode | DSpark acceptance |

|---|---|---|

| step-wise math | 28–33 tok/s | 89 % |

| Italian prose (on v0.4.2) | 14–17 tok/s | 68–72 % |

Acceptance is strongly workload-dependent — structured math/code drafts well, free prose does not.

The engine version matters just as much: the same file on v0.4.2 was roughly half as fast. Both

variables moved between those two rows, so do not read the difference as attributable to either

one alone.

First request after a boot is slower (~10–20 tok/s) while CUDA graphs are captured.

These are single-sample, non-paired measurements — an order of magnitude, not a benchmark. Deep

context is much slower for everyone: published figures for this engine are ~22 tok/s at 240k and

~776 tok/s sustained prefill at 518k.

Quality of the quantization itself has not been evaluated. No perplexity, no benchmark suite.

If you measure either, please open a discussion.

Credits

• DeepSeek — base model, MIT.
• antirez — the ds4 engine, the quantizer,

the 2-bit recipe, and both the template and imatrix this build depends on.

• Entrpi — the Blackwell/GB10 fork.
• bleysg — the 0731 drafter.

MIT, following the base model.