---

license: mit

library_name: gguf

tags:

• deepseek
• deepseek-v4
• deepseek-v4-flash
• moe
• reap
• gguf
• dgx-spark
• gb10

---

DeepSeek V4 Flash 0731 REAP K216 GGUF

An experimental, single-DGX-Spark build of DeepSeek V4 Flash 0731 that retains 216 of 256 routed experts per MoE layer.

This repository uses a different construction order from earlier K160 conversions:

1. Start from Unsloth's stock DeepSeek V4 Flash 0731 GGUF quantization.
2. Apply 0xSero's K216 REAP expert map directly to the already-quantized GGUF.
3. Copy every retained quantized expert row byte-for-byte; do not dequantize or requantize it.
4. Replace the compact routing tables for layers 0-2 with the exact tables carried by 0xSero's checkpoint.

This preserves the stock quantization exactly for all retained tensors and avoids quantizing an already transformed/repacked REAP checkpoint.

Files

| Variant | Size | Status |

|---|---:|---|

| DeepSeek-V4-Flash-0731-REAP-K216-UD-IQ3_XXS-00001-of-00003.gguf | 82.94 GiB total | Validated on NVIDIA GB10 |

The first shard is the entry point; llama.cpp discovers the remaining shards automatically.

Provenance

• Original model: deepseek-ai/DeepSeek-V4-Flash-0731
• Stock GGUF quant: unsloth/DeepSeek-V4-Flash-0731-GGUF, UD-IQ3_XXS
• K216 REAP map: 0xSero/deepseek-v4-flash-0731-spark, revision 22f28d32b9b29b4352eaa380ff8c2c170b2847ab

The included structured report and plan record the selected experts. The included validator compared all 1,328 output tensors and all 89,060,075,612 tensor-data bytes against the stock quant and K216 routing source:

• 1,113 unchanged tensors matched the stock GGUF byte-for-byte;
• 212 compacted expert/router tensors matched their selected stock rows byte-for-byte;
• 3 tid2eid routing tables matched 0xSero's carried tensors exactly.

GB10 test result

Tested on an NVIDIA DGX Spark / GB10 with llama.cpp b10223 (11924d4c1), CUDA architecture 121a.

• Full CUDA offload completed successfully.
• Prompt processing: 33.2 tokens/s in the short smoke test.
• Generation: 15.9 tokens/s in the short smoke test.
• Exact-shape MUL_MAT_ID correctness tests passed for the IQ2_XS gate/up projections and IQ3_XXS down projections with 216 experts and top-6 routing.

Configured-context smoke tests used one slot, Q8_0 K/V cache, full CUDA offload, -b 1024 -ub 512, and the same short non-thinking API prompt:

| Configured context | Prompt | Generation | CUDA allocation | Result |

|---:|---:|---:|---:|---|

| 32,768 | 46.8 tok/s | 16.8 tok/s | 85,192 MiB | Pass |

| 65,536 | 48.1 tok/s | 16.6 tok/s | 85,330 MiB | Pass |

| 131,072 | 44.1 tok/s | 16.3 tok/s | 85,640 MiB | Pass |

| 262,144 | — | — | 86,330 MiB | Initializes; first prompt crashes in the current llama.cpp CUDA path |

Recommended GB10 context: 131,072 tokens. This is the largest configuration currently validated to initialize and generate successfully. These are short-prompt allocation/generation smoke tests, not tests with the entire context window occupied; sustained long-context quality and memory behavior still need evaluation.

llama.cpp example

Use a recent llama.cpp build with deepseek4 support. On GB10, mmap is recommended; --no-mmap needlessly duplicates memory pressure for this model.

llama-cli \
  -m DeepSeek-V4-Flash-0731-REAP-K216-UD-IQ3_XXS-00001-of-00003.gguf \
  -ngl 999 \
  --flash-attn on \
  -c 131072 \
  -b 1024 \
  -ub 512 \
  --cache-type-k q8_0 \
  --cache-type-v q8_0 \
  --temp 1.0 \
  --top-p 1.0 \
  --top-k 0 \
  --min-p 0 \
  --repeat-penalty 1.0 \
  -n 256 \
  -p "Write one sentence about the moon."

DeepSeek V4 Flash defaults to a thinking-style template. For direct-answer diagnostics, disable reasoning when your llama.cpp client or API exposes that option. Do not treat a repetition penalty as a fix for a conversion or template problem.

Reproducibility

• tools/reap_k216_gguf.py performs the byte-preserving rewrite.
• tools/validate_reap_k216_gguf.py performs exhaustive tensor-data validation.
• metadata/REAP_K216_PLAN.json records the rewrite plan.
• metadata/REAP_K216_STRUCTURED_REPORT.json records 0xSero's structured K216 selection.

This is frontier/experimental work. Please include your llama.cpp commit, backend, hardware, context, exact command, and minimal prompt when reporting results.

License and acknowledgements

MIT, inherited from the source model.

Thanks to DeepSeek, Unsloth, 0xSero, the REAP authors, and llama.cpp contributors.