---

license: mit

library_name: ds4

pipeline_tag: text-generation

base_model: deepseek-ai/DeepSeek-V4-Flash

base_model_relation: quantized

quantized_by: antirez

language:

• en

tags:

• gguf
• ds4
• deepseek
• deepseek-v4
• deepseek-v4-flash
• apple-silicon
• metal
• moe
• expert-major
• iq2_xxs
• q2_k
• imatrix

inference: false

---

DeepSeek V4 Flash DS4 GGUF releases

This repository is the single release container for the versioned DeepSeek V4

Flash artifacts used by andreaborio/ds4.

The current runtime artifact embeds its routed MoE weights once as a

self-describing ds4.expert_major.v2 store. It needs no sidecar, runtime

repack, or second routed-weight copy.

Versions

| File | Status | Runtime |

| --- | --- | --- |

| DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix-DS4-ExpertMajor-v2.gguf | Current | DS4 main at 57acfd4 or newer |

| DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix.gguf | Legacy canonical source and conversion input | Generic GGUF tooling; rejected by current DS4 inference |

DSBox and the root dsbox.json select only ExpertMajor v2. The canonical file

is retained for provenance, checksum-stable conversion, and generic tooling;

it is not a fallback for current DS4. ExpertMajor v1, sidecars, canonical MoE

inference, and non-Metal inference are intentionally unsupported.

The current file is DS4-native rather than a portable GGUF layout. Generic

GGUF runtimes cannot reconstruct or execute its opaque routed store.

Current artifact

| Property | Value |

| --- | --- |

| Base model | deepseek-ai/DeepSeek-V4-Flash |

| File | DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix-DS4-ExpertMajor-v2.gguf |

| Size | 86,720,114,272 bytes (80.76 GiB) |

| SHA-256 | 8378080263eb9224f7228d72e2afa4ac3cf74a116023fdec2c596ff228a33e3f |

| Container | GGUF v3 with one embedded ds4.expert_major.v2 store |

| Routed geometry | 43 routed layers, 256 experts per layer, top-6 |

| Expert record | gate + up + down, 7,077,888 contiguous bytes |

| Routed payload | 77,913,391,104 bytes |

| Quantization | IQ2_XXS routed gate/up, Q2_K routed down, selected Q8_0/F16/F32 dense tensors |

| Qualified hardware | Apple Silicon Metal, at least 64 GiB unified memory |

The ExpertMajor v2 conversion changes only routed-weight storage order and the

GGUF tensor directory. It does not requantize model values. Non-routed tensors

are copied byte-for-byte from the canonical source and the routed payload is

stored once.

The retained canonical source is 86,720,111,488 bytes with SHA-256

efc7ed607ff27076e3e501fc3fefefa33c0ed8cf1eff483a2b7fdc0c2e616668.

It was mirrored checksum-for-checksum from

antirez/deepseek-v4-gguf

revision 9170bf42beb77f38006e016503ecace31f2bd9a0. Quantization credit belongs

to antirez and the DS4 project.

Run with DS4

Use andreaborio/ds4 main at commit

57acfd408a3154851a0c59be432904300abb3b6c

or a compatible descendant:

git clone https://github.com/andreaborio/ds4.git
cd ds4
make -j

./ds4 \
  -m /absolute/path/to/DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix-DS4-ExpertMajor-v2.gguf \
  --ctx 8192

The ExpertMajor v2 store, Metal backend, and memory policy are selected

automatically. No Metal, SSD-streaming, power, sidecar, pack, or format flag is

needed. The qualified minimum is 64 GiB unified memory; 128 GiB is the more

comfortable tier for this 80.76 GiB artifact.

Verification

The deterministic converter verifies source identity, GGUF metadata,

non-routed tensors, every routed expert component, alignment, embedded

manifest, and the complete ExpertMajor payload. Canonical and ExpertMajor v2

runs produced identical frontier-logit hashes in the retained 128- and

768-token correctness gates. The 768-token prefill comparison measured 95.82

versus 95.74 tokens/s, so the layout is a locality/runtime contract rather than

a claim of standalone universal acceleration.

The clean v2-only refactor gate on an M5 Pro with 64 GiB measured 20.93/13.20

tokens/s prefill/decode from a colder file cache and 23.78/13.95 tokens/s on the

immediate warm repeat. Both runs produced frontier-logit SHA-256

71fd3be0732e0fe97b9f104112911dc937896257c604c64eae851e36fa142441

with zero swapout.

The release runtime uses a pressure-gated cache phase: 259 expert records for

large prefill, 4,129 through 32K decode, and 2,065 at 65K and above. On the same

64 GiB M5 Pro, the zero-swap prose lanes measured:

| Context | Prefill | Decode | TPOT p50/p95 |

| --- | ---: | ---: | ---: |

| 32K+128 | 164.43 t/s | 7.27 t/s | 105.410/149.943 ms |

| 65K+128 | 137.29 t/s | 6.59 t/s | n/a |

| 100K+128 | 145.11 t/s | 6.44 t/s | n/a |

The original 32K AUTO controls can swap, so no 32K speedup percentage is

published. The promoted result is the safer adaptive policy, exact output, and

bounded long-tail latency under the qualified workload.

Limitations

• Apple Silicon Metal is the only qualified inference backend.
• At least 64 GiB unified memory is required.
• Current DS4 deliberately rejects the retained canonical file, ExpertMajor

v1, sidecars, CPU, CUDA, ROCm, and distributed inference for this model.

• Performance depends on storage state, memory pressure, context length,

routing locality, and thermals.

License and attribution

The source repository declares MIT. DeepSeek retains the base-model copyright;

the GGUF is redistributed under the base model's release terms. Quantization

credit belongs to antirez and the DS4 project.