---

license: mit

library_name: gguf

pipeline_tag: text-generation

base_model: deepseek-ai/DeepSeek-V4-Flash

base_model_relation: quantized

quantized_by: antirez

language:

- en

tags:

- gguf

- quantized

- deepseek

- deepseek-v4

- deepseek-v4-flash

- moe

- mixture-of-experts

- 2-bit

- iq2_xxs

- q2_k

- ds4

- expert-major

- apple-silicon

- metal

- experimental

---

DeepSeek V4 Flash DS4 ExpertMajor v2 GGUF

This is a native-format repository for the

andreaborio/ds4 runtime. It contains one

experimental ds4.expert_major.v2 artifact and intentionally does not

duplicate the canonical GGUF.

The native file is derived from one checksum-pinned DeepSeek V4 Flash Q2

imatrix GGUF by changing routed-expert storage order only. It is not a new

quantization, no tensor is requantized, and model math is unchanged.

Use this file only with a DS4 revision that implements ExpertMajor v2. For

llama.cpp, MLX, generic GGUF tools, or DS4 backends other than the qualified

Apple Metal path, use the canonical file in

andreaborio/DeepSeek-V4-Flash-DS4-GGUF.

Artifact

| File | Layout | Bytes | SHA-256 |

| --- | --- | ---: | --- |

| DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix-DS4-ExpertMajor-v2.gguf | Experimental ds4.expert_major.v2 for Apple Metal | 86,720,114,272 | 8378080263eb9224f7228d72e2afa4ac3cf74a116023fdec2c596ff228a33e3f |

The native artifact contains one 77,913,391,104-byte routed-weight payload,

not a canonical payload plus an optimization sidecar. It is only 2,784 bytes

larger than its canonical source.

Quantization provenance

The source quantization was produced by antirez for the

ds4 inference engine. The exact canonical

source remains available in the compatibility mirror linked above and was

mirrored byte-for-byte from antirez/deepseek-v4-gguf revision

9170bf4:

• routed expert gate/up tensors: IQ2_XXS;
• routed expert down tensors: Q2_K;
• attention projections, shared experts, and output head: Q8_0;
• embedding, router, indexer, compressor, and hyper-connection tensors:

selected F16/F32 types;

• routed experts use the published DS4 activation imatrix.

The canonical source file is

DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix.gguf,

86,720,111,488 bytes, with SHA-256

efc7ed607ff27076e3e501fc3fefefa33c0ed8cf1eff483a2b7fdc0c2e616668.

Quantization credit belongs to antirez and the DS4 project. This repository

publishes only the deterministic storage-layout conversion and does not claim

independent authorship of the quantization.

How the expert-major file is made

The deterministic converter shipped with DS4 reads the canonical GGUF v3,

copies its full metadata block and every non-routed tensor byte-for-byte, and

rewrites each routed layer as complete expert records:

record(layer, expert) = gate | up | down

The output embeds a self-describing manifest with component types, dimensions,

quantization block geometry, offsets, record extents, and checksums. Generic

loaders see that the canonical routed tensors are absent and fail explicitly;

they cannot silently interpret interleaved expert records as canonical data.

From a checkout containing gguf-tools/ds4-expert-major.py, the release command

is:

python3 gguf-tools/ds4-expert-major.py build \
  --reserve-bytes 1GiB \
  DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix.gguf \
  DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix-DS4-ExpertMajor-v2.gguf

python3 gguf-tools/ds4-expert-major.py verify \
  DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix.gguf \
  DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix-DS4-ExpertMajor-v2.gguf

The converter writes a same-filesystem temporary file, fsyncs it, performs a

full byte-level verification, and atomically installs the result. The release

verification recorded:

• canonical source SHA-256:

efc7ed607ff27076e3e501fc3fefefa33c0ed8cf1eff483a2b7fdc0c2e616668;

• routed expert-major payload SHA-256:

e5d34567e2d5397d37127ee1af5dfb8e70c2327a7eb4bd6a5a7162b6f53abf36;

• embedded manifest SHA-256:

9ee2cc83c6d9c553ca0423074ab78d07156e512c40855c3fe96a9089facb0107.

The embedded manifest describes 43 routed layers, 256 experts per layer, and

six selected experts per token. For this artifact, each complete expert record

is 7,077,888 bytes (6.75 MiB), and one physical routed layer is

1,811,939,328 bytes (1.6875 GiB).

Why this layout exists

Canonical DeepSeek GGUFs group each layer's gate, up, and down banks by tensor.

DS4 streams selected experts from SSD and, during long prefill, consumes a

complete routed layer. Expert-major order gives the runtime a validated

physical record for every expert and a real contiguous layer view without

maintaining a second copy or repacking weights at token time.

This is primarily a runtime and locality format. On the first M5 Pro

qualification tranche, native and canonical decode were effectively tied, and

the 768-token prefill comparison was 95.82 versus 95.74 tokens/s. The native

layout enables the phase-aware prefill and cache policies used by later DS4

work; it is not presented as a standalone quality or universal speedup.

Verification and current status

The converter's full verifier checks the source identity, GGUF metadata,

non-routed tensors, every routed expert component, alignment/padding, manifest,

and complete expert-major payload. Canonical and native runs produced identical

complete frontier-logit hashes at 128 and 768 tokens:

| Context | Frontier logits SHA-256 |

| ---: | --- |

| 128 | 84482c0ada999d2e7ff41d1a5f54dc5df4bab96be64b5eca8ecb0ef6b4976b23 |

| 768 | 84ddca7a8356d053627115f333ec26aeff04998cd998347fd282875c9385ce43 |

This repository and its native artifact remain experimental. The initial Metal qualification

covered exactness, SSD decode translation, physical long-prefill mapping, and

bounded performance checks on a 64 GiB M5 Pro. It has not completed the full

multi-tier 2K/8K/16K promotion gate, so the canonical GGUF remains the default

compatibility recommendation.

Run with DS4

The expert-major artifact requires andreaborio/ds4 production main commit

bd62a0bf36336fc5e3b199ac97bdacf820c4c7f0

or a descendant:

git clone https://github.com/andreaborio/ds4.git
cd ds4
make -j

./ds4 \
  --metal --power 100 --ssd-streaming \
  -m /absolute/path/to/DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix-DS4-ExpertMajor-v2.gguf \
  --ctx 8192

On a 64 GB Mac, SSD streaming is required. The 128 GB class is the more

comfortable target for this 80.76 GiB artifact. The embedded v2 store activates

automatically; there is no sidecar path or hash environment variable.

The qualified native path is currently a complete local Apple Metal model.

CPU, CUDA, ROCm, distributed, and generic GGUF loaders must use the canonical

file until those backends implement and qualify the v2 translator.

Hotlist priority policy

DS4 seeds the streamed expert cache in a useful pretrained order, then lets

real request routes replace unused entries immediately. The default adaptive

policy gives every seed an initial LFU priority of one. It can be rolled back

or fixed for controlled comparisons:

# Default; live selections immediately outrank unused preload entries.
DS4_METAL_STREAMING_EXPERT_HOTLIST_PRIORITY=adaptive ./ds4 ...

# Historical built-in ranks or per-file hit counts.
DS4_METAL_STREAMING_EXPERT_HOTLIST_PRIORITY=legacy ./ds4 ...

# Uniform positive initial priority.
DS4_METAL_STREAMING_EXPERT_HOTLIST_PRIORITY=8 ./ds4 ...

This policy changes cache admission/eviction only. It does not modify routing,

weights, logits, or the GGUF.

DSBox

This repository's dsbox.json identifies the single native file, declares the

ds4-expert-major format at version 2 and its ds4.expert_major.v2 tensor,

pins the minimum compatible DS4 commit, and marks the artifact experimental

with recommended: false. DSBox can expose it for deliberate testing, but it

must not replace the stable canonical recommendation while the remaining

promotion gates are open.

License and attribution

The source repository declares MIT. DeepSeek retains the base-model copyright;

the GGUF is redistributed under the base model's release terms. Quantization

credit belongs to antirez and the DS4 project.