---

license: apache-2.0

license_link: https://huggingface.co/Qwen/Qwen3.6-35B-A3B/blob/main/LICENSE

base_model:

- Qwen/Qwen3.6-35B-A3B

tags:

- gguf

- ds4

- qwen3.6

- qwen35moe

- apple-silicon

- metal

- quantized

- experimental

- expert-major

pipeline_tag: text-generation

inference: false

---

Qwen3.6-35B-A3B DS4 ExpertMajor v1 GGUF

This repository contains the text-only Q4_K_S artifacts qualified by the

experimental Qwen3.6 Metal path in

andreaborio/ds4. The native

ds4.expert_major.v1 file is the preferred DS4 artifact. The canonical GGUF

remains available for migration and for runtimes that do not implement the DS4

extension.

This is not generic Qwen GGUF support. DS4 validates the exact Qwen3.6 model

geometry, tokenizer metadata, tensor inventory, layout, and static payload

contract before starting inference.

Artifacts

Base model: Qwen/Qwen3.6-35B-A3B

Architecture: qwen35moe

Parameters: 35B total, approximately 3B active per token

Vision / MTP payload: not included

| File | Intended runtime | Bytes | SHA-256 |

| --- | --- | ---: | --- |

| Qwen3.6-35B-A3B-DS4-ExpertMajor-v1-Q4_K_S.gguf | DS4 ExpertMajor v1 on Apple Metal | 20,808,970,240 | fb2b344d49f0c3dfd854cfc11d92ffc873cc93a1d30bf4664e5aea6f1bfef839 |

| Qwen3.6-35B-A3B-ds4-Q4_K_S.gguf | Canonical migration artifact and generic GGUF tooling | 20,808,563,424 | c33efb67bde86c9ba1f9e79c2dc42627170963bef0e915ab9b91a55cfb6d0fcd |

The native file replaces the 120 canonical routed tensor descriptors with one

opaque ds4.expert_major.v1 tensor. DS4 reconstructs the 733-tensor logical

inventory at load time. Routed weights are stored exactly once in the order

used by resident and SSD execution, so the file is only 406,816 bytes (about

0.002%) larger than the canonical GGUF. It is not a canonical layout hidden

behind a new filename.

The native GGUF is a valid GGUF v3 container, but generic loaders cannot run

its routed weights. Use the canonical file with llama.cpp, MLX, or any runtime

that does not explicitly implement ds4.expert_major.v1.

Quantization provenance

The artifact starts from Unsloth's

Qwen3.6-35B-A3B-UD-Q4_K_S.gguf

at revision a483e9e6cbd595906af30beda3187c2663a1118c. DS4 normalization restores the

official padding token ID and chat template. Of the 733 tensor payloads, 729

remain byte-identical to that pinned source. The only conversions are:

• blk.34.ffn_down_exps.weight: Q6_K to Q4_K
• blk.38.ffn_down_exps.weight: Q6_K to Q4_K
• blk.39.ffn_down_exps.weight: Q6_K to Q4_K
• output.weight: Q6_K to Q8_0

The three expert-bank conversions give all routed experts one uniform Q4_K

cache layout. The output head is retained at the higher-precision Q8_0 type.

Run with DS4

Use andreaborio/ds4 production main at commit

bd62a0bf36336fc5e3b199ac97bdacf820c4c7f0

or a descendant:

git clone https://github.com/andreaborio/ds4.git
cd ds4
make -j

DS4_QWEN_EXPERIMENTAL_METAL=1 ./ds4 \
  --metal --power 100 \
  -m /absolute/path/to/Qwen3.6-35B-A3B-DS4-ExpertMajor-v1-Q4_K_S.gguf \
  --ctx 8192 --tokens 4096

The embedded store activates automatically. No sidecar path or hash variables

are required.

Leave residency on AUTO unless running a controlled benchmark. AUTO maps the

complete model when DS4's working-set and live-pressure checks admit it. On a

memory-constrained Mac it falls back to SSD streaming and chooses a bounded

expert cache from the live budget. --resident and --ssd-streaming remain

available as explicit overrides.

DSBox

The root dsbox.json selects the native single-file artifact and declares the

ds4-expert-major format at version 1 with tensor

ds4.expert_major.v1. It pins the compatible DS4 runtime and marks the model

experimental with recommended: false. The manifest also records the former

repository ID so DSBox can recognize installations completed before the Hub

rename.

In DSBox, open Models, refresh the catalog, and explicitly confirm the

download. Downloads are resumable and pinned to one Hub revision; selecting

the model does not start the inference server.

Measured performance

Measurements below use the byte-identified native or canonical GGUF named in

each row on Metal. They describe bounded test workloads, not universal

throughput guarantees.

| Host and mode | Prefill | Generation | Notes |

| --- | ---: | ---: | --- |

| MacBook Pro M5 Pro, 64 GB, resident AUTO | 258.08 t/s | 57.81 t/s | Six-run interleaved optimization A/B; identical greedy output |

| MacBook Pro M1 Pro, 16 GB, canonical migration Q4_K_S, AUTO | 15.04 t/s | 9.77 t/s | Four-run warm median across distinct non-thinking prompts; production bd62a0b |

The definitive 2026-07-18 M1 Pro 16 GB verification used macOS 26.5 with the

battery at 56% and discharging. DS4 was built locally from production main

commit

bd62a0bf36336fc5e3b199ac97bdacf820c4c7f0.

The file was a hardlink to the canonical migration

Qwen3.6-35B-A3B-ds4-Q4_K_S.gguf, with the same published size of

20,808,563,424 bytes. The run used context 8192, AUTO selected 321 experts

for prefill and a 2,241-expert decode target, macro prefill was 64, and no

expert pack was active.

Five distinct non-thinking prompts avoided live-KV reuse. Prompt lengths were

43–46 tokens and generations were 53–64 tokens:

| M1 Pro run state | Prefill | Generation |

| --- | ---: | ---: |

| Cold prompt 1 | 10.56 t/s | 8.24 t/s |

| Warm prompt 2 | 13.85 t/s | 9.71 t/s |

| Warm prompt 3 | 16.24 t/s | 9.83 t/s |

| Warm prompt 4 | 12.61 t/s | 9.25 t/s |

| Warm prompt 5 | 18.41 t/s | 10.29 t/s |

The four-run warm median was 15.04 t/s prefill and 9.77 t/s generation. The

five-run overall median, including the cold run, was 13.85 t/s prefill and

9.71 t/s generation. Output hashes for the first two prompts matched the

earlier fc4badd

run on the same prompts.

Memory pressure remained normal. Free memory moved from 76% to 49%, process

RSS peaked at approximately 3.70 GiB, swapouts remained exactly 2,209,613,

and swap stayed at 830.06 MiB throughout the run. This is a canonical migration

GGUF measurement. The native ExpertMajor v1 file was not copied to the remote

test Mac because only 3.6 GB of disk space was free, so no native-v1-on-16-GB

measurement is claimed.

For historical context only, the earlier fc4badd cold 321-expert, 43+32-token

compatibility smoke recorded 4.06 t/s prefill and 7.03 t/s generation; it is

not normal warmed production throughput. A separate controlled AUTO

1281-expert configuration reached a 9.63 t/s generation peak. That is evidence

for a different cache setting, not part of the definitive production run above.

On the 64 GB host, a separate pinned llama.cpp reference recorded 252.1 t/s

prefill and 60.3 t/s generation. Because it was captured in a different

session, it is context rather than a controlled cross-runtime speedup claim.

Full benchmark commands and retained evidence are published in the

andreaborio/ds4 Qwen benchmark notes.

The native publication gate compared native/sidecar/native at 2K, 4K, 8K,

and 16K. All 12 decode-evidence files were byte-identical, including 16 sampled

token IDs and the complete final logits at every frontier. Exact commands,

hashes, timing spread, and memory observations are in the

v0.1.0 native GGUF gate.

Current limitations

• Metal on Apple Silicon is the qualified backend.
• DS4_QWEN_EXPERIMENTAL_METAL=1 and effective Metal power 100 are required.
• The path supports this normalized text-only artifact, not arbitrary Qwen

models, Qwen3.7 aliases, separate Coder variants, vision, or MTP payloads.

• The ExpertMajor file requires the pinned DS4 runtime or a descendant.

Generic GGUF runtimes must use the canonical file in this repository.

• Neural CPU+GPU layer or expert splitting is not implemented. Neural math is

executed on Metal; CPU work remains for orchestration such as tokenization,

sampling, and SSD cache I/O.

• Metal residency is a budgeted mapping request, not a promise that macOS will

keep every mapped page physically resident under future memory pressure.

• The 16 GB result qualifies the canonical migration GGUF only. It proves a

bounded safe run, not native-v1 performance, resident operation, or desktop

responsiveness for every workload.

License and attribution

The base model is released by the Qwen team under Apache 2.0. This repository

redistributes a normalized quantization derived from the pinned Unsloth GGUF;

see the linked source repositories for their model cards and attribution.