---

license: apache-2.0

license_link: https://huggingface.co/Qwen/Qwen3.6-35B-A3B/blob/main/LICENSE

base_model: mlx-community/Qwen3.6-35B-A3B-8bit

pipeline_tag: text-generation

tags:

- gguf

- moe

- knowledge-distillation

- mixture-of-experts

---

!Saikei

Qwen3.6-35B-A3B-Saikei-8bit-gguf

**Q8_0 GGUF of Qwen3.6-35B-A3B-Saikei — the intact 35B MoE improved by a

short knowledge-distillation pass from Qwen3.8-2.4T-A95B. Nothing pruned,

nothing removed; runs anywhere llama.cpp runs.**

Saikei (栽景) are Japan's planted landscapes — miniature living scenes

grown whole on a tray, where nothing is cut away: the art is in the

cultivation. Where our Niwaki releases prune the tree, this model prunes

nothing. It is the reference model, refined in place.

A paper with the full method and measurements is coming soon.

What this is

The unmodified 35B/3B-active MoE distilled (a 3M-token plan stopped at

~2.1M tokens at a probe plateau; released checkpoint step 1769, ~1.8M

tokens) against top-20

teacher logprobs from the 2.4T teacher (tokenizer-identical); trainables

(per-layer output maps + shared experts) folded back, then exported

bf16 → quantized Q8_0 with llama.cpp. See the

mlx release

for the paired MLX-protocol benchmarks (wt2/c4/tasks/gsm8k vs reference).

Benchmarks (llama-perplexity space)

Measured on this exact Q8_0 file (llama-perplexity, full-file

WikiText-2 test; C4 slice, 512 chunks):

| file | wiki ppl ↓ | c4 ppl ↓ |

|---|---|---|

| Qwen3.6-35B-A3B-Saikei-Q8_0.gguf (34 GB) | 6.59 | 11.64 |

Note llama-perplexity numbers are not directly comparable to the MLX

protocol on the mlx card — different windows and tokenizer handling;

within-file comparisons are exact.

Usage (llama.cpp)

llama-cli -m Qwen3.6-35B-A3B-Saikei-Q8_0.gguf -cnv \
  -p "Explain speculative decoding in two paragraphs."

Provenance

Distillation heal of mlx-community/Qwen3.6-35B-A3B-8bit (teacher

Qwen/Qwen3.8-2.4T-A95B), folded, exported via convert_hf_to_gguf.py

(bf16, --no-mtp), quantized Q8_0 (llama.cpp).