---

license: other

license_name: swift-open-license-v1.0

license_link: LICENSE

base_model: ukisai/Swift-1.5-Qwen3.8-Flash-Next-GSQ-RCO-GGUF

base_model_relation: finetune

library_name: gguf

pipeline_tag: text-generation

language:

- de

- en

tags:

- sakura

- sakura-micro

- gguf

- llama.cpp

- moe

- qwen4exp

- expert-pruning

- strix-halo

- experimental

---

Sakura — Qwen3.8-Flash-Next (Swift 1.5), 352 of 512 experts, IQ2_XS

<img src="assets/sakura-logo.png" alt="Sakura logo" width="280" />

> ## Fits a 32 GB GPU

> About 26 GiB of weights sit on the GPU; the 26.8 GiB per-layer token table stays on disk/RAM (lazy loading, -lm dio --lazy-mode on).

> With 32k context, q8_0 KV cache, the MTP head and -ub 2048 the server used 29.9 GiB of dedicated GPU memory (plus 5.7 GiB of shared memory) in our test; without the MTP head it needs about 1.8 GiB less.

> Measured on a Radeon 8060S with a 32 GiB dedicated carve-out (Strix Halo); we did not test a discrete 32 GB card.

> The 3-bit sibling needs about 31.4 GiB of GPU weights and does not fit a 32 GB card.

An experimental expert-pruned GGUF of UkisAI's Swift 1.5 fine-tune of Qwen3.8-Flash-Next: 352 of 512 routed experts per layer kept

(chosen on German + English routing), the rest cut out of the about 2.5-bit (IQ2_XS, GSQ-RCO) quantization without re-quantizing. Size 53.08 GiB

in two shards. This is the smaller sibling of the 3-bit build

(Sakura-Qwen3.8-Flash-Next-Swift-DE-EN-K352-IQ3_XXS-GGUF): about 26 GiB of weights sit on the GPU, so it also fits

a 32 GB card. Made for a single 64 GB AMD Strix Halo (Radeon 8060S); German and English are the target languages. Independent community build,

not an official Qwen or UkisAI release.

Part of the Sakura Micro line.

Provenance

• Base: Qwen/Qwen3.8-Flash-Next (arch qwen4exp, 48 layers, 512 experts, 10 active).
• Fine-tune + quantization: ukisai/Swift-1.5-Qwen3.8-Flash-Next-GSQ-RCO-GGUF (IQ2_XS tier).

Swift is the reason for this build: it uses clearly fewer thinking tokens than the base model.

• What this build changes: expert slicing only (see NOTICE.md). Kept experts per layer: kept_experts_K352.json.
• Expert selection: router statistics (routing mass) of the ISTA-quantized Qwen3.8-Flash-Next on a German + English calibration mix; an expert is kept if it is hot in

either language. The same measurement on the Swift model itself gives a keep set that overlaps this one by 98 %. Measured with llama-expert-stats,

a small tool that runs the model and counts router use (much faster than llama-imatrix).

Files

| File | Size |

|---|---:|

| Sakura-Qwen3.8-Flash-Next-Swift-K352-IQ2_XS-00001-of-00002.gguf | 34.31 GiB (dense weights, the large per-layer token table and part of the experts) |

| Sakura-Qwen3.8-Flash-Next-Swift-K352-IQ2_XS-00002-of-00002.gguf | 18.77 GiB (rest of the routed experts) |

SHA256SUMS lists the checksums. Load shard 1; llama.cpp finds shard 2 itself. Use -lm dio --lazy-mode on (or your build's equivalent) so the large token table is

read lazily instead of held in memory.

Measured results (own tests, this exact file)

| Check | Result |

|---|---|

| Language gate, 14 German + 14 English prompts: share of wrong-language words | DE 0.005 / EN 0.01 (gate: each <= 0.06) |

| Loops / CJK answers | 0 / 0 |

| Held-out perplexity (German mix / English mix) | 2.0681 / 1.8229 |

| Decode / prefill speed | 37.6 tok/s decode, 238 tok/s prefill with speculation (Vulkan, see the speed table) |

| Coding tasks (own 8 LeetCode/AtCoder-style tasks) | 7/8 passed (33,430 generated tokens, about 18.0 min) |

| Harder tasks (tasks a 27B dense model missed) | 3 of the 5 that were run passed |

The language gate and the perplexity were measured with the official llama.cpp build b11259 (Vulkan); speed and coding tasks with the fork build described below.

Held-out perplexity: llama-perplexity, context 512, 40 chunks of own German and English held-out texts (not in the calibration data), raw {"de": 2.0681, "en": 1.8229}.

For orientation, same texts: Swift 512-expert IQ2_XS 2.0605 / 1.8245, the 3-bit sibling of this build 2.0514 / 1.8231. The differences between these builds are within about +-1 %;

perplexity cannot separate them.

Runtime used for speed and coding tests

The shared MTP head needs a llama.cpp fork. We used github.com/danielhanchen/llama.cpp, branch qwen4exp/mtp, commit 6fcaa16 (21 Sep 2026), built by us for Vulkan

(MSVC, Vulkan SDK 1.4.357: cmake -B build -G Ninja -DCMAKE_BUILD_TYPE=Release -DGGML_VULKAN=ON -DBUILD_SHARED_LIBS=ON -DLLAMA_OPENSSL=OFF -DLLAMA_CURL=OFF).

Speed (AMD Strix Halo, 64 GB, Radeon 8060S, Windows, Vulkan)

Same ~1700-token prompt each time, 32k context, KV q8_0, -lm dio --lazy-mode on, shared MTP sidecar mtp-Qwen3.8-Flash-Next-shared-Q4_K_M.gguf (1.78 GiB):

| -b/-ub | Speculation | Prefill | Decode |

|---|---|---:|---:|

| 2048 | MTP n-max 2 | 238.0 tok/s | 37.6 tok/s |

| 2048 | MTP n-max 3 | 253.3 tok/s | 35.2 tok/s |

Without speculation the same file decodes at about 24 tok/s in official llama.cpp (Vulkan, b11259 and b11297). On ROCm (community halo-box build, no speculation) we measured

322 tok/s prefill and 21.6 tok/s decode for this file, because its resident weights (about 26 GiB) fit the dedicated GPU memory; the 3-bit sibling does not and collapses there.

Coding check (own tasks, thinking budget 10000, max 12288 tokens, temperature 0.6, top-p 0.95, top-k 20, seeds 1000 + task number, one run each)

• Eight anchor tasks: 7/8 passed. Not passed: task(s) 8.
• Harder tasks that a dense 27B model (Swift 1.5 27B Q5_K_M) missed in our protocol: passed: 14, 16, 19; not passed: 9, 15; task 21 was not run.

Single runs on one machine; they are indicators, not a benchmark suite. Vision was not tested (no mmproj is included; use the BF16 projector from the Swift repository).

Known limits

• Cutting experts without retraining costs some quality; the size of that cost was checked on German/English text and a small coding set only. At 2.5 bit the quantization itself

is coarser than in the 3-bit sibling; if you have the memory, prefer the 3-bit build.

• Speculation needs the fork above. The shared MTP sidecar is not part of this GGUF and does not load in official llama.cpp (b11259 and b11294 fail with token_embd.weight not found;

the upstream loader has no MTP-only path). Unsloth publishes the shared sidecar as MTP/mtp-Qwen3.8-Flash-Next-shared-Q8_0.gguf in unsloth/Qwen3.8-Flash-Next-GGUF;

we measured with a Q4_K_M sidecar and did not test the Q8_0 one. Without speculation the file works in official llama.cpp.

• An earlier Unsloth Vulkan package (build 10830, commit 0cd7a79f5) gave wrong results with the 3-bit sibling in our tests: after the thinking budget ran out, the model kept

reasoning in the answer part instead of writing the code. We did not test this file on that build and do not recommend it for this model family.

• Licenses: Swift Open License v1.0 (commercial-use threshold, section 5) and Qwen Community License 1.0 (Model-as-a-Service / AI Work Assistant clause). Read both before commercial use.

Run example

llama-server -m Sakura-Qwen3.8-Flash-Next-Swift-K352-IQ2_XS-00001-of-00002.gguf --jinja -fa on -ngl 99 -c 32768 -ctk q8_0 -ctv q8_0 -b 2048 -ub 2048 -lm dio --lazy-mode on --device Vulkan0 \
  --spec-type draft-mtp --spec-draft-model mtp-Qwen3.8-Flash-Next-shared-Q4_K_M.gguf --spec-draft-ngl 99 --spec-draft-n-max 2 --spec-draft-n-min 0

This is the command line used for the speed numbers with the fork build. Leave out the last line to run without speculation.