---

license: other

license_name: swift-open-license-v1.0

license_link: LICENSE

base_model: ukisai/Swift-1.5-Qwen3.8-Flash-Next-GSQ-RCO-GGUF

base_model_relation: finetune

library_name: gguf

pipeline_tag: text-generation

language:

- de

- en

tags:

- sakura

- sakura-micro

- gguf

- llama.cpp

- moe

- qwen4exp

- expert-pruning

- strix-halo

- experimental

---

Sakura — Qwen3.8-Flash-Next (Swift 1.5), 352 of 512 experts, IQ3_XXS

<img src="assets/sakura-logo.png" alt="Sakura logo" width="280" />

> ## GPU memory: about 31.4 GiB of weights — too big for a 32 GB card

> This 3-bit build keeps about 31.4 GiB of weights on the GPU (the 26.8 GiB per-layer token table stays on disk/RAM via lazy loading). It runs on a 64 GB

> Strix Halo (unified memory: in our test 30.4 GiB dedicated plus 10.1 GiB shared memory with 32k context, q8_0 KV cache and the MTP head), but it does not fit a

> 32 GB GPU. For a 32 GB card use the 2.5-bit sibling

> Sakura-Qwen3.8-Flash-Next-Swift-DE-EN-K352-IQ2_XS-GGUF (about 26 GiB of GPU weights).

An experimental expert-pruned GGUF of UkisAI's Swift 1.5 fine-tune of Qwen3.8-Flash-Next: 352 of 512 routed experts per layer kept

(chosen on German + English routing), the rest cut out of the ~3-bit (IQ3_XXS, GSQ-RCO) quantization without re-quantizing. Size 58.19 GiB

in two shards. Made for a single 64 GB AMD Strix Halo (Radeon 8060S); German and English are the target languages. Independent community build,

not an official Qwen or UkisAI release.

Part of the Sakura Micro line.

Provenance

• Base: Qwen/Qwen3.8-Flash-Next (arch qwen4exp, 48 layers, 512 experts, 10 active).
• Fine-tune + quantization: ukisai/Swift-1.5-Qwen3.8-Flash-Next-GSQ-RCO-GGUF (IQ3_XXS tier).

Swift is the reason for this build: it uses clearly fewer thinking tokens than the base model.

• What this build changes: expert slicing only (see NOTICE.md). Kept experts per layer: kept_experts_K352.json.
• Expert selection: router statistics of the Swift model itself on a German + English calibration mix (routing mass); an expert is kept if it is hot in

either language. Measured with llama-expert-stats, a small tool that runs the model and counts router use (much faster than llama-imatrix).

Files

| File | Size |

|---|---:|

| Sakura-Qwen3.8-Flash-Next-Swift-K352-IQ3_XXS-00001-of-00002.gguf | 34.32 GiB (dense weights, the large per-layer token table and part of the experts) |

| Sakura-Qwen3.8-Flash-Next-Swift-K352-IQ3_XXS-00002-of-00002.gguf | 23.87 GiB (rest of the routed experts) |

SHA256SUMS lists the checksums. Load shard 1; llama.cpp finds shard 2 itself. Use -lm dio --lazy-mode on (or your build's equivalent) so the large token table is

read lazily instead of held in memory.

Measured results (own tests, this exact file)

| Check | Result |

|---|---|

| Language gate, 14 German + 14 English prompts: share of wrong-language words | DE 0.000 / EN 0.005 (gate: each <= 0.06) |

| Loops / CJK answers | 0 / 0 |

| Held-out perplexity (German mix / English mix) | 2.0514 / 1.8231 (details below) |

| Decode / prefill speed | 34.5 tok/s decode, 314 tok/s prefill with speculation (Vulkan, see the speed table) |

| Coding tasks (own 8 LeetCode/AtCoder-style tasks) | 8/8 passed (14,443 generated tokens, about 8.6 min) |

| Tasks 9-20 of our frozen 32-task set (harder tasks) | 11/12 passed (98,937 generated tokens), see below |

The language gate and the perplexity were measured with the official llama.cpp build b11259 (Vulkan); speed and coding tasks with the fork build described below.

<details><summary>Held-out perplexity raw</summary>

{"de": 2.0514, "en": 1.8231}

</details>

llama-perplexity, context 512, 40 chunks of own German and English held-out texts (not in the calibration data). For orientation, same texts: Swift 512-expert IQ2_XS

2.0605 / 1.8245, Swift K352 IQ2_XS 2.0681 / 1.8229, the same recipe cut to K320 (IQ3_XXS) 2.0437 / 1.8284. The differences between these builds span about 1.2 % (German);

perplexity cannot separate them. (No perplexity of the unpruned IQ3_XXS: the 76 GB file does not fit the test machine.)

Runtime used for speed and coding tests

The shared MTP head needs a llama.cpp fork. We used github.com/danielhanchen/llama.cpp, branch qwen4exp/mtp, commit 6fcaa16 (21 Sep 2026), built by us for Vulkan

(MSVC, Vulkan SDK 1.4.357: cmake -B build -G Ninja -DCMAKE_BUILD_TYPE=Release -DGGML_VULKAN=ON -DBUILD_SHARED_LIBS=ON -DLLAMA_OPENSSL=OFF -DLLAMA_CURL=OFF).

Speed (AMD Strix Halo, 64 GB, Radeon 8060S, Windows, Vulkan)

Same ~1700-token prompt each time, 32k context, KV q8_0, -lm dio --lazy-mode on, shared MTP sidecar mtp-Qwen3.8-Flash-Next-shared-Q4_K_M.gguf (1.78 GiB):

| -b/-ub | Speculation | Prefill | Decode |

|---|---|---:|---:|

| 2048 | MTP n-max 2 | 314.5 tok/s | 34.5 tok/s |

| 2048 | MTP n-max 3 | 316.7 tok/s | 33.6 tok/s |

| 512 | MTP n-max 3 | 257.5 tok/s | 33.4 tok/s |

| 512 | MTP n-max 2 | 253.4 tok/s | 32.8 tok/s |

Without speculation in the official llama.cpp b11259 (Vulkan) the same file decodes at about 21 tok/s (measured during a coding task).

ROCm is not recommended for this file: on Windows the decode collapses to about 5 tok/s as soon as the resident weights

(file size minus the token table, here about 31 GiB) exceed the dedicated GPU memory; smaller cuts (K304 and below) stay fast on ROCm but were not evaluated for quality.

Coding check (own tasks, thinking budget 10000, max 12288 tokens, temperature 0.6, top-p 0.95, top-k 20, seeds 1000 + task number, one run each)

• Eight anchor tasks: 8/8 passed, 30 to 34 tok/s decode, including the hardest one of the set (3,664 tokens).
• Tasks 9 to 20 of the frozen 32-task set (the harder part; same questions, grader and seeds for every model we test): 11 of 12 passed. Task 15 failed (a hidden test returned 12 instead of 8).

For orientation, in our earlier runs a dense Swift 1.5 27B Q5_K_M solved 6 of these 12 tasks and a 35B-A3B coder 8 of 12. These are not like-for-like numbers: the 27B ran without a

thinking budget, this build with a budget of 10,000 tokens, and every task was run once. Task 21 of the 32-task set was not run.

Single runs on one machine; they are indicators, not a benchmark suite. Vision was not tested (no mmproj is included; use the BF16 projector from the Swift repository).

Known limits

• Cutting experts without retraining costs some quality; the size of that cost was checked on German/English text and a small coding set only.
• Speculation needs the fork above. The shared MTP sidecar is not part of this GGUF and does not load in official llama.cpp (b11259 and b11294 fail with token_embd.weight not found;

the upstream loader has no MTP-only path). Unsloth publishes the shared sidecar as MTP/mtp-Qwen3.8-Flash-Next-shared-Q8_0.gguf in unsloth/Qwen3.8-Flash-Next-GGUF;

we measured with a Q4_K_M sidecar and did not test the Q8_0 one. Without speculation the file works in official llama.cpp.

• An earlier Unsloth Vulkan package (build 10830, commit 0cd7a79f5) gave wrong results with this file in our tests: after the thinking budget ran out, the model kept

reasoning in the answer part instead of writing the code (4 of 4 variants failed the same task that the fork and the official b11259 solved). We do not recommend that build for this model.

• Licenses: Swift Open License v1.0 (commercial-use threshold, section 5) and Qwen Community License 1.0 (Model-as-a-Service / AI Work Assistant clause). Read both before commercial use.

Run example

llama-server -m Sakura-Qwen3.8-Flash-Next-Swift-K352-IQ3_XXS-00001-of-00002.gguf --jinja -fa on -ngl 99 -c 32768 -ctk q8_0 -ctv q8_0 -b 2048 -ub 2048 -lm dio --lazy-mode on --device Vulkan0 \
  --spec-type draft-mtp --spec-draft-model mtp-Qwen3.8-Flash-Next-shared-Q4_K_M.gguf --spec-draft-ngl 99 --spec-draft-n-max 2 --spec-draft-n-min 0

This is the command line used for the speed numbers with the fork build. Leave out the last line to run without speculation.