---

license: other

license_name: swift-open-license-v1.0

license_link: LICENSE

base_model: ukisai/Swift-1.5-Qwen3.8-Flash-Next-GSQ-RCO-GGUF

base_model_relation: quantized

library_name: gguf

pipeline_tag: text-generation

language:

- de

- en

tags:

- sakura

- sakura-micro

- gguf

- llama.cpp

- moe

- qwen4exp

- expert-pruning

- strix-halo

- experimental

---

Sakura — Qwen3.8-Flash-Next (Swift 1.5), 352 of 512 experts, IQ3_XXS

<img src="assets/sakura-logo.svg" alt="Sakura logo" width="560" />

> ## GPU memory: about 31.4 GiB of weights — too big for a 32 GB card

> This 3-bit build keeps about 31.4 GiB of weights on the GPU (the 26.8 GiB per-layer token table stays on disk/RAM via lazy loading). It runs on a 64 GB

> Strix Halo (unified memory: in our test 30.4 GiB dedicated plus 10.1 GiB shared memory with 32k context, q8_0 KV cache and the MTP head), but it does not fit a

> 32 GB GPU. For a 32 GB card use the 2.5-bit sibling

> Sakura-Qwen3.8-Flash-Next-Swift-2.5bit-352E-32GB-GGUF (about 26 GiB of GPU weights).

An experimental expert-pruned GGUF of UkisAI's Swift 1.5 fine-tune of Qwen3.8-Flash-Next: 352 of 512 routed experts per layer kept

(chosen on German + English routing), the rest cut out of the ~3-bit (IQ3_XXS, GSQ-RCO) quantization without re-quantizing. Size 58.19 GiB

in two shards. Made for a single 64 GB AMD Strix Halo (Radeon 8060S); German and English are the target languages. Independent community build,

not an official Qwen or UkisAI release.

Part of the Sakura Micro line.

All Sakura Flash-Next builds side by side (what differs, measured table, runtime notes, tools): Sakura-Qwen3.8-Flash-Next-Overview.

Provenance

• Base: Qwen/Qwen3.8-Flash-Next (arch qwen4exp, 48 layers, 512 experts, 10 active).
• Fine-tune + quantization: ukisai/Swift-1.5-Qwen3.8-Flash-Next-GSQ-RCO-GGUF (IQ3_XXS tier).

Swift is the reason for this build: it uses clearly fewer thinking tokens than the base model.

• What this build changes: expert slicing only (see NOTICE.md). Kept experts per layer: kept_experts_K352.json.
• Expert selection: router statistics of the Swift model itself on a German + English calibration mix (routing mass); an expert is kept if it is hot in

either language. Measured with llama-expert-stats, a small tool that runs the model and counts router use (much faster than llama-imatrix).

Files

| File | Size |

|---|---:|

| Sakura-Qwen3.8-Flash-Next-Swift-K352-IQ3_XXS-00001-of-00002.gguf | 34.32 GiB (dense weights, the large per-layer token table and part of the experts) |

| Sakura-Qwen3.8-Flash-Next-Swift-K352-IQ3_XXS-00002-of-00002.gguf | 23.87 GiB (rest of the routed experts) |

SHA256SUMS lists the checksums. Load shard 1; llama.cpp finds shard 2 itself. Use -lm dio --lazy-mode on (or your build's equivalent) so the large token table is

read lazily instead of held in memory.

Measured results (own tests, this exact file)

| Check | Result |

|---|---|

| Language gate, 14 German + 14 English prompts: share of wrong-language words | DE 0.000 / EN 0.005 (gate: each <= 0.06) |

| Loops / CJK answers | 0 / 0 |

| Held-out perplexity (German mix / English mix) | 2.0514 / 1.8231 (details below) |

| Decode / prefill speed | 34.5 tok/s decode, 314 tok/s prefill with speculation (Vulkan, see the speed table) |

| Coding tasks (own 8 LeetCode/AtCoder-style tasks) | 8/8 passed (14,443 generated tokens, about 8.6 min) |

| Tasks 9-20 of our frozen 32-task set (harder tasks) | 11/12 passed (98,937 generated tokens), see below |

The language gate and the perplexity were measured with the official llama.cpp build b11259 (Vulkan); speed and coding tasks with the fork build described below.

<details><summary>Held-out perplexity raw</summary>

{"de": 2.0514, "en": 1.8231}

</details>

llama-perplexity, context 512, 40 chunks of own German and English held-out texts (not in the calibration data). For orientation, same texts: Swift 512-expert IQ2_XS

2.0605 / 1.8245, Swift K352 IQ2_XS 2.0681 / 1.8229, the same recipe cut to K320 (IQ3_XXS) 2.0437 / 1.8284. The differences between these builds span about 1.2 % (German);

perplexity cannot separate them. (No perplexity of the unpruned IQ3_XXS: the 76 GB file does not fit the test machine.)

Runtime used for speed and coding tests

The shared MTP head needs a llama.cpp fork. We used github.com/danielhanchen/llama.cpp, branch qwen4exp/mtp, commit 6fcaa16 (21 Sep 2026), built by us for Vulkan

(MSVC, Vulkan SDK 1.4.357: cmake -B build -G Ninja -DCMAKE_BUILD_TYPE=Release -DGGML_VULKAN=ON -DBUILD_SHARED_LIBS=ON -DLLAMA_OPENSSL=OFF -DLLAMA_CURL=OFF).

Speed (AMD Strix Halo, 64 GB, Radeon 8060S, Windows, Vulkan)

Same ~1700-token prompt each time, 32k context, KV q8_0, -lm dio --lazy-mode on, shared MTP sidecar mtp-Qwen3.8-Flash-Next-shared-Q4_K_M.gguf (1.78 GiB):

| -b/-ub | Speculation | Prefill | Decode |

|---|---|---:|---:|

| 2048 | MTP n-max 2 | 314.5 tok/s | 34.5 tok/s |

| 2048 | MTP n-max 3 | 316.7 tok/s | 33.6 tok/s |

| 512 | MTP n-max 3 | 257.5 tok/s | 33.4 tok/s |

| 512 | MTP n-max 2 | 253.4 tok/s | 32.8 tok/s |

Without speculation in the official llama.cpp b11259 (Vulkan) the same file decodes at about 21 tok/s (measured during a coding task).

ROCm is not recommended for this file: on Windows the decode collapses to about 5 tok/s as soon as the resident weights

(file size minus the token table, here about 31 GiB) exceed the dedicated GPU memory; smaller cuts (K304 and below) stay fast on ROCm but were not evaluated for quality.

Coding check (own tasks, thinking budget 10000, max 12288 tokens, temperature 0.6, top-p 0.95, top-k 20, seeds 1000 + task number, one run each)

• Eight anchor tasks: 8/8 passed, 30 to 34 tok/s decode, including the hardest one of the set (3,664 tokens).
• Tasks 9 to 20 of the frozen 32-task set (the harder part; same questions, grader and seeds for every model we test): 11 of 12 passed. Task 15 failed (a hidden test returned 12 instead of 8).

For orientation, in our earlier runs a dense Swift 1.5 27B Q5_K_M solved 6 of these 12 tasks and a 35B-A3B coder 8 of 12. These are not like-for-like numbers: the 27B ran without a

thinking budget, this build with a budget of 10,000 tokens, and every task was run once. Task 21 of the 32-task set was not run.

Single runs on one machine; they are indicators, not a benchmark suite. Vision was not tested (no mmproj is included; use the BF16 projector from the Swift repository).

Known limits

• Cutting experts without retraining costs some quality; the size of that cost was checked on German/English text and a small coding set only.
• Speculation needs the fork above. The shared MTP sidecar is not part of this GGUF and does not load in official llama.cpp (b11259 and b11294 fail with token_embd.weight not found;

the upstream loader has no MTP-only path). Unsloth publishes the shared sidecar as MTP/mtp-Qwen3.8-Flash-Next-shared-Q8_0.gguf in unsloth/Qwen3.8-Flash-Next-GGUF;

we measured with a Q4_K_M sidecar and did not test the Q8_0 one. Without speculation the file works in official llama.cpp.

• An earlier Unsloth Vulkan package (build 10830, commit 0cd7a79f5) gave wrong results with this file in our tests: after the thinking budget ran out, the model kept

reasoning in the answer part instead of writing the code (4 of 4 variants failed the same task that the fork and the official b11259 solved). We do not recommend that build for this model.

• Licenses: Swift Open License v1.0 (commercial-use threshold, section 5) and Qwen Community License 1.0 (Model-as-a-Service / AI Work Assistant clause). Read both before commercial use.

Run example

llama-server -m Sakura-Qwen3.8-Flash-Next-Swift-K352-IQ3_XXS-00001-of-00002.gguf --jinja -fa on -ngl 99 -c 32768 -ctk q8_0 -ctv q8_0 -b 2048 -ub 2048 -lm dio --lazy-mode on --device Vulkan0 \
  --spec-type draft-mtp --spec-draft-model mtp-Qwen3.8-Flash-Next-shared-Q4_K_M.gguf --spec-draft-ngl 99 --spec-draft-n-max 2 --spec-draft-n-min 0

This is the command line used for the speed numbers with the fork build. Leave out the last line to run without speculation.

---

<details><summary>中文说明 · 樱花 (Simplified Chinese)</summary>

> English above. 本节为上文的中文翻译(Sakura = 樱花 yīnghuā);完整的独立中文版见 README_zh.md。

Sakura — Qwen3.8-Flash-Next (Swift 1.5),512 个专家中保留 352 个,IQ3_XXS

<img src="assets/sakura-logo.svg" alt="Sakura logo" width="560" />

> ## GPU 内存:权重约 31.4 GiB — 对 32 GB 显卡来说太大

> 这个 3 比特构建在 GPU 上保留约 31.4 GiB 的权重(26.8 GiB 的逐层 token 表通过懒加载留在磁盘/内存中)。它可以在 64 GB 的

> Strix Halo(统一内存:在我们的测试中,32k 上下文、q8_0 KV 缓存和 MTP 头的情况下占用 30.4 GiB 专用内存加 10.1 GiB 共享内存)上运行,但 不能 放进

> 32 GB 的 GPU。对于 32 GB 的显卡,请使用 2.5 比特的同系列版本

> Sakura-Qwen3.8-Flash-Next-Swift-2.5bit-352E-32GB-GGUF(GPU 权重约 26 GiB)。

这是 UkisAI 对 Qwen3.8-Flash-Next 的 Swift 1.5 微调版本的一个实验性专家剪枝 GGUF:每层 512 个路由专家中保留 352 个

(依据德语 + 英语的路由情况选出),其余的从约 3 比特(IQ3_XXS,GSQ-RCO)量化中被裁掉,没有重新量化。大小 58.19 GiB,

分为两个分片。为单台 64 GB 的 AMD Strix Halo(Radeon 8060S)制作;目标语言是德语和英语。这是独立的社区构建版本,

不是 Qwen 或 UkisAI 的官方发布。

属于 Sakura Micro 系列。

所有 Sakura Flash-Next 构建的并排比较(区别、实测表格、运行时说明、工具):Sakura-Qwen3.8-Flash-Next-Overview。

来源

• 基础模型:Qwen/Qwen3.8-Flash-Next(架构 qwen4exp,48 层,512 个专家,激活 10 个)。
• 微调 + 量化:ukisai/Swift-1.5-Qwen3.8-Flash-Next-GSQ-RCO-GGUF(IQ3_XXS 档位)。

Swift 是制作这个构建的原因:它使用的思考 token 明显少于基础模型。

• 本构建的改动:仅为专家切片(见 NOTICE.md)。每层保留的专家:kept_experts_K352.json。
• 专家选择:Swift 模型自身在德语 + 英语校准混合文本上的路由器统计(路由质量);只要一个专家在任一语言中是热点,就予以保留。使用 llama-expert-stats 测量,这是一个运行模型并统计路由器使用情况的小工具(比 llama-imatrix 快得多)。

文件

| 文件 | 大小 |

|---|---:|

| Sakura-Qwen3.8-Flash-Next-Swift-K352-IQ3_XXS-00001-of-00002.gguf | 34.32 GiB(稠密权重、巨大的逐层 token 表和一部分专家) |

| Sakura-Qwen3.8-Flash-Next-Swift-K352-IQ3_XXS-00002-of-00002.gguf | 23.87 GiB(其余的路由专家) |

SHA256SUMS 列出了校验和。加载分片 1;llama.cpp 会自己找到分片 2。请使用 -lm dio --lazy-mode on(或你所用构建中的等效选项),使巨大的 token 表被懒加载读取,而不是常驻内存。

测量结果(自己的测试,这一确切的文件)

| 检查项 | 结果 |

|---|---|

| 语言关卡,14 个德语 + 14 个英语提示:使用了错误语言的词所占比例 | DE 0.000 / EN 0.005(关卡:各 <= 0.06) |

| 循环 / CJK 回答 | 0 / 0 |

| 留出困惑度(德语混合 / 英语混合) | 2.0514 / 1.8231(详见下文) |

| 解码 / 预填充速度 | 使用推测解码时 34.5 tok/s 解码,314 tok/s 预填充(Vulkan,见速度表) |

| 编程任务(自己的 8 个 LeetCode/AtCoder 风格任务) | 8/8 通过(生成 14,443 个 token,约 8.6 分钟) |

| 我们冻结的 32 个任务集中的任务 9-20(更难的任务) | 11/12 通过(生成 98,937 个 token),见下文 |

语言关卡和困惑度是用官方 llama.cpp 构建 b11259(Vulkan)测得的;速度和编程任务使用下文所述的分支构建。

<details><summary>留出困惑度原始值</summary>

{"de": 2.0514, "en": 1.8231}

</details>

llama-perplexity,上下文 512,在自己的德语和英语留出文本(不在校准数据中)上取 40 个块。作为参考,在相同文本上:Swift 512 专家 IQ2_XS 为

2.0605 / 1.8245,Swift K352 IQ2_XS 为 2.0681 / 1.8229,相同配方裁剪到 K320(IQ3_XXS)为 2.0437 / 1.8284。这些构建之间的差异约为 1.2 %(德语);

困惑度无法将它们区分开。(没有未剪枝的 IQ3_XXS 的困惑度:76 GB 的文件放不进测试机器。)

用于速度和编程测试的运行时

共享 MTP 头需要 llama.cpp 分支。我们使用了 github.com/danielhanchen/llama.cpp,分支 qwen4exp/mtp,提交 6fcaa16(2026 年 9 月 21 日),由我们自己为 Vulkan 构建

(MSVC,Vulkan SDK 1.4.357:cmake -B build -G Ninja -DCMAKE_BUILD_TYPE=Release -DGGML_VULKAN=ON -DBUILD_SHARED_LIBS=ON -DLLAMA_OPENSSL=OFF -DLLAMA_CURL=OFF)。

速度(AMD Strix Halo,64 GB,Radeon 8060S,Windows,Vulkan)

每次使用相同的约 1700 个 token 的提示,32k 上下文,KV q8_0,-lm dio --lazy-mode on,共享 MTP 旁路文件 mtp-Qwen3.8-Flash-Next-shared-Q4_K_M.gguf(1.78 GiB):

| -b/-ub | 推测解码 | 预填充 | 解码 |

|---|---|---:|---:|

| 2048 | MTP n-max 2 | 314.5 tok/s | 34.5 tok/s |

| 2048 | MTP n-max 3 | 316.7 tok/s | 33.6 tok/s |

| 512 | MTP n-max 3 | 257.5 tok/s | 33.4 tok/s |

| 512 | MTP n-max 2 | 253.4 tok/s | 32.8 tok/s |

在官方 llama.cpp b11259(Vulkan)中不使用推测解码时,同一个文件的解码速度约为 21 tok/s(在一次编程任务中测得)。

不推荐在此文件上使用 ROCm: 在 Windows 上,一旦常驻权重(文件大小减去 token 表,这里约为 31 GiB)超过专用 GPU 内存,解码速度就会骤降到约 5 tok/s;较小的裁剪(K304 及以下)在 ROCm 上仍然很快,但没有评估其质量。

编程检查(自己的任务,思考预算 10000,最多 12288 个 token,temperature 0.6,top-p 0.95,top-k 20,随机种子 1000 + 任务编号,各运行一次)

• 八个锚点任务:8/8 通过,解码 30 至 34 tok/s,包括这一组中最难的一个(3,664 个 token)。
• 冻结的 32 个任务集中的任务 9 至 20(更难的部分;我们测试的每个模型都使用相同的问题、评分器和随机种子):12 个中通过 11 个。任务 15 失败(一个隐藏测试返回了 12 而不是 8)。

作为参考,在我们较早的运行中,稠密的 Swift 1.5 27B Q5_K_M 解出了这 12 个任务中的 6 个,一个 35B-A3B 编程模型解出了 12 个中的 8 个。这些不是同口径的数据:27B 是在没有思考预算的情况下运行的,而这个构建的预算是 10,000 个 token,并且每个任务只运行了一次。32 个任务集中的任务 21 没有运行。

单次运行,在一台机器上;它们是指示性的,不是基准测试套件。视觉能力未经测试(未包含 mmproj;请使用 Swift 仓库中的 BF16 投影器)。

已知局限

• 不经再训练就裁剪专家会损失一些质量;这一损失的大小只在德语/英语文本和一个小型编程集上检查过。
• 推测解码需要上述分支。 共享 MTP 旁路文件不属于这个 GGUF,在官方 llama.cpp 中无法加载(b11259 和 b11294 会以 token_embd.weight not found 失败;上游加载器没有仅含 MTP 的路径)。Unsloth 在 unsloth/Qwen3.8-Flash-Next-GGUF 中把共享旁路文件发布为 MTP/mtp-Qwen3.8-Flash-Next-shared-Q8_0.gguf;我们使用 Q4_K_M 旁路文件测量,没有测试 Q8_0 的那个。不使用推测解码时,该文件可在官方 llama.cpp 中运行。
• 较早的 Unsloth Vulkan 包(构建 10830,提交 0cd7a79f5)在我们的测试中对此文件给出了错误的结果: 在思考预算用完之后,模型仍在回答部分继续推理,而没有写出代码(4 个变体中有 4 个在同一个任务上失败,而该任务分支和官方 b11259 都解出了)。我们不推荐把这个构建用于该模型。
• 许可证:Swift Open License v1.0(商业使用门槛,第 5 节) 以及 Qwen Community License 1.0(Model-as-a-Service / AI Work Assistant 条款)。商业使用之前请阅读两者。

运行示例

llama-server -m Sakura-Qwen3.8-Flash-Next-Swift-K352-IQ3_XXS-00001-of-00002.gguf --jinja -fa on -ngl 99 -c 32768 -ctk q8_0 -ctv q8_0 -b 2048 -ub 2048 -lm dio --lazy-mode on --device Vulkan0 \
  --spec-type draft-mtp --spec-draft-model mtp-Qwen3.8-Flash-Next-shared-Q4_K_M.gguf --spec-draft-ngl 99 --spec-draft-n-max 2 --spec-draft-n-min 0

这是使用分支构建得出速度数据时所用的命令行。省略最后一行即可在不使用推测解码的情况下运行。

</details>