---

license: other

license_name: swift-open-license-v1.0

license_link: LICENSE

base_model: ukisai/Swift-1.5-Qwen3.8-Flash-Next-GSQ-RCO-GGUF

base_model_relation: quantized

library_name: gguf

pipeline_tag: text-generation

language:

- de

- en

tags:

- sakura

- sakura-micro

- gguf

- llama.cpp

- moe

- qwen4exp

- expert-pruning

- strix-halo

- experimental

---

Sakura — Qwen3.8-Flash-Next (Swift 1.5), 352 of 512 experts, IQ2_XS

<img src="assets/sakura-logo.svg" alt="Sakura logo" width="560" />

> ## Fits a 32 GB GPU

> About 26 GiB of weights sit on the GPU; the 26.8 GiB per-layer token table stays on disk/RAM (lazy loading, -lm dio --lazy-mode on).

> With 32k context, q8_0 KV cache, the MTP head and -ub 2048 the server used 29.9 GiB of dedicated GPU memory (plus 5.7 GiB of shared memory) in our test; without the MTP head it needs about 1.8 GiB less.

> Measured on a Radeon 8060S with a 32 GiB dedicated carve-out (Strix Halo); we did not test a discrete 32 GB card.

> The 3-bit sibling needs about 31.4 GiB of GPU weights and does not fit a 32 GB card.

An experimental expert-pruned GGUF of UkisAI's Swift 1.5 fine-tune of Qwen3.8-Flash-Next: 352 of 512 routed experts per layer kept

(chosen on German + English routing), the rest cut out of the about 2.5-bit (IQ2_XS, GSQ-RCO) quantization without re-quantizing. Size 53.08 GiB

in two shards. This is the smaller sibling of the 3-bit build

(Sakura-Qwen3.8-Flash-Next-Swift-3bit-352E-GGUF): about 26 GiB of weights sit on the GPU, so it also fits

a 32 GB card. Made for a single 64 GB AMD Strix Halo (Radeon 8060S); German and English are the target languages. Independent community build,

not an official Qwen or UkisAI release.

Part of the Sakura Micro line.

All Sakura Flash-Next builds side by side (what differs, measured table, runtime notes, tools): Sakura-Qwen3.8-Flash-Next-Overview.

Provenance

• Base: Qwen/Qwen3.8-Flash-Next (arch qwen4exp, 48 layers, 512 experts, 10 active).
• Fine-tune + quantization: ukisai/Swift-1.5-Qwen3.8-Flash-Next-GSQ-RCO-GGUF (IQ2_XS tier).

Swift is the reason for this build: it uses clearly fewer thinking tokens than the base model.

• What this build changes: expert slicing only (see NOTICE.md). Kept experts per layer: kept_experts_K352.json.
• Expert selection: router statistics (routing mass) of the ISTA-quantized Qwen3.8-Flash-Next on a German + English calibration mix; an expert is kept if it is hot in

either language. The same measurement on the Swift model itself gives a keep set that overlaps this one by 98 %. Measured with llama-expert-stats,

a small tool that runs the model and counts router use (much faster than llama-imatrix).

Files

| File | Size |

|---|---:|

| Sakura-Qwen3.8-Flash-Next-Swift-K352-IQ2_XS-00001-of-00002.gguf | 34.31 GiB (dense weights, the large per-layer token table and part of the experts) |

| Sakura-Qwen3.8-Flash-Next-Swift-K352-IQ2_XS-00002-of-00002.gguf | 18.77 GiB (rest of the routed experts) |

SHA256SUMS lists the checksums. Load shard 1; llama.cpp finds shard 2 itself. Use -lm dio --lazy-mode on (or your build's equivalent) so the large token table is

read lazily instead of held in memory.

Measured results (own tests, this exact file)

| Check | Result |

|---|---|

| Language gate, 14 German + 14 English prompts: share of wrong-language words | DE 0.005 / EN 0.01 (gate: each <= 0.06) |

| Loops / CJK answers | 0 / 0 |

| Held-out perplexity (German mix / English mix) | 2.0681 / 1.8229 |

| Decode / prefill speed | 37.6 tok/s decode, 238 tok/s prefill with speculation (Vulkan, see the speed table) |

| Coding tasks (own 8 LeetCode/AtCoder-style tasks) | 7/8 passed (33,430 generated tokens, about 18.0 min) |

| Tasks 9-20 of our frozen 32-task set (harder tasks) | 10/12 passed (85,714 generated tokens), see below |

The language gate and the perplexity were measured with the official llama.cpp build b11259 (Vulkan); speed and coding tasks with the fork build described below.

Held-out perplexity: llama-perplexity, context 512, 40 chunks of own German and English held-out texts (not in the calibration data), raw {"de": 2.0681, "en": 1.8229}.

For orientation, same texts: Swift 512-expert IQ2_XS 2.0605 / 1.8245, the 3-bit sibling of this build 2.0514 / 1.8231. The differences between these builds are within about +-1 %;

perplexity cannot separate them.

Runtime used for speed and coding tests

The shared MTP head needs a llama.cpp fork. We used github.com/danielhanchen/llama.cpp, branch qwen4exp/mtp, commit 6fcaa16 (21 Sep 2026), built by us for Vulkan

(MSVC, Vulkan SDK 1.4.357: cmake -B build -G Ninja -DCMAKE_BUILD_TYPE=Release -DGGML_VULKAN=ON -DBUILD_SHARED_LIBS=ON -DLLAMA_OPENSSL=OFF -DLLAMA_CURL=OFF).

Speed (AMD Strix Halo, 64 GB, Radeon 8060S, Windows, Vulkan)

Same ~1700-token prompt each time, 32k context, KV q8_0, -lm dio --lazy-mode on, shared MTP sidecar mtp-Qwen3.8-Flash-Next-shared-Q4_K_M.gguf (1.78 GiB):

| -b/-ub | Speculation | Prefill | Decode |

|---|---|---:|---:|

| 2048 | MTP n-max 2 | 238.0 tok/s | 37.6 tok/s |

| 2048 | MTP n-max 3 | 253.3 tok/s | 35.2 tok/s |

Without speculation the same file decodes at about 24 tok/s in official llama.cpp (Vulkan, b11259 and b11297). On ROCm (community halo-box build, no speculation) we measured

322 tok/s prefill and 21.6 tok/s decode for this file, because its resident weights (about 26 GiB) fit the dedicated GPU memory; the 3-bit sibling does not and collapses there.

Coding check (own tasks, thinking budget 10000, max 12288 tokens, temperature 0.6, top-p 0.95, top-k 20, seeds 1000 + task number, one run each)

• Eight anchor tasks: 7/8 passed. Not passed: task(s) 8.
• Tasks 9 to 20 of the frozen 32-task set (the harder part; same questions, grader and seeds for every model we test): 10 of 12 passed. Not passed: task(s) 9, 15.

Single runs on one machine; they are indicators, not a benchmark suite. Vision was not tested (no mmproj is included; use the BF16 projector from the Swift repository).

Known limits

• Cutting experts without retraining costs some quality; the size of that cost was checked on German/English text and a small coding set only. At 2.5 bit the quantization itself

is coarser than in the 3-bit sibling; if you have the memory, prefer the 3-bit build.

• Speculation needs the fork above. The shared MTP sidecar is not part of this GGUF and does not load in official llama.cpp (b11259 and b11294 fail with token_embd.weight not found;

the upstream loader has no MTP-only path). Unsloth publishes the shared sidecar as MTP/mtp-Qwen3.8-Flash-Next-shared-Q8_0.gguf in unsloth/Qwen3.8-Flash-Next-GGUF;

we measured with a Q4_K_M sidecar and did not test the Q8_0 one. Without speculation the file works in official llama.cpp.

• An earlier Unsloth Vulkan package (build 10830, commit 0cd7a79f5) gave wrong results with the 3-bit sibling in our tests: after the thinking budget ran out, the model kept

reasoning in the answer part instead of writing the code. We did not test this file on that build and do not recommend it for this model family.

• Licenses: Swift Open License v1.0 (commercial-use threshold, section 5) and Qwen Community License 1.0 (Model-as-a-Service / AI Work Assistant clause). Read both before commercial use.

Run example

llama-server -m Sakura-Qwen3.8-Flash-Next-Swift-K352-IQ2_XS-00001-of-00002.gguf --jinja -fa on -ngl 99 -c 32768 -ctk q8_0 -ctv q8_0 -b 2048 -ub 2048 -lm dio --lazy-mode on --device Vulkan0 \
  --spec-type draft-mtp --spec-draft-model mtp-Qwen3.8-Flash-Next-shared-Q4_K_M.gguf --spec-draft-ngl 99 --spec-draft-n-max 2 --spec-draft-n-min 0

This is the command line used for the speed numbers with the fork build. Leave out the last line to run without speculation.

---

<details><summary>中文说明 · 樱花 (Simplified Chinese)</summary>

> English above. 本节为上文的中文翻译(Sakura = 樱花 yīnghuā);完整的独立中文版见 README_zh.md。

Sakura — Qwen3.8-Flash-Next (Swift 1.5),512 个专家中保留 352 个,IQ2_XS

<img src="assets/sakura-logo.svg" alt="Sakura logo" width="560" />

> ## 可放进 32 GB GPU

> 约 26 GiB 的权重放在 GPU 上;26.8 GiB 的逐层 token 表留在磁盘/内存中(懒加载,-lm dio --lazy-mode on)。

> 在 32k 上下文、q8_0 KV 缓存、MTP 头和 -ub 2048 的情况下,服务器在我们的测试中使用了 29.9 GiB 的专用 GPU 内存(另加 5.7 GiB 共享内存);不使用 MTP 头时大约少需要 1.8 GiB。

> 测量是在带有 32 GiB 专用划分的 Radeon 8060S(Strix Halo)上进行的;我们没有测试独立的 32 GB 显卡。

> 3 比特的同系列版本需要约 31.4 GiB 的 GPU 权重,不能 放进 32 GB 的显卡。

这是 UkisAI 对 Qwen3.8-Flash-Next 的 Swift 1.5 微调版本的一个实验性专家剪枝 GGUF:每层 512 个路由专家中保留 352 个

(依据德语 + 英语的路由情况选出),其余的从约 2.5 比特(IQ2_XS,GSQ-RCO)量化中被裁掉,没有重新量化。大小 53.08 GiB,

分为两个分片。这是 3 比特构建

(Sakura-Qwen3.8-Flash-Next-Swift-3bit-352E-GGUF)的较小同系列版本:约 26 GiB 的权重放在 GPU 上,因此它也能放进

32 GB 的显卡。为单台 64 GB 的 AMD Strix Halo(Radeon 8060S)制作;目标语言是德语和英语。这是独立的社区构建版本,

不是 Qwen 或 UkisAI 的官方发布。

属于 Sakura Micro 系列。

所有 Sakura Flash-Next 构建的并排比较(区别、实测表格、运行时说明、工具):Sakura-Qwen3.8-Flash-Next-Overview。

来源

• 基础模型:Qwen/Qwen3.8-Flash-Next(架构 qwen4exp,48 层,512 个专家,激活 10 个)。
• 微调 + 量化:ukisai/Swift-1.5-Qwen3.8-Flash-Next-GSQ-RCO-GGUF(IQ2_XS 档位)。

Swift 是制作这个构建的原因:它使用的思考 token 明显少于基础模型。

• 本构建的改动:仅为专家切片(见 NOTICE.md)。每层保留的专家:kept_experts_K352.json。
• 专家选择:经 ISTA 量化的 Qwen3.8-Flash-Next 在德语 + 英语校准混合文本上的路由器统计(路由质量);只要一个专家在任一语言中是热点,就予以保留。在 Swift 模型自身上做相同的测量,得到的保留集与这一个有 98 % 的重叠。使用 llama-expert-stats 测量,

这是一个运行模型并统计路由器使用情况的小工具(比 llama-imatrix 快得多)。

文件

| 文件 | 大小 |

|---|---:|

| Sakura-Qwen3.8-Flash-Next-Swift-K352-IQ2_XS-00001-of-00002.gguf | 34.31 GiB(稠密权重、巨大的逐层 token 表和一部分专家) |

| Sakura-Qwen3.8-Flash-Next-Swift-K352-IQ2_XS-00002-of-00002.gguf | 18.77 GiB(其余的路由专家) |

SHA256SUMS 列出了校验和。加载分片 1;llama.cpp 会自己找到分片 2。请使用 -lm dio --lazy-mode on(或你所用构建中的等效选项),使巨大的 token 表被懒加载读取,而不是常驻内存。

测量结果(自己的测试,这一确切的文件)

| 检查项 | 结果 |

|---|---|

| 语言关卡,14 个德语 + 14 个英语提示:使用了错误语言的词所占比例 | DE 0.005 / EN 0.01(关卡:各 <= 0.06) |

| 循环 / CJK 回答 | 0 / 0 |

| 留出困惑度(德语混合 / 英语混合) | 2.0681 / 1.8229 |

| 解码 / 预填充速度 | 使用推测解码时 37.6 tok/s 解码,238 tok/s 预填充(Vulkan,见速度表) |

| 编程任务(自己的 8 个 LeetCode/AtCoder 风格任务) | 7/8 通过(生成 33,430 个 token,约 18.0 分钟) |

| 我们冻结的 32 个任务集中的任务 9-20(更难的任务) | 10/12 通过(生成 85,714 个 token),见下文 |

语言关卡和困惑度是用官方 llama.cpp 构建 b11259(Vulkan)测得的;速度和编程任务使用下文所述的分支构建。

留出困惑度:llama-perplexity,上下文 512,在自己的德语和英语留出文本(不在校准数据中)上取 40 个块,原始值 {"de": 2.0681, "en": 1.8229}。

作为参考,在相同文本上:Swift 512 专家 IQ2_XS 为 2.0605 / 1.8245,该构建的 3 比特同系列版本为 2.0514 / 1.8231。这些构建之间的差异在约 +-1 % 以内;

困惑度无法将它们区分开。

用于速度和编程测试的运行时

共享 MTP 头需要 llama.cpp 分支。我们使用了 github.com/danielhanchen/llama.cpp,分支 qwen4exp/mtp,提交 6fcaa16(2026 年 9 月 21 日),由我们自己为 Vulkan 构建

(MSVC,Vulkan SDK 1.4.357:cmake -B build -G Ninja -DCMAKE_BUILD_TYPE=Release -DGGML_VULKAN=ON -DBUILD_SHARED_LIBS=ON -DLLAMA_OPENSSL=OFF -DLLAMA_CURL=OFF)。

速度(AMD Strix Halo,64 GB,Radeon 8060S,Windows,Vulkan)

每次使用相同的约 1700 个 token 的提示,32k 上下文,KV q8_0,-lm dio --lazy-mode on,共享 MTP 旁路文件 mtp-Qwen3.8-Flash-Next-shared-Q4_K_M.gguf(1.78 GiB):

| -b/-ub | 推测解码 | 预填充 | 解码 |

|---|---|---:|---:|

| 2048 | MTP n-max 2 | 238.0 tok/s | 37.6 tok/s |

| 2048 | MTP n-max 3 | 253.3 tok/s | 35.2 tok/s |

不使用推测解码时,同一个文件在官方 llama.cpp(Vulkan,b11259 和 b11297)中的解码速度约为 24 tok/s。在 ROCm 上(社区的 halo-box 构建,无推测解码),我们为该文件测得

322 tok/s 预填充和 21.6 tok/s 解码,因为它的常驻权重(约 26 GiB)可放进专用 GPU 内存;3 比特的同系列版本放不进去,在那里速度会骤降。

编程检查(自己的任务,思考预算 10000,最多 12288 个 token,temperature 0.6,top-p 0.95,top-k 20,随机种子 1000 + 任务编号,各运行一次)

• 八个锚点任务:7/8 通过。未通过:任务 8。
• 冻结的 32 个任务集中的任务 9 至 20(更难的部分;我们测试的每个模型都使用相同的问题、评分器和随机种子):12 个中通过 10 个。未通过:任务 9、15。

单次运行,在一台机器上;它们是指示性的,不是基准测试套件。视觉能力未经测试(未包含 mmproj;请使用 Swift 仓库中的 BF16 投影器)。

已知局限

• 不经再训练就裁剪专家会损失一些质量;这一损失的大小只在德语/英语文本和一个小型编程集上检查过。在 2.5 比特下,量化本身比 3 比特的同系列版本更粗糙;如果你的内存足够,请优先选择 3 比特构建。
• 推测解码需要上述分支。 共享 MTP 旁路文件不属于这个 GGUF,在官方 llama.cpp 中无法加载(b11259 和 b11294 会以 token_embd.weight not found 失败;上游加载器没有仅含 MTP 的路径)。Unsloth 在 unsloth/Qwen3.8-Flash-Next-GGUF 中把共享旁路文件发布为 MTP/mtp-Qwen3.8-Flash-Next-shared-Q8_0.gguf;我们使用 Q4_K_M 旁路文件测量,没有测试 Q8_0 的那个。不使用推测解码时,该文件可在官方 llama.cpp 中运行。
• 较早的 Unsloth Vulkan 包(构建 10830,提交 0cd7a79f5)在我们的测试中对 3 比特同系列版本给出了错误的结果: 在思考预算用完之后,模型仍在回答部分继续推理,而没有写出代码。我们没有在该构建上测试这个文件,也不推荐把它用于这一模型系列。
• 许可证:Swift Open License v1.0(商业使用门槛,第 5 节) 以及 Qwen Community License 1.0(Model-as-a-Service / AI Work Assistant 条款)。商业使用之前请阅读两者。

运行示例

llama-server -m Sakura-Qwen3.8-Flash-Next-Swift-K352-IQ2_XS-00001-of-00002.gguf --jinja -fa on -ngl 99 -c 32768 -ctk q8_0 -ctv q8_0 -b 2048 -ub 2048 -lm dio --lazy-mode on --device Vulkan0 \
  --spec-type draft-mtp --spec-draft-model mtp-Qwen3.8-Flash-Next-shared-Q4_K_M.gguf --spec-draft-ngl 99 --spec-draft-n-max 2 --spec-draft-n-min 0

这是使用分支构建得出速度数据时所用的命令行。省略最后一行即可在不使用推测解码的情况下运行。

</details>