---

license: other

license_name: qwen-community-1.0

license_link: https://huggingface.co/Qwen/Qwen3.8-Flash-Next/blob/f5d08274bafd880402bd16f5e3e6c514136ec06c/LICENSE

base_model: Qwen/Qwen3.8-Flash-Next

base_model_relation: quantized

pipeline_tag: image-text-to-text

tags:

- gguf

- mixed-quant

- qwen4exp

- qwen3.8-flash-next

- dgx-spark

- ds4

- ssd-offload

---

Qwen3.8-Flash-Next Mixed-Quant SSD-PLE GGUF

> Two accelerator-resident precision variants are published with one shared BF16 SSD-PLE sidecar. MQ-Q6-SSD-PLE-BF16 is the original Q5/Q6 release. MQ-Q5-SSD-PLE-BF16 lowers only its Q5_K and Q6_K targets by one tier for 13.4082 GiB more resident-memory headroom, while retaining the optimized Q5_0 down tail. Both variants have completed the separate single-DGX-Spark serving checks described below. The Q5 check used a server configured for 262,144 tokens and prompts up to 21,037 tokens; it was not a full-length 262K prompt run.

This is a memory-hierarchy-aware mixed-precision conversion of Qwen/Qwen3.8-Flash-Next, pinned to revision f5d08274bafd880402bd16f5e3e6c514136ec06c.

The design moves the 51.2B-parameter predictive latent embedding (PLE) table out of the accelerator-resident GGUF weight set and stores it as BF16 SSD-backed sparse lookup memory. Nearly all of the released residency budget is then reassigned to the 128.8B-parameter compute backbone. Compatibility with upstream llama.cpp, vLLM, SGLang, or other GGUF runtimes is not implied; a dedicated ds4 SSD-PLE loader is required.

Variants and selective download

| Variant | Routed expert tiers | Main payload | Shared PLE | Runtime status |

|---|---|---:|---:|---|

| MQ-Q6-SSD-PLE-BF16 | Q5_K / Q6_K / Q5_0 | 90.9538 GiB | 95.3682 GiB | DGX Spark smoke passed |

| MQ-Q5-SSD-PLE-BF16 | Q4_K / Q5_K / Q5_0 | 77.5456 GiB | same files | DGX Spark 262K-config profile passed |

Download only the original Q6 variant, including its canonical PLE directory:

qwen_repo=Baekpica/Qwen3.8-Flash-Next-Mixed-Quant-SSD-PLE-GGUF
qwen_root=./Qwen3.8-Flash-Next-Mixed-Quant-SSD-PLE-GGUF
hf download "$qwen_repo" \
  --include 'MQ-Q6-SSD-PLE-BF16/*' \
  --local-dir "$qwen_root"
(cd "$qwen_root/MQ-Q6-SSD-PLE-BF16" && sha256sum -c SHA256SUMS)

Download only the lower-residency Q5 main GGUF plus the same canonical PLE files. The local link preserves the ple/ple-manifest.json path embedded in the GGUF without storing a second 95.37 GiB copy:

qwen_repo=Baekpica/Qwen3.8-Flash-Next-Mixed-Quant-SSD-PLE-GGUF
qwen_root=./Qwen3.8-Flash-Next-Mixed-Quant-SSD-PLE-GGUF
qwen_variant=MQ-Q5-SSD-PLE-BF16

hf download "$qwen_repo" \
  --include "$qwen_variant/*" \
  --local-dir "$qwen_root"
hf download "$qwen_repo" \
  --include 'MQ-Q6-SSD-PLE-BF16/ple/*' \
  --local-dir "$qwen_root"
test -e "$qwen_root/$qwen_variant/ple" || \
  ln -s ../MQ-Q6-SSD-PLE-BF16/ple "$qwen_root/$qwen_variant/ple"
(cd "$qwen_root/$qwen_variant" && sha256sum -c SHA256SUMS)

If the Q6 PLE directory is already present, only the first hf download and the link are needed; the additional transfer is about 77.56 GiB rather than downloading the PLE again.

The three Q5 main weight files occupy exactly 83,274,984,384 bytes / 77.5559 GiB. A fresh local setup with the four shared PLE files occupies 185,675,770,816 weight-file bytes / 172.9240 GiB; the PLE is not duplicated in this repository.

Quantization targets by variant

| Model region | Q6 variant | Q5 variant | Residency policy |

|---|---:|---:|---|

| 51.2B-parameter PLE n-gram table (128 shards) | BF16 SSD sidecar | same files | non-resident; bounded pinned page cache only |

| Routed expert gate/up, interior layers 2–45 | Q5_K | Q4_K | accelerator-resident |

| Routed expert gate/up, edge layers 0, 1, 46, 47 | Q6_K | Q5_K | accelerator-resident |

| Routed expert down, main 512 columns | Q6_K | Q5_K | accelerator-resident |

| Routed expert down, 128-column tail | Q5_0 | unchanged Q5_0 | accelerator-resident |

| MTP routed experts and most always-active matrices | primarily Q8_0 | unchanged | accelerator-resident |

| Hyper-connection, non-quantizable convolution/vision tensors | BF16 | unchanged | accelerator-resident |

| Norms, gates, recurrent/control state | F32 where required | unchanged | accelerator-resident |

| Integer PLE controls | I64 | unchanged | accelerator-resident |

The original 128-column tail target was Q5_K, but standard GGUF Q5_K uses 256-value blocks and therefore cannot encode a tensor whose row width (ne[0]) is 128. The Q6 artifact realizes that tier as width-compatible Q5_0, and the Q5 variant deliberately retains all 48 Q5_0 tail tensors so ds4 can continue using its current optimized tail accumulator and dispatch. Only the K-quant tiers are lowered: Q5_K -> Q4_K and Q6_K -> Q5_K. No Q5_0, Q8_0, BF16, F32, or I64 target changes between variants, and both are generated directly from the pinned source BF16 checkpoint rather than by requantizing the Q6 GGUF.

Baekpica/ds4@0951d03 recognizes the MQ-Q5-SSD-PLE-BF16 label and validates its exact per-layer precision map. The existing Q4_K paired gate/up, routed Q5_K MMQ, and Q5_0 tail paths are reused.

Separate memory metrics

Overall model BPW is not the useful metric for this artifact because the shared PLE is intentionally non-resident. The audited resident-backbone census is:

| Metric | Q6 variant | Q5 variant |

|---|---:|---:|

| Main GGUF logical parameters | 128,799,735,699 | 128,799,735,699 |

| Main GGUF tensor count | 1,628 | 1,628 |

| Resident backbone payload | 97,660,877,400 bytes / 90.9538 GiB | 83,263,928,920 bytes / 77.5456 GiB |

| Resident backbone effective BPW | 6.0659 | 5.1717 |

| Actual main GGUF files | 97,671,932,960 bytes / 90.9641 GiB | 83,274,984,384 bytes / 77.5559 GiB |

| Resident savings versus Q6 | — | 14,396,948,480 bytes / 13.4082 GiB |

| Shared PLE parameters | 51,200,245,760 | same |

| Shared PLE storage dtype | BF16 (16 bpw) | same files |

| PLE payload on SSD | 102,400,491,520 bytes / 95.3679 GiB | same files |

| PLE files including 4 KiB alignment | 102,400,786,432 bytes / 95.3682 GiB | not duplicated |

The published resident-PLE baseline spends 38,400,184,320 bytes / 35.7630 GiB on a Q5_1 PLE table. The original Q6 compute backbone costs 37,570,478,080 bytes / 34.9902 GiB more, so its resident payload is 829,706,240 bytes / 0.7727 GiB smaller than that baseline. The lower-tier Q5 resident payload is 15,226,654,720 bytes / 14.1809 GiB smaller than the same baseline. These are exact tensor payload calculations using GGUF block sizes; final file size, CUDA allocations, page cache, KV state, and runtime workspace are separate.

Published artifact layout

MQ-Q6-SSD-PLE-BF16/
├── Qwen3.8-Flash-Next-MQ-Q6-SSD-PLE-BF16-*.gguf
├── SHA256SUMS
├── quant-recipe.yaml
├── artifact-manifest.json
└── ple/
    ├── ple-bf16-00001-of-00004.bin
    ├── ple-bf16-00002-of-00004.bin
    ├── ple-bf16-00003-of-00004.bin
    ├── ple-bf16-00004-of-00004.bin
    └── ple-manifest.json

MQ-Q5-SSD-PLE-BF16/
├── Qwen3.8-Flash-Next-MQ-Q5-SSD-PLE-BF16-*.gguf
├── SHA256SUMS
├── SHA256SUMS.main
├── shared-ple-reference.json
├── quant-recipe.yaml
└── artifact-manifest.json

The Q5 directory intentionally does not duplicate the PLE files. After selective download, its local ple symlink points at the Q6 directory's canonical sidecar. The 128 logical checkpoint shards remain individually addressable through ple-manifest.json; four larger physical files reduce descriptor and Hub-transfer overhead. Every logical-shard start is 4 KiB aligned. The manifest records the source tensor, global row range, physical file and offset, row stride, per-head prime vocabulary sizes and offsets, SplitMix64-derived layer multipliers, total bytes, and checksums.

SSD runtime contract

The full 95.37 GiB PLE sidecar must never become resident in the 128 GB unified-memory pool. The planned ds4 path uses exact CPU-side Qwen n-gram row derivation, asynchronous SSD prefetch overlapped with decoder layer 0, a bounded 512 MiB / 1 GiB / 2 GiB pinned mapped page cache, and a separate CUDA UVA gather kernel. O_DIRECT plus aligned asynchronous reads is preferred; a buffered fallback must evict sidecar pages with an explicit POSIX_FADV_DONTNEED-class policy.

Registering the whole file with CUDA, relying on unbounded mmap, or allowing Linux page cache to retain the full sidecar fails the residency objective. A successful load of the main GGUF alone is not proof that SSD-PLE works.

The current ds4 handoff implementation covers the correctness-critical core of

this contract: exact EOS-bounded CPU hashing, strict manifest validation,

bounded four-way page caching, asynchronous worker prefetch, O_DIRECT with a

page-cache-dropping fallback, mapped-cache UVA gathering, and cache-slot leases

held until the consuming CUDA stream has completed. Decoder-layer-0 overlap

and detailed SSD latency/counter profiling remain target-hardware integration

work. A preliminary cache-size A/B is reported below.

Q6 variant DGX Spark ds4 serving verification

The results in this section apply to MQ-Q6-SSD-PLE-BF16, not yet to the new Q5 variant. On 2026-08-27, Baekpica/ds4@a06f06d

on branch dfm was verified on one

DGX Spark (NVIDIA GB10) with the server configured for a 262,144-token context.

• The worker imported a content-verified 91.05 GiB shared VMM weight cache and

reserved a logical 14.33 GiB 262K session graph. QSA index/KV state was

demand-mapped: the session census was 0.45 GiB before a request and 0.55 GiB

after a 289-token cold prompt, with 91.65 GiB total device-live memory. These

figures are for that short prompt; QSA pages grow as more context is used.

• The four PLE sidecars opened with O_DIRECT (4/4) behind a bounded 512 MiB

cache. A bounded mincore check after inference found 0.00 MiB of the

95.37 GiB sidecar set resident in Linux page cache; the sidecars were not

present in the server's memory map.

• Chat Completions and Responses returned coherent deterministic output.

Buffered and streaming Responses, a function call, and its

function_call_output continuation completed successfully.

• Two simultaneous requests completed through the current serial lane with no

request, graph-fit, memory-census, or memory-governor failures.

• Across three fresh eager-graph server processes, the same deterministic

289-token synthetic cold prompt averaged 131.6 prefill tok/s, 2,254.9 ms

TTFT, and 17.7 decode tok/s. Each run produced the same 74 output tokens.

This is a short microbenchmark, not a sustained-throughput result.

• A 3,069-token marker-retrieval prompt returned the exact marker at 83.0

prefill tok/s. This is the longest prompt used in this smoke test.

The host run used a 115 GiB hard memory cap and a 6 GiB admission floor; no

cgroup high/max/OOM event occurred. These observations establish functional

serving with the runtime configured for 262,144 tokens. They do not establish

a completed 262,144-token prompt run, full-model quality, sustained throughput,

or SSD latency percentiles.

Q5 variant DGX Spark ds4 serving and prefill profile

On 2026-08-27, Baekpica/ds4@0951d03

on branch dfm was verified on one

DGX Spark (NVIDIA GB10) with MQ-Q5-SSD-PLE-BF16 and the server configured for

a 262,144-token context. The measured configuration used

DS4_QWEN_PREFILL_CHUNK=8192, DS4_QWEN_PLE_CACHE_MB=512, O_DIRECT on 4/4

sidecars, a 115 GiB hard memory cap, and a 2 GiB admission floor. Prefix-cache

reuse was zero for every reported request.

| Workload | Measured requests | Mean prefill | Mean TTFT | Mean decode |

|---|---:|---:|---:|---:|

| 5,384-5,385 prompt tokens + 64 output tokens | 3 | 154.6 tok/s | 34.91 s | 18.7 tok/s |

| 21,037 prompt tokens + 64 output tokens | 2 | 149.1 tok/s | 141.28 s | 18.8 tok/s |

For the same 24,000-byte story workload family, three 256-token-chunk requests

(5,392-5,393 prompt tokens) averaged 139.7 prefill tok/s, 38.67 s TTFT, and

18.8 decode tok/s. The 8,192-token configuration improved mean prefill by

10.7% and reduced mean TTFT by 9.7%; decode differed by -0.5%. The request

prefix labels and resulting token counts differed slightly, so this is a live

serving A/B rather than a bit-exact kernel microbenchmark.

The 262K session graph reported a 24.85 GiB logical plan and 11,238.2 MiB of

measured committed memory. The 512 MiB-cache run peaked at 60,942,938,112 bytes

(56.76 GiB) in its guarded cgroup, with zero high/max/OOM events. A separate

21,037-token two-request check with a 2 GiB PLE cache averaged 150.0 prefill

tok/s, 140.43 s TTFT, and 18.7 decode tok/s: about +0.6%, -0.6%, and -0.5%

relative to 512 MiB. The 512 MiB default was retained because the larger cache

did not produce a material gain in this workload.

/v1/models, /v1/stats, Chat Completions, and text Completions all returned

successfully, and the generated text was coherent. This establishes functional

Q5 serving and preliminary prefill/TTFT/decode measurements with a 262K-configured

server. It does not establish a completed 262K prompt, full-model quality,

sustained multi-request throughput, or SSD latency percentiles.

Optimized prefill rerun

Later on 2026-08-27, Baekpica/ds4@64f7e41

was tested on the same DGX Spark and Q5 artifact after Q5_0 down-tail tiling,

Gated DeltaNet state-column parallelization, and a compact bounded Q5_K routed

MMQ worklist were added. A fixed 8,192-token ds4-bench workload with no

decode measured 275.81, 293.20, and 293.55 prefill tok/s: 287.52 tok/s mean

(275.81-293.55 range), versus 145.02 tok/s before those changes using the same

command and input (+98.3%).

A subsequent QSA reduction change in

Baekpica/ds4@4b5eb39

reused each selected value across the 12 query heads sharing a KV head. The

real-weight QSA check retained 1.2470e-07 relative RMS error and

1.4544e-14 one-minus-cosine distance. The identical fixed 8,192-token direct

benchmark then measured 293.58, 321.00, and 285.32 prefill tok/s:

299.97 tok/s mean (285.32-321.00 range), +4.3% over the preceding

287.52 tok/s result and +106.8% over the initial 145.02 tok/s baseline.

This direct run used --gen-tokens=0, so API TTFT and decode throughput are

not applicable; the latest end-to-end values remain the 64f7e41 results

reported below.

The next PLE change in

Baekpica/ds4@11e1d219

kept the shared 2,560-row key/value prefix on the paired Q8 kernel and sent the

remaining 7,680 key-only rows through the existing single-output kernel. On

the same fixed 8,192-token direct workload, a same-binary 3-by-3 A/B measured

302.46, 332.15, and 312.64 tok/s with the split enabled versus 287.06, 297.64,

and 286.06 tok/s with DS4_CUDA_NO_Q8_PAIR_ASYM_SPLIT=1: **315.75 versus

290.25 tok/s mean (+8.78%)**. The optimized mean is 5.3% above the previously

published 299.97 tok/s run, but the same-binary comparison is the controlled

result. A full-model 256-token on/off check matched all 248,320 logits exactly,

including the argmax and top-10 set.

The 64f7e41 build was also exercised through Chat Completions on a server

configured for a 262,144-token context, 8,192-token prefill chunks, two

independent Qwen banks, a 512 MiB PLE cache, and O_DIRECT on 4/4 sidecars:

| Prompt / output tokens | Prefill | TTFT | Decode |

|---:|---:|---:|---:|

| 8,774 / 64 | 317.8 tok/s | 27.637 s | 22.8 tok/s |

| 12,256 / 64 | 317.5 tok/s | 38.617 s | 22.7 tok/s |

| 8,095 / 64 | 296.0 tok/s | 27.360 s | 22.8 tok/s |

| Mean (9,708 prompt tokens) | 310.43 tok/s | 31.204 s | 22.77 tok/s |

All three requests were cold (cached_tokens=0); token-weighted prefill was

311.30 tok/s. /v1/models advertised 262,144 tokens, all six requests in the

complete smoke sequence succeeded, and the server reported zero continuous

fallbacks and zero governor faults. Two simultaneous 3.8K prompts also

completed as served=2 fallback=0, establishing two-bank function but not

row-batched kernel throughput; their per-request decode rates were uneven and

are not included in the single-request mean above.

This 8,192-by-two-bank run required DS4_MEMGOV=observe because the internal

governor rejected cold materialization under enforcement despite the external

memory guard. The 109 GiB reclaim threshold and 115 GiB hard cgroup limit

remained active. Device-live memory stabilized at 97.88 GiB and observed system

available memory stayed at or above 15.46 GiB during the reported requests.

This remains a 262K-configured serving check, not a full-length 262K prompt run.

The 11e1d219 build was then checked through the same API shape with three

distinct cold prompts (cached_tokens=0):

| Prompt / output tokens | Prefill | TTFT | Decode |

|---:|---:|---:|---:|

| 7,121 / 64 | 360.3 tok/s | 19.799 s | 18.1 tok/s |

| 9,769 / 58 | 232.5 tok/s | 42.057 s | 18.0 tok/s |

| 5,660 / 64 | 258.3 tok/s | 21.954 s | 18.1 tok/s |

| Mean (7,517 prompt tokens) | 283.70 tok/s | 27.937 s | 18.07 tok/s |

The server advertised a 262,144-token context and reported two banks. A

barrier-synchronized two-request smoke also completed, with 5/5 total requests

completed and zero failures. The 9,769-token request crossed the 8,192-token

chunk boundary and included second-chunk/bank warm-up, so these content- and

chunk-sensitive API values are reported as a serving check rather than a

controlled comparison with the earlier table. The external 115 GiB guard

remained active throughout.

On 2026-08-28, the QSA scoring path in

Baekpica/ds4@f9f14a7

replaced the fixed 24-query/2-KV shared-memory block reduction with a four-warp

register/shuffle reduction. A controlled cold fixed-8K 3-by-3 A/B measured

306.45, 309.58, and 334.18 tok/s before versus 369.77, 347.64, and 362.98 tok/s

after: 316.74 versus 360.13 tok/s mean (+13.70%). The real-weight arbitrary-

chunk and single-token checks retained 1.2470e-07 relative RMS error and

1.4544e-14 one-minus-cosine distance.

The following Q5_0 expert-major tail change in

Baekpica/ds4@c733f5e

staged the existing 32 warp sums and let the first warp write contiguous output

columns. Its controlled cold fixed-8K 3-by-3 A/B measured 364.05, 359.54, and

370.08 tok/s before versus 384.28, 389.49, and 368.47 tok/s after:

364.56 versus 380.75 tok/s mean (+4.44%). The full MMQ parity suite and the

real-weight MoE forward gate passed after this change.

The c733f5e build was then exercised through Chat Completions with three

distinct cold prompts (cached_tokens=0):

| Prompt / output tokens | Prefill | TTFT | Decode |

|---:|---:|---:|---:|

| 6,384 / 64 | 466.5 tok/s | 13.731 s | 17.9 tok/s |

| 9,807 / 64 | 474.6 tok/s | 20.687 s | 17.7 tok/s |

| 8,151 / 64 | 472.3 tok/s | 17.278 s | 17.2 tok/s |

| Mean (8,114 prompt tokens) | 471.13 tok/s | 17.232 s | 17.60 tok/s |

Token-weighted prefill was 471.68 tok/s. This serving check used a 262,144-token

context, 8,192-token prefill chunks, two Qwen banks, the 2 GiB PLE cache and 16

PLE workers, O_DIRECT on 4/4 sidecars, DS4_MEMGOV=observe, and the external

109/115 GiB reclaim/hard guard. /v1/models advertised 262,144 tokens; all six

requests in the complete smoke sequence finished with zero failures, and a

barrier-synchronized pair completed as served=2 fallback=0. Final stats

reported 97.72 GiB device-live memory, 14.32 GiB system-available memory, and

zero governor/census faults. These content-sensitive API values are a serving

check, not a controlled comparison with earlier tables, a full-length 262K

prompt run, sustained-concurrency throughput, quality, or SSD-latency validation.

Prefix reuse, recurrent disk KV, and embedded MTP

On 2026-08-28, Baekpica/ds4@c8bc5c9

completed three Qwen runtime items that were previously listed as pending:

• Qwen recurrent session state is now serialized with the normal disk-KV

payload. The payload includes PLE convolution state, all 36 Gated DeltaNet

convolution/recurrent states, QSA KV/frontier state, and the live token

checkpoint. Tagged-layout, truncation, and corruption checks fail closed.

A real-Q5 cross-process API check restored all 1,152 cached prompt tokens

after a server restart, computed zero prompt tokens, and returned the same

deterministic continuation. Idle-bank stage/restore also passed.

• The opt-in two-bank runtime now retains a bounded 32-slot recurrent-state

checkpoint pool and can fork either an exact frontier or the longest safe

partial prefix. A divergent API branch reused 3,873 prompt tokens and

computed only its 21-token suffix; the real-weight test also matched a cold

oracle after a partial fork.

• The embedded Qwen MTP block now proposes one token and the target graph

verifies [normal, draft] before accepting it. Rejected drafts roll the

recurrent target state back, and a timing-based auto-quench returns to plain

decode if speculation is slower. The real-Q5 gate produced the identical

target token stream with multi-token accepts. This initial path was enabled

with --mtp-draft 2 for greedy scalar/session decode.

The combined guarded API check used a 196,608-token configured context,

DS4_QWEN_BATCH=1, two banks, 8,192-token prefill chunks, a bounded 512 MiB

PLE cache, and O_DIRECT on all four sidecars. Seven of seven requests

completed with zero failures; exact and partial forks were both observed, and

a synchronized three-request epoch reported served=3 fallback=0. Device-live

memory moved from 98.08 GiB after boot to 99.00 GiB during the run, observed

system-available memory remained at least 14.24 GiB, and the external 109/115

GiB reclaim/hard guard stayed active with zero governor or census faults. This

is a 196K-configured serving and feature check, not a full-length 196K or 262K

prompt run, sustained-concurrency throughput result, or proof of row-batched

Qwen kernels.

Two-bank embedded-MTP follow-up

Later on 2026-08-28,

Baekpica/ds4@4226b60

extended the same target-verified embedded MTP path to the opt-in continuous

two-bank loop. Each bank keeps independent draft state, verifies

[current, draft] in one two-row target call, rolls recurrent target state

back on rejection, and automatically quenches speculation when its measured

cost exceeds plain decode. Bank reset/copy, partial-prefix restore, disk-KV

restore, and graph retire/rebuild invalidate or rebuild the draft state.

The real-Q5 two-bank regression matched the existing serial target-verified

MTP output for both prompts. Both banks drafted (6/6 cycles), accepted 5/4

drafts, and then passed the existing disk-KV, partial-prefix, and graph

lifecycle gates.

A same-binary API A/B used four distinct cold 58-token prompts with a

128-token output budget:

| Decode mode | Mean prefill | Mean TTFT | Mean decode | Draft acceptance |

|---|---:|---:|---:|---:|

| Plain | 225.38 tok/s | 260.7 ms | 23.65 tok/s | n/a |

| Embedded MTP | 223.35 tok/s | 261.1 ms | 28.65 tok/s | 84.98% |

This is a +21.1% mean decode-throughput result with essentially unchanged

TTFT and prefill in that short workload. Device-live memory was 100.02 GiB

after the plain run and 101.00 GiB after the MTP run. One low-acceptance prompt

was slower than plain decode, so the gain is content-dependent and the

auto-quench remains part of the serving path. A synchronized two-request run

also completed, followed by Chat Completions, Responses streaming, and

Anthropic Messages checks. The final server counters were 10/10 requests

completed, 419 drafts, 362 accepts (86.40%), zero quench events, and zero

governor faults under the external 109/115 GiB reclaim/hard guard. This is a

two-bank functional and short-workload A/B, not a sustained-concurrency result

or a claim of true row-batched Qwen kernels.

262K API-surface and serial-continuation follow-up

On 2026-08-28, Baekpica/ds4@63d4ce7 was checked with the Q5 artifact at a

262,144-token configured context, two Qwen banks, 8,192-token prefill chunks,

a 2 GiB PLE cache with 16 workers, partial-prefix reuse, and a 32 GiB disk-KV

budget. Chat Completions and Anthropic Messages ran simultaneously and returned

their native HTTP 200 response shapes. A high-reasoning Responses function

call and its function_call_output continuation also both returned HTTP 200;

the second turn reused 377 cached tokens and computed its 23-token suffix.

Final counters reported two serial Responses requests, zero serial refusals,

zero failed requests, and zero governor or census faults.

The serial lane is required for this agent shape. Some live Responses

reasoning/tool frontiers deliberately route there; a cont-only launch with

--no-serial returns a typed 503 rather than silently recomputing or using the

wrong state. With two banks, boot device-live memory was 96.49 GiB with

15.46 GiB system-available. After the serial tool continuation allocated its

right-sized session graph, device-live memory was 106.22 GiB and

system-available memory was 5.55 GiB under the external 109/115 GiB guard.

A three-bank 262K trial was not retained on the 128 GiB reference host. It

booted at 107.47 GiB device-live with 4.25 GiB system-available, but a small

serial tool continuation reduced system-available memory to 1.12 GiB, below

the configured 2 GiB floor. This establishes two banks, not three, as the

guarded 262K agent-serving configuration on this machine. These were API and

memory checks, not full-length 262K prompt or sustained-throughput results.

Resident BF16 versus SSD-offload validation

The handoff gate compares the production offload path against the 128 resident

BF16 PLE tensors in the public BF16 GGUF. It does not use the SSD cache as its

own reference.

• Three tokens, including an EOS boundary, produced 48 row lookups and 7,680

BF16 values.

• A deliberately tiny 16 KiB/four-page cache was churned with 257 unrelated

reads, producing 366 evictions; gathering the original rows then required 48

new physical reads.

• Both the first and post-eviction SSD gathers matched the resident BF16 rows

byte-for-byte.

• Key/value projections, transformed gate, gated value, convolution input,

BF16 dilated convolution, final 122,880-byte PLE injection, and 368,640-byte

persistent convolution state were all byte-exact between the two paths.

• O_DIRECT was active on 4/4 sidecars. Repeated ordinary and

CUDA_LAUNCH_BLOCKING=1 runs stayed around 402 MiB peak process RSS; the

95.3682 GiB sidecar was never mapped or registered in full.

Broader rerun gates also passed: 595,616 row IDs against the pinned

Transformers implementation, 1,023 BF16 rows across 33 original safetensor

shards, an eight-reader cache stress, and two concurrent CUDA gathers of 4,112

rows each (including 253 page-crossing rows). These are correctness and bounded

residency observations on the H200 development host, not DGX Spark throughput

measurements.

The corresponding implementation and tests are checkpointed in

Baekpica/ds4@ebc3b72

on branch

feature/qwen38-ssd-ple-handoff.

Current verification status

• Passed: exact source topology and revision pin.
• Passed: Q4_K/Q5_K/Q6_K block-writer round-trip, finite-output, RMSE, and thread-determinism unit gates, plus retained-Q5_0 tail payload regression.
• Passed: both main-GGUF template audits: 1,628 tensors, expected type/shape/byte census, zero verifier errors.
• Passed: bit-exact BF16 sidecar extraction, 128 logical-part hashes, reread whole-file checksums, and four-of-four remote LFS SHA-256/byte-count matches.
• Passed for the Q6 variant: complete four-shard mixed-Q5/Q6 main-GGUF conversion, 1,628-tensor structural audit with zero errors, independent full-file SHA-256 checks, and four-of-four remote LFS SHA-256/byte-count matches.
• Passed for the Q5 variant: complete three-shard conversion, exact Q4_K=88, Q5_K=56, Q5_0=48 census with no Q4_0, zero structural errors, independent full-file SHA-256 matches on three of three shards, and three-of-three remote LFS SHA-256/byte-count matches.
• Passed: exact PLE row-ID derivation, source-safetensor row checks, bounded-cache concurrency, CUDA gather, forced-eviction reload, resident-BF16 A/B, complete PLE forward, and persistent-state comparisons.
• Passed in isolated production-shape/real-weight paths: PLE chunk/decode state, all 36 GDN blocks' required primitives, all 12 QSA blocks' required primitives, four-stream hyper-connections, and Q5/Q6/Q5_0 text-MoE kernels.
• Passed on DGX Spark for the Q6 variant: integrated 48-layer prefill/decode, native tokenizer/final logits, Chat and Responses serving, streaming, and tool-output continuation with a server configured for a 262,144-token context.
• Passed on DGX Spark for the Q6 variant: bounded-residency smoke test with a 91.65 GiB device-live census after the short demand-mapped QSA prompt, 512 MiB PLE cache, O_DIRECT on 4/4 sidecars, and 0.00 MiB sidecar page-cache residency after inference.
• Passed on DGX Spark for the Q5 variant: integrated 48-layer Chat/Completions prefill and decode with a 262,144-token configured context, including three 5.38K-token and two 21,037-token measured prompts under a 115 GiB hard cap.
• Passed on DGX Spark for the Q5 variant: native Chat Completions, Responses function-call/output continuation, and Anthropic Messages responses with a 262,144-token configured context; live reasoning/tool continuations require the serial fallback, so agent deployments must not use --no-serial.
• Passed on DGX Spark for the Q5 variant: the latest accepted fixed-8K controlled A/Bs measured 316.74 to 360.13 tok/s for the QSA reduction (+13.70%) and 364.56 to 380.75 tok/s for the Q5_0 tail store (+4.44%); a separate three-request cold API serving check at c733f5e measured 471.13 prefill tok/s, 17.232 s TTFT, and 17.60 decode tok/s.
• Passed: opt-in two-bank continuous serving with independent Qwen graphs, exact-frontier copying, and bounded partial-prefix checkpoints; the latest synchronized epoch completed as served=3 fallback=0 with no failures.
• Passed: recurrent Qwen disk-KV serialization and validation for serial sessions and idle banks, including a 1,152-token cross-process API restore with zero prompt recomputation.
• Passed: target-verified embedded MTP for greedy scalar/session decode, with token-stream identity, multi-token accepts, rollback on rejection, and quench telemetry.
• Passed: target-verified embedded MTP in the continuous two-bank loop, including two active drafting banks, rollback, lifecycle regression, API serving, and a controlled four-prompt decode A/B.
• Pending: true row-batched Qwen kernels, full-model quality comparisons, and a full-length 262,144-token prompt run.
• Pending: full-context performance characterization and SSD latency-distribution measurements.

No full-model quality, full-length 262K prompt, sustained-throughput, or

SSD-stall-distribution claim is made before those remaining gates pass.

Required performance reporting

The final report will separate model resident bytes, bounded PLE-cache bytes, CUDA allocations, system used/available memory, KV/cache allocations, and peak prefill/decode memory. SSD counters include unique pages per token, cache hit/miss ratio, logical/physical bytes per token, IOPS/bandwidth, read latency p50/p95/p99, prefetch overlap, and—most importantly—PLE wait-at-layer mean/p50/p95/p99. Serving metrics include TTFT, TPOT, prefill/decode tok/s, and MTP draft/accepted-token statistics.

Runtime implementation references

The ds4 implementation will consult the public serving paths where they provide relevant Qwen3.8-Flash-Next architecture and scheduling evidence:

• SGLang Qwen3.8-Flash-Next cookbook
• vLLM Qwen3.8-Flash-Next recipe
• TokenSpeed Qwen3.8-Flash-Next recipe
• Qwen3.8-Flash-Next official repository

These engines do not establish compatibility with this external-PLE GGUF layout.

Public mixed-quant references

The workflow is informed by the publicly accessible model releases in Baekpica/ds4-mixed-quant-for-spark. They are referenced as public precedents for region-aware quantization, artifact auditing, and Spark memory budgeting; no private handoff bucket is required to understand this model card.

License

The converted weights remain governed by the original Qwen Community License 1.0. The exact pinned upstream license is included with the artifact. No Apache-2.0 license is claimed.