---

pipeline_tag: text-generation

library_name: gguf

base_model: z-lab/Qwen3.5-397B-A17B-DFlash

base_model_relation: finetune

license: apache-2.0

inference: false

tags:

- dflash

- speculative-decoding

- speculative-decoding-draft

- draft-model

- block-diffusion

- gguf

- ik_llama.cpp

- qwen3.5

- ornith

- iq2_xxs

- moe

---

DFlash Drafter for Ornith 1.5 397B A17B IQ2_XXS (GGUF)

This is a drafter, not a language model. On its own it produces nothing useful. It is

loaded through --model-draft alongside Ornith-1.5-397B-A17B-IQ2_XXS and predicts that

model's next few tokens from its hidden states.

It runs on ik_llama.cpp only. Mainline

llama.cpp and the poolside fork both reject it; see Compatibility before

downloading.

Against the previous drafter for this target, on code prompts it was never trained on, it

decodes +6.5% faster at n_max=3 (95% CI +0.3% to +12.6%, four server boots,

counterbalanced). Acceptance rate rises from 0.47 to 0.52 at the same setting. Full numbers

and their weaknesses are in Evaluation.

Files

| File | Size | Precision | SHA-256 |

|---|---|---|---|

| Ornith15-DFLASH-codering-q4.gguf | 704 MiB | mixed iq4_kss / iq3_ks | 5ee79024b8337367f222ceaa050739b03553d4669551db394aa21227e2f271f6 |

This is the measured artifact: every number below was produced with this exact file, and the

SHA-256 is there so you can confirm you are reading results about the bytes you downloaded.

A pure q8_0 build of the same weights exists locally and is not published, because it was

never benchmarked and would only invite the assumption that a larger file is a better one.

Ask if you want it.

The q4 mix was chosen to match the incumbent drafter's, so that an A/B between the two

compares weights rather than quantization:

dflash_fc.weight=iq4_kss, ffn_down=iq4_kss, ffn_gate=iq3_ks, ffn_up=iq3_ks

Attention projections and norms stay at Q8_0.

Usage

llama-server \
  --model       Ornith-1.5-397B-IQ2_XXS-00001-of-00003.gguf \
  --model-draft Ornith15-DFLASH-codering-q4.gguf \
  --spec-type   dflash:n_max=3,cross_ctx=512 \
  --n-gpu-layers 99 --n-gpu-layers-draft 99 \
  --draft-params "-wgt 17" -wgt 8 \
  --flash-attn on --parallel 1

All three target shards must sit in one directory; naming shard one loads the rest.

--parallel 1 is a hard requirement, not a tuning choice. This ik_llama.cpp build **refuses

to start** with speculative decoding and more than one slot, because the accept path indexes

the batch absolutely. It does not degrade to serialized slots.

The two -wgt flags reserve worst-case graph space for the target and draft contexts. They

were set during measurement, so omitting them will not reproduce the numbers below.

cross_ctx=512 is the drafter's ring depth and is fixed by the architecture. It also

constrains capture: exporting training features with a prefill batch larger than 512 evicts

ring rows before they are written, which silently corrupts the pack.

Choosing n_max

Swept 2026-08-28 on one boot at N_MAX=6, four unseen ~32k-token prompts, two repetitions,

speculative.n_max overridden per request so every depth shares a single model load:

| n_max | tok/s | vs spec-off | acceptance |

|---:|---:|---:|---:|

| 0 | 15.00 | — | — |

| 2 | 19.07 | +27.3% | 0.562 |

| 3 | 19.18 | +28.0% | 0.462 |

| 4 | 19.82 | +32.4% | 0.431 |

| 5 | 17.72 | +18.5% | 0.334 |

The curve rises to 4 and turns over at 5, so the optimum is bracketed and depths of 6 and

above need no test. Acceptance falls monotonically with depth, as expected.

Depths 2, 3 and 4 are not statistically separable on this data. Paired by prompt against

the shipped setting: n_max=2 is −0.0% (t=0.00), n_max=4 is +4.5% (t=0.75, 95% CI

[−14.5%, +23.4%]), n_max=5 is −6.7% (t=−1.30).

The fall-off at 5 needs stating precisely, because which depth you compare it to changes the

answer. Against n_max=4 it is −2.10 tok/s with t = 10.9 and every prompt agreeing, which is

about as clean as this rig produces. Against the shipped n_max=3 it is −1.46 tok/s with

t = 1.45 and one prompt going the other way — not separable. So 5 is reliably worse than the

peak, and only ambiguously worse than what is shipped.

n_max=3 ships unchanged. 4 is the point-estimate peak, but nothing here distinguishes it

from 3, and the previous drafter's optimum of 4 is a prior rather than evidence about this

one.

Worth knowing if you extend this: repetition-to-repetition spread within a cell had a median

of 0.06 tok/s across 20 cells (one outlier at 1.27), while prompt-to-prompt spread ran 2.7 to

5.8 tok/s depending on depth, and one prompt (fresh-03) sat 3–5 tok/s below the others at

every depth. The uncertainty here is prompt heterogeneity, not measurement noise, so **more

prompts would sharpen this and more repetitions would not.**

Verify cost on this target is roughly 16.2 ms per extra draft position, and Ornith 1.5 is a

sparse MoE whose experts do not amortize across draft positions. Deeper drafting therefore

stops paying well before acceptance reaches zero.

Architecture

1.29B parameters in 6 layers. The drafter reads the target's hidden states from 8 layers

(1, 9, 17, 25, 33, 41, 49, 57), concatenated to a 32768-wide input, and projects them

through dflash_fc into its own 4096-wide residual stream. A block of 16 mask slots seeded

with the anchor token is decoded in one forward pass, and the resulting hidden states are

pushed through the target's frozen embedding and output head.

| | |

|---|---|

| Layers | 6 (sliding_attention ×5, full_attention ×1) |

| Hidden / intermediate | 4096 / 12288 |

| Attention heads | 32 query, 8 KV, head_dim 128 |

| Sliding window | 4096 |

| RoPE theta | 1e7 |

| Block size | 16 (mask token 248077) |

| Vocab | 248320 |

RoPE theta must read 1e7 in the GGUF metadata. At the Qwen default of 1e6 acceptance

collapses, and the converter reads the top-level rope_theta key rather than the nested

rope_parameters block, so a config carrying only the nested value converts to a drafter

that loads cleanly and drafts badly.

The file contains no target weights. Its 69 tensors are the six blocks, dflash_fc,

dflash_hidden_norm and output_norm. There is no token_embd and no output.weight;

both come from the target at runtime. Redistributing this drafter therefore redistributes

only weights derived from the z-lab DFlash initialization.

Known trainer/runtime divergence

The training script forces all six layers to full attention, while the shipped config.json

correctly declares sliding attention on layers 0–4. The drafter was therefore trained able to

attend to later slots inside its block, and is served without that ability in five of six

layers.

This was measured rather than assumed. A fixture that reproduces both graphs on the same

weights and anchors puts the difference at ±0.8 pp on per-position match, in both

directions, at every slot count tested. The window is 4096 while intra-block distances are at

most 15, so it never binds. The divergence is real and worth knowing about; it is not worth

correcting.

Training

Initialized from the Ornith-1.0 drafter (itself initialized from z-lab/Qwen3.5-397B-A17B-DFlash),

then fine-tuned on target features captured from the served IQ2_XXS model.

The motivating observation: three previous retrains had produced almost nothing at deployed

settings, and every one of them trained on anchors from long prose generations off short

prompts. Production's drafter ring holds code at absolute positions above 30,000. The

untested variable was the data distribution, not the loss or the schedule.

| | |

|---|---|

| Capture | 12 prompts × ~128k chars of C/C++ from the ik_llama.cpp tree, 4096 generated tokens each |

| Pack | 457,790 rows, 30 GB, 12 segments, 0 malformed |

| Split | 9 segments train (348,025 rows) / 3 eval (109,765), split by captured request |

| Mixed with | the previous prose pack, pooling to 21,950 anchors |

| Schedule | 2 epochs, 2,742 steps at ~1.3 s/step, batch 16, lr 1e-4, stride 5 |

| Loss | position-weighted CE, gamma 1.5, weights normalized to sum 15 (w₁..w₅ = 7.30 / 3.75 / 1.92 / 0.99 / 0.51) |

The weighting concentrates on positions 1–3 because tau@3 ≈ 1 + p₁ + p₁p₂ — position 1 caps

everything downstream, and the deployed configuration only ever requests 3 or 4 slots.

Best tau@15 on held-out code was 4.42 against 3.91 for the initialization.

Evaluation

Two things are worth separating: whether the drafter is better, and by how much at the

setting you will actually serve.

Offline, on the deployed graph

Measured with the runtime's mask and slot count rather than the trainer's, on 250 anchors,

against the previous drafter for this target. Only genuinely held-out segments are included —

see Data contamination.

| Pack | Metric | Previous | This | Δ |

|---|---|---:|---:|---:|

| Held-out code | tau@3 (serving n_max=2) | 2.592 | 2.704 | +4.3% |

| Held-out code | tau@4 (serving n_max=3) | 3.100 | 3.268 | +5.4% |

| Held-out code | position-1 match | 0.908 | 0.928 | +2.0 pp |

| Prose (old distribution) | tau@3 / tau@4 | 2.452 / 2.916 | 2.504 / 2.976 | +2.1% |

The prose row matters more than its size suggests: this drafter was trained on code, and it

did not get worse anywhere measured.

End-to-end throughput

Four server boots in A-B-B-A order, four unseen ~32k-token code prompts, three repetitions,

256 tokens generated per request. Counterbalancing is necessary because boot-to-boot drift on

this hardware measured ~8%, larger than the effect. Deltas are paired within each half and

clustered by prompt, so the unit of analysis is the prompt rather than the request.

| n_max | Δ tok/s | 95% CI | t (df=3) |

|---:|---:|---|---:|

| 0 (control) | +0.6% | [+0.3%, +0.9%] | 6.70 |

| 2 | +3.9% | [−0.4%, +8.2%] | 2.88 |

| 3 | +6.5% | [+0.3%, +12.6%] | 3.34 |

Every prompt improved at n_max=3 (+11.2 / +7.3 / +1.9 / +5.5%). Acceptance rate rose in

both boots independently — 0.472 / 0.474 to 0.508 / 0.531 — which is read from the server's

own counters rather than a wall clock.

The control row is the honest caveat. With speculation disabled the two arms should be

identical, and instead the candidate's boots ran 0.6% faster with a CI excluding zero. Some

small boot-level advantage was not fully cancelled. Subtracting it gives roughly +5.9% at

n_max=3.

n_max=2 is directionally positive and not separable from zero.

Data contamination

The first version of this evaluation was partly contaminated, and the correction is

documented here rather than quietly fixed.

The prompt generator rotated a six-entry list of source files, so requesting 12 prompts

produced 12 prompts with only 9 unique bodies. The four benchmark prompts were the first

four of that same generator's output, and three of them were in the training split. One of

the three offline evaluation segments duplicated a trained segment.

After rebuilding the evaluation on one distinct unseen source file per prompt, hash-guarded

against the training corpus at build time:

• Offline recomputed on clean segments only: +6.9% / +8.6% fell to +4.3% / +5.4%. Part of

the original gap was memorization.

• End-to-end on fresh prompts: +5.6% became +6.5%, with the CI excluding zero for the

first time.

The contaminated online number was not inflated. That is the outcome, not the justification —

the original measurement could not have told you either way.

What is not established

• Anything outside code. The code corpus is nine unique prompt bodies drawn from six

C/C++ source files in one project. The evidence for general text is the +2.1%

no-regression row above, which rests on two held-out prose segments.

• The optimal n_max. See Choosing n_max.
• Other quantizations of the target. The drafter is distilled against IQ2_XXS

specifically, using that quantization's embedding and output head. Behaviour on IQ2_KS,

Q4_K_M or bf16 is unmeasured.

• Long-horizon serving. All measurements are 256-token generations on a benchmark rig,

not production traffic over days.

Four prompts and four boots is a small experiment. Treat +6.5% as the center of a wide

interval.

Compatibility

| Runtime | Result |

|---|---|

| ik_llama.cpp (HEAD f96eaddb or later) | Works |

| mainline llama.cpp | unknown model architecture: 'dflash-draft' |

| poolside llama.cpp | unknown model architecture: 'dflash-draft' — its DFlash arch is registered as dflash |

The published q4 file additionally uses ik-native quantization types that mainline ggml

does not define, so it fails at tensor load before even reaching the architecture check. The

quantization is not the obstacle, though: a pure q8_0 build of these weights, using only

mainline types, still fails on the architecture name. The format is the wall, not the mix.

Both forks implement DFlash. Their drafter formats are not interchangeable, and running this

drafter on poolside would require retraining and reconverting to that format.

Reproducing

Measured on an NVIDIA GB10 (DGX Spark), 121.6 GiB unified memory shared between CPU and GPU.

The target at IQ2_XXS with a 131,072-token context and q8_0 KV cache leaves a few GiB

free, so nothing else of size can run alongside it — including, on this hardware, an

evaluation script that expects to use "the CPU" as if that were a separate pool.

The full command sequence, the capture settings, the split, the training invocation and the

statistical analysis are in the source repository under kernel-lab/dflash-retrain/:

CODERING-REPRO-RUNBOOK.md for the commands in execution order,

CODERING-AB-STATISTICAL-NOTES.md for the A/B analysed with its weaknesses stated.

License

Apache 2.0, inherited from z-lab/Qwen3.5-397B-A17B-DFlash, of which these weights are a

fine-tune.

Training features were distilled from Ornith-1.5-397B outputs. If you intend to redistribute

or use this commercially, check Ornith 1.5's own license terms on distillation — that

question is not resolved by this card.