---

license: apache-2.0

base_model:

- ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF

- Qwen/Qwen3.8-27B

base_model_relation: finetune

tags:

- gguf

- qwen3.8

- gated-deltanet

- scale-recovery

- distillation

- xyntetik-runner

---

Qwen3.8-27B GSQ-RCO IQ3_S with distillation-recovered scales (GGUF)

<!-- BEGIN xyntetik-status -->

> Research artifact, not a general-purpose release.

> - What was changed: only the fp16 d/dmin scale fields of every quantised block in 399 tensors, retrained by distillation against the BF16 parent; integer codes, format, byte length and offsets identical to the source file.

> - Why it exists: to test whether scale-only recovery brings an existing low-bit file inside the fidelity bar without re-quantising it.

> - Parent model: ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF (IQ3_S), from Qwen/Qwen3.8-27B

> - Useful for normal inference? yes, wherever the source IQ3_S file was: same loader path, and it tracks the parent closer (0.0450 / 97.80% vs the source's matched row on this card).

> - Experiment: Model Surgery and Scale Recovery

> - Evidence: scale-recovery evidence (dataset: run record, logs, fidelity and HellaSwag records, code), also copied under evidence/ here

> - Runner compatibility: Runner v0.5.3 or later serves the whole file on CUDA (i-quant kernels on the tensor cores, no CPU fallback); Runner v0.5.6 or later serves it on Metal too (codebook i-quant kernels, byte-exact against the CPU on the fixture gate). Any engine that reads the source file reads this one.

> - MTP draft head: Qwen3.8-27B-MTP-Q8_0.gguf in this repo, a speculative-decoding companion (llama.cpp on a GPU: about 1.9x decode; Runner 1.1.3 on CUDA: about 1.1x, 1.5x on its opt-in int8 route; same text); see the head section.

<!-- END xyntetik-status -->

<!-- BEGIN xyntetik-actions -->

Want this for another model? The tools behind these files are open: requantization, expert pruning and sublayer removal in Xyntetik Runner, and the method for this file is on this card or in its evidence dataset.

Request a model · Report your result · Community results · All files

<p align="center"><a href="https://buy.stripe.com/9B69AUddpdx9auHgP27N600"><img src="https://xyntetik.com/assets/support-button.svg" alt="Support this work" height="44"></a></p>

<p align="center">If this file is useful to you, a contribution funds the hardware time behind the next one.</p>

<!-- END xyntetik-actions -->

ISTA-DASLab's GSQ-RCO IQ3_S file of Qwen3.8-27B, with every quantised

block's fp16 scales retrained end to end against the BF16 parent by knowledge

distillation on a half-prose corpus. **Same integer codes, same format, same

11,771,546,784 bytes, same loader path as the source file.** The only bytes

that changed are the d/dmin scale fields.

Runner fidelity against the BF16 parent on 500 held-out prose positions:

mean KLD 0.0450, margin-qualified top-1 97.80%, inside the house bar

(≤ 0.05, ≥ 97%). The matched-position row for the unmodified source file is

below, so the reader can see exactly what recovery changed.

The MTP draft head (added 2026-10-08)

Qwen3.8-27B-MTP-Q8_0.gguf is a speculative-decoding draft file for this

trunk: with llama.cpp on a GPU that holds trunk and head it decodes about

1.9x faster with the same greedy text, and it is not a model of its own (it

cannot generate without the trunk, it does not change what the trunk says,

and nothing in it was trained here).

The file (3,164,006,784 bytes, sha256

89e31f13400bac27b888f10bab02ff4d8da1f8610fa078dc32f9f3f0d2b6f6c9) holds the

publisher's own multi-token-prediction block: the 15 mtp.* tensors of

Qwen/Qwen3.8-27B (revision

1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0, shard model-00018-of-00018),

exported with llama.cpp's converter (`convert_hf_to_gguf.py --mtp --outtype

q8_0`): norms F32, matrices Q8_0, with the token embedding, output norm and

LM head of the same checkpoint that the converter writes beside the block.

No other change.

Measured on one MIG 1g.24gb slice of an RTX PRO 6000 Blackwell (trunk and

head fully on the device), 512-token and 4,096-token prompts, 128 greedy

tokens:

| Engine | plain decode tok/s | with the head | drafts accepted | text vs plain |

|---|---|---|---|---|

| llama.cpp b11433, -md ... --spec-type draft-mtp | 29.5-29.9 | 56.4-59.3 | 77% | not compared (server rows) |

| Xyntetik Runner 1.1.3, --mtp-file, CUDA (512-token prompt, two interleaved rounds) | 13.8-13.9 | 15.4 (k=2), 15.3 (k=3) | 2.72 / 3.20 tokens per round | byte-identical |

| Xyntetik Runner 1.1.3, CUDA with RUNNER_CUDA_Q8X=1 (experimental int8 route, off by default) | 14.8 | 22.1-22.3 (k=2), 22.0-22.1 (k=3) | same | byte-identical to the route's own plain decode; 3 argmax flips in 480 positions against the CPU, hence opt-in |

| Xyntetik Runner 1.1.2, CPU only, 16 cores | 4.3-4.6 | 4.2-4.6 | same | byte-identical |

The head pays most where the engine's batched rows are nearly free, which is

llama.cpp on a GPU. On Runner 1.1.3 it pays a little on CUDA (about 1.1x at

--draft-k 2; the default of 4 is slower than plain decoding on this file)

and is a correctness feature on the CPU: the drafted text is the plain text,

byte for byte. Runner's plain decode on these i-quant formats is still about

half of llama.cpp's on the same slice; that is an engine item being worked

on, not a property of the head.

# llama.cpp b11433 or newer
llama-server -m Qwen3.8-27B-GSQ-RCO-IQ3_S-recovered.gguf \
  -md Qwen3.8-27B-MTP-Q8_0.gguf --spec-type draft-mtp -ngl 99
# Xyntetik Runner 1.1.3 or newer (CUDA and CPU; 1.1.1 and 1.1.2 draft on the CPU only)
runner -m Qwen3.8-27B-GSQ-RCO-IQ3_S-recovered.gguf --mtp-file Qwen3.8-27B-MTP-Q8_0.gguf --draft-k 2

The head is Qwen's weights under the same Apache-2.0 license as the trunk;

the conversion is deterministic from the stated revision.

Serve with xyntetik-runner

runner -m Qwen3.8-27B-GSQ-RCO-IQ3_S-recovered.gguf --serve --port 8080

Runner v0.5.3 or later serves every tensor in this file on CUDA, i-quants

included, with prefill on the tensor cores; the whole 11.8 GB file is

device-resident on a 16 GB card. Runner v0.5.6 or later serves it on Metal

as well: the seven codebook i-quants have Metal kernels, measured 2.2 to 2.5x

the CPU and gated byte-exact against the CPU encoders on the 29-fixture set, so

on Apple silicon with enough unified memory the file runs on the GPU. Any engine

that reads the source file reads this one: nothing in the header, the tensor

list, or the offsets moved.

Quickstart

runner -m Qwen3.8-27B-GSQ-RCO-IQ3_S-recovered.gguf -p "The capital of France is" -n 32

What was changed, exactly

• Changed: the fp16 d field of every block in 399 quantised tensors

(103,567,360 blocks), plus the fp16 dmin field where the format carries

one (Q2_K, Q4_K). Nothing else.

• Unchanged: every integer code, codebook index and sign; every F32, BF16

and IQ1_M tensor; the embedding and output-head scales; all metadata; all

tensor offsets. The output has the source's exact byte length.

• Verified after writing: every patched tensor re-decoded with the reference

gguf decoder equals the trained weights; every unpatched tensor is

byte-identical to the source. Largest change to any single decoded weight:

0.0195.

RECOVERY.json records the source file's SHA-256, the parent revision, the

training checkpoint hash, the output hash, the counts above and the

measurements below.

Why scales

GSQ optimises the integer grid assignments with a gradient method and RCO

allocates a type per tensor under a size budget; neither trains anything

through the model. In a fixed GGUF the remaining continuous degree of freedom

is the per-block scale, which is linear in the decoded weight for every block

format, so it can be trained end to end against the parent's distribution on

the student's own forward pass without reimplementing any codec; the second

phase of EfficientQAT applied to an existing file. The distillation blend was

half Wikitext-103 prose on purpose: prose is where every low-bit variant of

this model separates from its parent, and the one published trained quant of

it traded prose away.

Training: 300 steps × 4 sequences × 512 tokens (0.6 M tokens), lr 3e-4 cosine,

top-64 forward-KL to the parent with the tail as one bucket, only scales

trainable, one decoder block resident on a 24 GB slice at a time.

Measured envelope

Two protocols, kept apart. The first is the release bar. The second is the

selection metric used during development and is not comparable to the first.

**Runner fidelity, scripts/kld-compare-raw.py, against the BF16 parent, the

same 500 held-out Wikitext-2 positions for both rows, tie band 0.5 nats:**

| vs BF16 parent | mean KLD | top-1 | margin-qualified top-1 | top-8 overlap |

| --- | ---: | ---: | ---: | ---: |

| source GSQ-RCO IQ3_S, unmodified | 0.0480 | 86.60% | 97.60% | 0.882 |

| this file | 0.0450 | 87.20% | 97.80% | 0.884 |

House bar: KLD ≤ 0.05 and margin-qualified top-1 ≥ 97%. This file: PASS.

Against the unmodified source on the same positions, every reported metric moved in the recovered file's favour (KLD 0.0480 → 0.0450, 261 of 500 positions improved), but the paired 95% interval on the KLD difference is [−0.0098, +0.0038] and includes zero at this sample size, so "passes the bar" is the claim, and "better than the source" is a consistent direction with an unresolved magnitude. A larger-sample comparison is being added.

Selection protocol (development only, not a release claim): top-64

forward KL and top-1 agreement against the parent on the student's own

forward, 32 mixed instruction examples and 32 Wikitext-2 prose examples of 512

tokens, measured before and after training.

| dev set | top-64 KL before → after | top-1 before → after |

| --- | --- | --- |

| mixed (SmolTalk test) | 0.1939 → 0.1379 | 89.38% → 89.95% |

| prose (Wikitext-2 test) | 0.0485 → 0.0437 | 89.74% → 90.64% |

Benchmark annex (added 2026-09-14): HellaSwag validation, fixed random

subsample of 1,000 items (seed 20260911), parent and candidate scored through

one float32 reference forward, paired. Fidelity and benchmark equivalence are

different quantities; see Limits.

| | acc | acc_norm |

| --- | ---: | ---: |

| BF16 parent | 63.4% | 84.6% |

| this file | 61.8% | 83.5% |

| paired Δ vs parent (points), 95% bootstrap | -1.60 [-2.80, -0.50] | -1.10 [-2.00, -0.20] |

| discordant items (wins / losses), exact binomial p | 34 (9 / 25), p = 0.009 | 23 (6 / 17), p = 0.035 |

| source GSQ-RCO IQ3_S, unmodified (added 2026-09-15) | 61.7% | 83.4% |

| source Δ vs parent | -1.70 [-2.80, -0.60] | -1.20 [-2.10, -0.30] |

| this file − source, paired | +0.10 [-0.30, +0.50] (3 / 2) | +0.10 [-0.40, +0.60] (4 / 3) |

Both differences from the parent fall outside a ±0.5-point equivalence

margin: the file is inside the fidelity bar and measurably below the parent on

this task, the divergence documented in the Muse-Glimmer surgery report. The

source-file row (added 2026-09-15) locates that cost: it is the source

quantisation's, and scale recovery neither reduced nor added to it; the

recovered and source files disagree on 5–7 of 1,000 items, net +0.1 point,

interval through zero.

Limits, read before quoting

• Parent-agreement is not a capability benchmark. The fidelity rows measure

how closely next-token distributions track the parent on held-out prose.

• "No quality loss" is not claimed and is not supported by this evidence.
• Two fidelity rows, two protocols. The Runner KLD is variant‖reference over

the union of both sides' top-20 log-probabilities; the selection KL is

teacher‖student over the top-64 with a tail bucket. They are not

interchangeable, and neither is the full-vocabulary KL some cards report.

• Metal needs Runner v0.5.6. Runner serves the file on CUDA since v0.5.3

and on Metal since v0.5.6, when the codebook i-quant kernels landed; an

older Runner on Apple silicon runs the whole file on the CPU.

• Selection data. The training corpus, both development sets and the

harness corpus were used to build or select this file. Only the benchmark

annex touches none of them.

• Thin margin on agreement. 97.80% against a 97% bar on 500 positions.
• Multimodal not evaluated. The parent includes vision; this is the text

backbone and text-only measurements cannot speak to it.

• One model, one corpus blend, one shared 24 GB slice.

Reproduce

# 1. Decompose the source file: integer codes frozen, scales trainable (bit-exact by construction).
python build_unit_cache.py --gguf Qwen3.8-27B-GSQ-RCO-IQ3_S.gguf --output cache/
# 2. Teacher targets from the BF16 parent on the training corpus.
python teacher_topk.py --corpus evidence/training-corpus.json --output evidence/teacher-train
# 3. Train the scales.
python train_scales.py --gguf Qwen3.8-27B-GSQ-RCO-IQ3_S.gguf --cache cache/ \
    --corpus evidence/training-corpus.json --teacher evidence/teacher-train \
    --steps 300 --batch 4 --lr 3e-4 --warmup 25 --output run/
# 4. Write them back into a byte copy of the source, with verification.
python write_scales.py --source Qwen3.8-27B-GSQ-RCO-IQ3_S.gguf --cache cache/ \
    --checkpoint run/checkpoint-00300.pt --output Qwen3.8-27B-GSQ-RCO-IQ3_S-recovered.gguf
# 5. Fidelity in the release protocol.
python scripts/kld-compare-raw.py --model-a Qwen3.8-27B-GSQ-RCO-IQ3_S-recovered.gguf \
    --model-b Qwen3.8-27B-BF16-backbone.gguf --runner ./runner \
    --corpus harness-prose-heldout.txt --max-positions 500 --out fidelity.json

Code and the full evidence record (every anchor, every run, the negative

results on the smaller IQ3_XXS file): the code/ folder of this repository (the study is not yet on GitHub) at commit 51cead8.

Provenance

• Parent: Qwen/Qwen3.8-27B, revision 1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0, Apache-2.0.
• Source quant: ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF,

Qwen3.8-27B-GSQ-RCO-IQ3_S.gguf, SHA-256 64b53b64c7aa39f2…, Apache-2.0.

GSQ: arXiv:2604.18556. RCO: arXiv:2605.00649. The hard part of this

artifact, the codes, is theirs.

• This file: SHA-256 873df48dd3ecf205…, 11,771,546,784 bytes.
• Training checkpoint: SHA-256 8a97f066829ec798…, step 300.
• Training data: Wikitext-103 (train) and SmolTalk (train shards 1–8), 2,048,000

tokens, hashes in RECOVERY.json. No development, harness or benchmark text.

<!-- xyntetik-program -->

About Xyntetik

This repository is published by Xyntetik, the Zenova AB program for running capable language models on hardware you own and proving what they did. Its released engine is Xyntetik Runner: a free, single-binary GGUF engine in C for CPU, CUDA and Metal that serves, scores and trains through the quantized file it deploys and records every run as a receipt. The measurements behind the program's claims are collected at xyntetik.com/evidence. The program's organization on Hugging Face is Xyntetik.