---

license: apache-2.0

base_model:

- Joakimpalm-Zen/Muse-Glimmer-30B-Surgical-Q8_0-GGUF

- meta-models/Muse-Glimmer-30B

base_model_relation: quantized

tags:

- gguf

- muse-glimmer

- depth-surgery

- scale-recovery

- distillation

- xyntetik-runner

---

Muse-Glimmer-30B, 6.34% of the decoder removed, Q4_K with distillation-recovered scales (GGUF)

<!-- BEGIN xyntetik-status -->

> Research artifact, not a general-purpose release.

> - What was changed: the healed 6.34% model at Q4_K with every block's fp16 scales retrained by distillation against the BF16 parent; integer codes unchanged.

> - Why it exists: to close the gate the surgery study left open: the 6.34% model could not ship at plain Q4_K (0.05815, fail); with recovered scales it reads 0.04949 / 97.90%, inside the bar.

> - Parent model: Muse-Glimmer-30B-Surgical-Q8_0-GGUF, itself from meta-models/Muse-Glimmer-30B

> - Useful for normal inference? as usable as the Q8_0 sibling at half the bytes, with the same benchmark-annex caveat.

> - Experiment: Model Surgery and Scale Recovery

> - Evidence: scale-recovery evidence (dataset: preregistration, four-arm scores, logs, code), also copied under evidence/ here

> - Runner compatibility: any Runner release; Q4_K is GPU-resident; any runtime that runs the parent runs this file. --kv k8v4 and --kv fp4 from Runner v0.5.6, measured on this file below.

<!-- END xyntetik-status -->

<!-- BEGIN xyntetik-actions -->

Want this for another model? The tools behind these files are open: requantization, expert pruning and sublayer removal in Xyntetik Runner, and the method for this file is on this card or in its evidence dataset.

Request a model · Report your result · Community results · All files

<p align="center"><a href="https://buy.stripe.com/9B69AUddpdx9auHgP27N600"><img src="https://xyntetik.com/assets/support-button.svg" alt="Support this work" height="44"></a></p>

<p align="center">If this file is useful to you, a contribution funds the hardware time behind the next one.</p>

<!-- END xyntetik-actions -->

The healed 6.34% surgical model — FFN sublayers of layers 4, 7, 9 and 48

removed, one FFN retrained per cut, the same weights as the published Q8_0

release — at Q4_K, 14.61 GiB, with every quantised block's scales retrained

against the frozen BF16 parent. **Mean KLD 0.04949, margin-qualified top-1

97.90% on the study's 45,056-position held-out split — inside the house bar.**

This closes the gate the surgery study left open. Its dense-ladder arithmetic

(L6) found the 6.34% model could not ship at Q4_K: BF16 surgery cost 0.03857

plus the Q4_K quantisation term 0.01930 lands near 0.056, and three levers on

the surgery term left the gap at +0.00561. Recovery attacks the quantisation

term instead — and the two turn out not to be independent once the scales are

trained.

Serve with xyntetik-runner

runner -m Muse-Glimmer-30B-surgical-6p34-Q4_K-recovered.gguf --serve --port 8080

Q4_K is GPU-resident in Runner. The 12 removed FFN tensors are stored as exact

zeros in standard shape, as in the Q8_0 release, so any runtime that runs the

parent GGUF runs this file with no loader changes.

Quickstart

runner -m Muse-Glimmer-30B-surgical-6p34-Q4_K-recovered.gguf -p "The lighthouse keeper" -n 64

What was changed, exactly

Two layers of change, each recorded in its own sidecar:

• Surgery (SURGERY.json, from the study): FFN sublayers at layers 4, 7, 9

and 48 zeroed; the first surviving FFN downstream of each cut (layers 5, 8,

10 and 49) retrained against the frozen parent on the run-9 audited corpus

(14.4× dose). These are the weights

of Muse-Glimmer-30B-Surgical-Q8_0-GGUF (run9_d14x_bf16.gguf).

• Quantisation: Runner's own quantiser, --quant q4_k, every quantised

tensor Q4_K (418 tensors), F32 elsewhere. Unmodified, this file scores

0.05615 / 97.41% — FAIL.

• Recovery (RECOVERY.json): the fp16 d and dmin scale fields of every

block in 416 Q4_K tensors (196,591,616 fields) retrained; every integer

code, every F32 tensor, the embedding and output-head scales, all metadata and

offsets unchanged. Byte length identical to the unrecovered file. Verified:

every patched tensor re-decodes to the trained weights, every other tensor is

byte-identical. Largest change to any decoded weight: 0.0071.

Training: 400 steps × one 2048-token window of corpus_run9 (0.82 M tokens,

the study's window-audited training corpus, nothing else; 20 of its 474 sequences

held aside as the kill-check dev proxy, the study's held-out split never read

during training), lr 3e-4 cosine,

top-64 forward KL to the parent through the study's own bf16 streamer and

softcapped logit function, only scales trainable.

Why scales, and what the control found

In a fixed GGUF the per-block scale is the remaining continuous degree of

freedom, linear in the decoded weight for every block format; it can be trained

end to end against the parent's distribution without reimplementing any codec.

The study's ladder had attacked only the surgery term (healing dose, seeds,

rung-2 adapters, wider repair windows). This is the first lever on the other

term.

The preregistered control — the same recovery on the parent's Q4_K, no

surgery — tested whether the study's independence law (quantisation and

surgery costs additive, five prior points at 0.965–0.996×) survives recovery.

It does not:

| | KLD | margin-q |

| --- | ---: | ---: |

| parent Q4_K, unmodified (quant term) | 0.01930 | 99.12% |

| parent Q4_K, recovered | 0.01518 | 99.43% |

| additive prediction for this file: 0.03857 + 0.01518 | 0.05375 | |

| this file, measured | 0.04949 | 97.90% |

| implied stacking factor | 0.921 | |

Recovery removed 21% of the quant term on the parent and 38% on the surgical

file: scales trained against the parent took back part of the surgery

error too. Recovered quantisation and surgery are the first sub-additive pair

in the study's compounding taxonomy. Preregistered as the refutation outcome

(supported-preregistered); the mechanism is an interpretation

(supported-exploratory). It moves the ladder's gate from BF16 surgery

≤ 0.0307 to ≤ ~0.039 at recovered Q4_K, which predicts the rung-2 6.34% model

at ≈ 0.0474 — a test written down before it is run.

Measured envelope

**Study scorer (run6_x1_score.py), 11 held-out sequences / 45,056 positions,

frozen BF16 parent, never trained on:**

| artifact | size | KLD | margin-q | verdict |

| --- | ---: | ---: | ---: | --- |

| this file: surgery + Q4_K + recovered scales | 14.61 GiB | 0.04949 | 97.90% | PASS |

| surgery + Q4_K, unrecovered | 14.61 GiB | 0.05615 | 97.41% | FAIL |

| same weights at Q8_0 (published) | 27.58 GiB | 0.03883 | 98.34% | PASS |

| same weights, BF16 | 51.90 GiB | 0.03857 | 98.38% | PASS |

| parent at Q4_K (control) | 14.61 GiB | 0.01930 | 99.12% | PASS |

| parent at Q4_K, recovered (control) | 14.61 GiB | 0.01518 | 99.43% | PASS |

**Runner fidelity (scripts/kld-compare-raw.py, both models on CPU, 500

positions of the detokenised held-out split, tie band 0.5 nats):**

| vs BF16 parent | mean KLD | top-1 | margin-q | top-8 |

| --- | ---: | ---: | ---: | ---: |

| surgery + Q4_K, unrecovered | 0.0519 | 95.20% | 98.80% | 0.837 |

| this file | 0.0474 | 95.40% | 98.80% | 0.849 |

Both instruments agree on the headline: the unrecovered file fails the KLD

term, this file passes it. The 500-position runner pair's paired interval

includes zero; the 45,056-position study scorer resolves the same pair at

9.4× the study's ±0.00071 noise floor.

Benchmark annex (separate quantity): HellaSwag validation, fixed random

1,000-item subsample, parent and candidate through one instrument, paired.

| | acc | acc_norm |

| --- | ---: | ---: |

| BF16 parent | 61.7% | 83.5% |

| this file | 61.2% | 82.2% |

| paired Δ (points), 95% bootstrap | −0.50 [−1.70, +0.70], p = 0.50 | −1.30 [−2.50, −0.10], p = 0.047 |

The study's published full-n figure for this surgery at BF16 is −1.52

[−1.87, −1.15] on acc_norm; quantisation plus recovery added no measurable

task loss beyond the surgery's own within this subsample's interval.

KV cache on a GPU (added 2026-09-22)

Measured on this file: 500 held-out positions, tie band 0.5, one RTX PRO 6000

Blackwell MIG 1g.24gb slice through --gpu auto, with Xyntetik Runner 0.5.5 at

the kv-split tip (the --kv k8v4 and --kv fp4 flags ship in

v0.5.6).

A 24 GB slice holds one copy of this model and not two, so the reference is a

CPU --kv f16 server and the first line prices the backend on its own.

| cache | KLD vs f16 | margin-qualified top-1 | top-8 overlap |

| --- | ---: | ---: | ---: |

| f16 on GPU (backend only) | 0.0000002 | 100.0% | 0.99975 |

| k8v4 (K q8_0, V fp4) | 0.0036 | 99.8% | 0.961 |

| fp4 | 0.0209 | 99.4% | 0.903 |

k8v4 costs 16 to 29x the q8 cache's KLD (0.002 to 0.004 absolute) at 41% of the

f16 bytes. The backend contributes 2.0e-07, four orders below either cache

format, and the CPU's own reassociation floor at prompt batch 1 against batch

64 is exactly 0.0 on the same 500 positions, so these numbers price the cache

format and not the port.

Scope: one model, one quant, one slice, greedy positions from one held-out

split. Models with an attention K bias behave differently under fp4 and are a

separate row class.

Reproduce: scripts/kld-compare-raw.py with --args-a "--kv k8v4" and

KLD_GPU=auto on side A against --args-b "--kv f16" on a CPU side B, same

GGUF on both sides. A second build of the runner on a different toolchain

agrees with the binary these rows were taken with to 3.3e-07 mean KLD at 100%

top-1 agreement on the same model and corpus. Records: the

scale-recovery evidence dataset,

evidence/kv-cache-2026-09-22/.

Limits, read before quoting

• **The margin under the bar is 0.00051 — 0.7× the study's ±0.00071 noise

floor.** This file passes because it was measured, not comfortably; a

different training seed could plausibly land either side. The same

qualification the study attached to its 4.75% Q4_K release (#30).

• Parent-agreement is not a capability benchmark. The annex shows the

surgery's task cost persists: fidelity and benchmark equivalence are

different quantities.

• "No quality loss" is not claimed and is not supported by this evidence.
• Two fidelity instruments, two protocols. Study-scorer KL is full-vocabulary

variant‖parent; the runner harness is variant‖reference over the top-20

union. Not interchangeable.

• Size is not reduced by the surgery. The zeroed FFN tensors are still

stored (0.84 GiB at Q4_K, verified still exact zeros after recovery);

dropping them is a runtime-side edit not applied here.

• Selection data. corpus_run9 trained the scales; the held-out split was

opened once, by the scorer, for the numbers above. Benchmark items touched

none of the study's data.

• One model, one corpus, one shared 24 GB slice.

Reproduce

# 0. Starting point: the study's healed BF16 (weights of the published Q8_0).
# 1. Quantise with Runner's own quantiser.
runner -m run9_d14x_bf16.gguf --quantize healed_6p34_run9_q4_k --quant q4_k
# 2. Decompose (codes frozen, scales trainable; bit-exact by construction).
python build_unit_cache.py --gguf healed_6p34_run9_q4_k.gguf --output cache/
# 3. Teacher targets from the frozen parent through the study's streamer.
python muse_teacher.py --output teacher-run9
# 4. Train the scales.
python train_muse_scales.py --gguf healed_6p34_run9_q4_k.gguf --cache cache/ \
    --teacher teacher-run9 --steps 400 --seq-tokens 2048 --lr 3e-4 --output run/
# 5. Write back with verification.
python write_scales.py --arch muse --source healed_6p34_run9_q4_k.gguf --cache cache/ \
    --checkpoint run/checkpoint-00400.pt --output Muse-Glimmer-30B-surgical-6p34-Q4_K-recovered.gguf
# 6. Score on the study's held-out split.
python scripts/run6_x1_score.py --variants variants.json --out score.json

Code in code/; the preregistration (hypothesis, lean, criteria, kill rule,

and the control) with its commit timestamps in evidence/PREREG_MUSE_SR1.md.

Provenance

• Parent: meta-models/Muse-Glimmer-30B, revision

a4e59da52a7bc87ae7251dd5545c0dd437c44b68, Apache-2.0.

• Surgical weights: the Muse-Glimmer surgery study's run9_d14x_bf16.gguf

(healed FFN[4,7,9,48], 14.4× dose), published as

Joakimpalm-Zen/Muse-Glimmer-30B-Surgical-Q8_0-GGUF; SURGERY.json included.

• Unrecovered Q4_K: SHA-256 7a780a3e21b34341…, 15,686,266,304 bytes.
• This file: SHA-256 12a429483b18aefc…, 15,686,266,304 bytes.
• Training checkpoint: SHA-256 9ed79cd4d6667a84…, step 400.
• Training data: corpus_run9 only (window-audited); the first 2048 tokens of

400 of its 454 training sequences, each seen once, seed 20260913; hashes in

RECOVERY.json.

<!-- xyntetik-program -->

About Xyntetik

This repository is published by Xyntetik, the Zenova AB program for running capable language models on hardware you own and proving what they did. Its released engine is Xyntetik Runner: a free, single-binary GGUF engine in C for CPU, CUDA and Metal that serves, scores and trains through the quantized file it deploys and records every run as a receipt. The measurements behind the program's claims are collected at xyntetik.com/evidence. The program's organization on Hugging Face is Xyntetik.