---

license: apache-2.0

base_model: meta-models/Muse-Glimmer-30B

tags:

• gguf
• quantized
• q4_k
• depth-pruning
• model-surgery
• structural-derivative
• xyntetik-runner

pipeline_tag: text-generation

---

Muse-Glimmer-30B, surgical: 4.75% of the decoder removed, healed, Q4_K

<!-- BEGIN runner-training -->

Serve with xyntetik-runner &mdash; the engine that trains, merges, and quantizes the GGUF it serves

This GGUF is built to be served with xyntetik-runner, a single-binary, dependency-free C11 inference engine (CPU / CUDA / Metal, OpenAI-compatible server). For this artifact the relevant differentiator is the training side:

> Runner trains, merges, and quantizes GGUFs deterministically: the same data and seed reproduce the same adapter file byte for byte, and every written artifact carries a provenance record.

One binary covers the loop: --train (AdamW LoRA on the served GGUF, deterministic by default, gated in CI by adapter sha256), --lora to serve the adapter beside the frozen base, --merge-lora to fold it in with per-tensor requantization and a merge.json provenance record, and --score for teacher-forced held-out evals. Adapters interoperate both ways: a Runner-trained adapter scores identically (1.000 on its held-out eval) served by stock llama.cpp, and community F16 adapters load straight back &mdash; measured, not assumed.

Measured &mdash; the finding every fine-tuner should know before merging (held-out eval where the fine-tune scores 1.00 and the base model 0.69; verified in two independent runtimes):

| merge output type | eval after merge | fine-tune survives? |

|---|---|---|

| F16 | 1.00 | yes |

| Q8_0 | 1.00 | yes |

| Q4_0 | 0.69 &mdash; the base model's score | no: erased |

A 4-bit merge silently deletes the adaptation while perplexity still looks fine. Merge to Q8_0 or F16, or keep serving base + adapter.

This artifact is that toolchain eating its own cooking. The surgery was applied by byte-level substitution into the parent GGUF, and the Q4_K file was written by Runner's own quantizer. The healed weights survive Q8_0 at a cost of +0.00006 KLD, and the Q4_K cost you see in the fidelity table was PREDICTED to 0.6% before the file was built, from the measured independence of quantization noise and surgery error. If you fine-tune on top of this model, the merge-erasure table above applies to you.

Quickstart

# the same binary that wrote this artifact:
runner -m model.gguf --train data.jsonl --train-out adapter.gguf
# same data + seed -> byte-identical adapter, build provenance in adapter.gguf.train.json
runner -m model.gguf --lora adapter.gguf --merge-lora merged --quant q8_0
# provenance lands in merged.gguf.merge.json; do not merge to 4-bit (table above)

The exact boundary of the determinism claim, including what is deliberately not promised: docs/determinism-scope.md.

<!-- END runner-training -->

Run it

This file is 14.61 GB, so it wants a machine with roughly 17 GB of usable

memory or more.

# Apple Silicon Mac
curl -L https://github.com/Joakimpalm-Zen/xyntetik-runner/releases/latest/download/runner-macos-arm64.tar.gz | tar xz
curl -L -o Muse-Glimmer-30B-surgical-4p75-Q4_K.gguf \
  https://huggingface.co/Joakimpalm-Zen/Muse-Glimmer-30B-Surgical-Q4_K-GGUF/resolve/main/Muse-Glimmer-30B-surgical-4p75-Q4_K.gguf
./runner-macos-arm64 -m Muse-Glimmer-30B-surgical-4p75-Q4_K.gguf -i

Linux and Windows builds are on the

same release page.

Any GGUF runtime with muse-glimmer support should load it: every tensor is

an ordinary GGUF type in the standard shape; 9 FFN tensors happen to be

exact zeros, which is the surgery.

A 14.61 GB GGUF of Meta's Muse-Glimmer-30B with the FFN sublayers of

layers 4, 9 and 48 structurally removed (weights exactly zero, standard

tensor shape), the damage repaired by short local distillation against

the frozen parent, and the result quantized to Q4_K.

It passes a strict fidelity bar against the unmodified BF16 parent on a

held-out split: **margin-qualified top-1 agreement 97.74% (bar: >=97%)

and mean KLD 0.04810 (bar: <=0.05)**.

Measured fidelity vs the BF16 parent

| artifact | size | mean KLD | margin-qualified top-1 | bar |

|---|---|---|---|---|

| this file (surgery + heal, Q4_K) | 14.61 GB | 0.04810 | 97.74% | PASS |

| same weights, Q8_0 | 27.58 GB | 0.03076 | 98.65% | PASS |

| same weights, BF16 | 51.90 GB | 0.03056 | 98.64% | PASS |

| parent at Q4_K (quant-alone control) | 14.61 GB | 0.01930 | 99.12% | PASS |

Per-domain behavior: the healed model sits within a few percent of the

parent in every measured domain (11-domain perplexity spread 0.046 vs

0.373 for un-healed surgery). Healing distils toward the parent: this

is a smaller model that behaves like the original, not a different

model that happens to score well.

Paired benchmark check (2026-08-29): the fidelity bar does not transfer

The fidelity table above is a real, strict measurement. This section

reports what it does not buy. One preregistered measurement night

translated the house bar into paired public-benchmark deltas versus

the frozen parent: lm-evaluation-harness 0.4.9, stock task configs,

both models scored through the identical loglikelihood path on

identical items, per-item pairing, exact binomial confidence intervals

on the discordant pairs, margins fixed before sampling.

| config | benchmark | preregistered margin | net paired delta | 95% CI | verdict |

|---|---|---|---|---|---|

| these weights at BF16 (pre-quant) | HellaSwag acc_norm | ±0.5 pts | −0.916 | [−1.230, −0.583] | FAIL |

The flip counts say the deficit is directional, not noise: on the

2.89% of items where the two models disagree, the surgical model loses

roughly 2:1 (191 parent-right-variant-wrong against 99 the reverse).

Stated plainly:

• This model is measurably worse than its parent on HellaSwag. The

deficit is about 0.9 points and the confidence interval lies entirely

beyond the preregistered ±0.5 equivalence margin. Do not read the

fidelity table above as benchmark equivalence: the two criteria

disagree, and the paired test is the stricter one.

• The deficit was measured at the BF16 surgery stage. The shipped

file adds Q4_K quantization on top (its own fidelity row is above). A

direct benchmark row for the quantized file is currently blocked for

tooling reasons (no available continuation-loglikelihood serving path

for this architecture), not by choice; it will be added when the path

exists, and there is no reason to expect quantization to improve it.

• **Across the two surgery levels the KLD-to-benchmark mapping is

monotone and super-linear** (30.0 to 39.5 points lost per unit of

KLD, from 4.75% to 6.34% removed): deeper surgery costs more

benchmark accuracy per unit of measured divergence, so fidelity

numbers under-predict benchmark harm as surgery grows.

The parent-fidelity table remains this card's primary claim: these

weights track the parent's token distribution closely, and that is

what they are for. The benchmark row exists so nobody has to take

distribution fidelity as a proxy for task equivalence. We measured

that it is not one.

How it was made, in five lines

1. Depth-sensitivity mapping found the FFN sublayers whose removal is

cheapest, chosen depth-separated (measured to matter).

1. The three FFNs were removed; un-healed, that fails the bar (KLD

0.10021).

1. One FFN per cut (the first surviving layer below it) was retrained

briefly against the frozen parent: 7.4 M tokens, ~28 minutes on one

24 GB GPU slice. Healed BF16: 0.03056, better than the parent with

only two un-healed cuts.

1. Quantization noise and surgery error were measured to be additive

and independent (0.965 to 0.994 times the naive sum across four

configurations), so the Q4_K cost was PREDICTED before the file was

built: predicted 0.0484, measured 0.04810.

1. The GGUF was produced by byte-level substitution of the 18 modified

tensors into the parent GGUF, then quantized by xyntetik-runner's

own quantizer. The BF16 path was validated exactly (KLD 0.00000

against the source checkpoint).

Notes and limits

• The 9 removed-FFN tensors are stored as exact zeros in standard

shape, so any runtime that runs the parent GGUF can run this file

with no loader changes. The zeros still occupy their quantized bytes;

true tensor removal would shave a further ~5%.

• Tested in xyntetik-runner

(native muse-glimmer support). Other GGUF runtimes should work if

they run the parent model at Q4_K; that is expected, not verified

here.

• The fidelity bar is parent-agreement on a held-out split (45,056

positions), not a capability benchmark. The full method write-up, including the depth-absorbability law and the quantization-independence result: Muse-Glimmer-30B-surgery-report.

• Healing used 135k unique training tokens (a deliberately small

budget); the numbers above are lower bounds on what the method

yields with more data.

Base model: Muse-Glimmer-30B by Meta, Apache-2.0. This derivative

keeps the license and changes 18 tensors (9 zeroed, 9 replaced with healed weights) as described.

Decontamination re-score (2026-08-29)

A window-level audit of the healing corpus found that 283 of the eval

split's 43,855 windows (0.65%, one domain) had leaked into the

training pool through near-duplicate source documents. This affected

every healed artifact's published number, so all of them were

re-scored on the decontaminated subset.

**This artifact passes on clean data: KLD 0.04903 (bar 0.05),

margin-qualified top-1 97.70% (bar 97%).** The published full-split

numbers reproduce from the file on disk to five decimals.

Stated plainly, because thin margins should be said and not

discovered: this artifact's clean-data headroom under the KLD bar is

0.00097, which is 1.37 times the measurement procedure's noise floor

(plus or minus 0.00071), and its original build gate was cleared by

0.00014. It is a genuine, exactly reproducible pass with thin margins

on every axis we have examined. If your use case needs comfortable

headroom rather than a demonstrated boundary case, use the

Q8_0 sibling,

whose headroom is roughly 15 times the noise floor.

A control decomposition isolated the leak's effect: healed models

score better on positions whose content appeared in healing data, by

+0.0012 to +0.0014 KLD across three independent artifacts. Small,

real, and measured; the naive before-and-after delta would have

hidden it behind an opposing domain-mix shift. Full detail will

accompany the method write-up's next revision.

Integrity

sha256(Muse-Glimmer-30B-surgical-4p75-Q4_K.gguf) =
103aa117dfce88974bd78e03ee6ca6601c720f01a1ec24ee1cfd0e5fe2f77ce6

SURGERY.json in this repo records the byte-substitution provenance:

the parent revision, which tensors were zeroed and which were replaced

with healed weights.