---

license: other

license_name: glm-5.3

license_link: https://huggingface.co/zai-org/GLM-5.3/blob/main/LICENSE

base_model:

• cloudyu/GLM-5.3-SLIM-E192
• zai-org/GLM-5.3

language:

• en
• zh

pipeline_tag: text-generation

tags:

• gguf
• moe
• expert-pruning
• iq2_xxs
• glm
• dwarfstar
• ds4

---

GLM-5.3-SLIM-E192 — IQ2_XXS GGUF: GLM 5.3 on a single machine

GLM 5.3, the 744B-parameter frontier MoE, running on one GPU or one Mac.

Expert pruning (192 of 256 routed experts, "SLIM") took 25 % off the model;

DwarfStar's 2-bit routed-expert recipe took the rest. The result is a

149.7 GiB file that stays fully resident on a single 180 GB GPU (B200,

21 tokens/s measured) or a single Mac Studio (256 GB; 192 GB with a small

context), and streams from SSD on a 128 GB Mac — hardware where the unpruned

model needs a multi-GPU node or does not fit at all.

| GLM 5.3 form | Size | What it takes to run it |

|---|---:|---|

| Original FP8 (zai-org/GLM-5.3) | 756 GB | 8× H200/B200 or 4× B300, tensor parallel (88 GiB/GPU at TP=8: too big for 80 GB cards) |

| Pruned FP8 (cloudyu/GLM-5.3-SLIM-E192) | 564 GB | 4× B200/B300 or 8× 80 GB cards |

| Full GLM 5.3 IQ2_XXS GGUF (antirez/glm-5.3-gguf) | 197 GiB | 256 GB+ Mac resident; 128 GB Mac via SSD streaming; does not fit one 180 GB GPU |

| This file — SLIM IQ2_XXS GGUF | 149.7 GiB | one 180 GB GPU resident (21 t/s); one Mac Studio resident (256 GB comfortably, 192 GB with a small context); 128 GB Mac via SSD streaming |

The pruning is the enabler: at 2 bits the unpruned experts alone are 187 GB,

so no single-device quantization of the original could fit a 180 GB card with

room for a context. Pruning removes 47 GB of expert bytes at this precision

and, per its author's A/B on the FP8 checkpoints, costs nothing measurable on

coding, cybersecurity, tool calling and math (GPQA −3.6 pt, C-Eval −7.3 pt).

What the file is: GLM-5.3-SLIM-E192 (attention with MLA + DSA sparse indexer,

shared experts, router, tokenizer and chat template identical to GLM 5.3; no

MTP head) with routed experts in IQ2_XXS (2.06 bits/weight) and everything

else in Q8_0 — the same recipe DwarfStar publishes for the full GLM 5.3, so

per-token compute and memory traffic are unchanged; only the footprint drops.

It runs with the **ds4-glm-slim

fork** of DwarfStar (a few changes on top of upstream antirez/ds4, see

Requirements). It is not a llama.cpp GGUF: the tensor layout,

quant mix and metadata follow DwarfStar's GLM-DSA format.

At a glance

| | |

|---|---|

| File | GLM-5.3-SLIM-E192-IQ2_XXS.gguf |

| Size | 160,760,301,792 bytes (149.7 GiB) |

| GGUF | v3, architecture glm-dsa, 1782 tensors |

| Parameters | ≈563 B total, ≈40 B active per token (8 of 192 routed experts + 1 shared) |

| Layers | 78 (3 dense + 75 MoE), no MTP block |

| Context | 1,048,576 positions in metadata; use what your memory allows |

| Routed experts | IQ2_XXS, 2.0625 bits/weight, weight-energy importance (no imatrix) |

| Everything else | Q8_0 (attention, shared experts, dense FFN, embeddings, output head); F32 norms/routers/indexer projections |

| Runtime | DwarfStar fork ds4-glm-slim (Metal, CUDA; ROCm untested) |

| Fits | one 180 GB GPU (B200/GB200) resident; one Mac Studio 256 GB resident, 192 GB resident with a small context; 128 GB Mac via SSD streaming |

| Source checkpoint | cloudyu/GLM-5.3-SLIM-E192 (FP8), revision e45b62eb |

| License | GLM-5.3 (same as the base model) |

What is in the file

| Role | Tensors | Type | Bytes |

|---|---|---:|---:|

| Routed experts gate / up / down (75 layers × 192 experts) | 225 | IQ2_XXS | 140.14 GB |

| Attention (MLA q_a/q_b/kv_a/kv_b/o, DSA indexer q_b/k) | 78 layers | Q8_0 | 14.50 GB |

| Shared experts gate / up / down | 75 layers | Q8_0 | 3.01 GB |

| Dense FFN (layers 0–2) | 3 layers | Q8_0 | 0.72 GB |

| Token embedding, output head | 2 | Q8_0 | 2.02 GB |

| Norms, routers, bias, indexer weights_proj | — | F32 | 0.42 GB |

Per-token decode reads the same amount of data as the full GLM 5.3 Q2 (8

routed experts per layer either way), so **speed per token is the same as the

197 GiB file; the gain is memory**.

Requirements

Upstream antirez/ds4 cannot load this file: its GLM-DSA loader pins the

shape to the official checkpoint (256 experts, 79 blocks, 1 MTP block) and its

CUDA backend has no kernels for IQ2_XXS down projections. The fork adds:

• runtime acceptance of expert-pruned GLM 5.2/5.3 shapes (expert count, block

count and MTP count read from the GGUF);

• CUDA kernels for all-IQ2_XXS routed layers (prefill on the mmq tier, decode

on mmvq vector kernels) and resident weights on discrete GPUs;

• a CUDA quantizer for IQ2_XXS (byte-identical to the C one) and the tooling

that produced this file.

Everything else — CLI, agent, HTTP server, KV snapshots, tool calling,

thinking control — is unchanged DwarfStar. Build it:

git clone https://github.com/yuhai-china/ds4-glm-slim.git ds4
cd ds4
make                # Apple Silicon / Metal
make cuda-generic   # NVIDIA, local GPU architecture (needs nvcc + cuBLAS)

Startup prints ds4: GLM DSA variant: 192 routed experts, 78 blocks, 0 MTP block(s)

when the fork recognises the file.

Quick start

Put the file in gguf/ inside the repository (or pass a full path with -m).

# interactive chat (thinking on by default; --nothink for direct answers)
./ds4 -m gguf/GLM-5.3-SLIM-E192-IQ2_XXS.gguf --ctx 32768

# OpenAI-compatible server on http://127.0.0.1:8000
./ds4-server -m gguf/GLM-5.3-SLIM-E192-IQ2_XXS.gguf --ctx 65536

# native coding agent
./ds4-agent -m gguf/GLM-5.3-SLIM-E192-IQ2_XXS.gguf --ctx 65536

Add --cuda on NVIDIA hosts. On a 128 GB Mac add --ssd-streaming. See

RUNNING.md for memory planning, streaming, server usage and

troubleshooting.

Quality

The pruned FP8 base (model card)

was measured against GLM-5.3 by its author: coding, cybersecurity, tool

calling and math within run-to-run noise; GPQA-Diamond −3.6 pt; C-Eval −7.3 pt

(the deliberate trade-off). The 2-bit routed experts of this file add their

own loss on top. Held-out perplexity of the FP8 base vs. the original: code

+1.0 %, English chat +4.7 %, Chinese +7.3 %.

This GGUF, ds4-eval (DwarfStar's built-in harness: GPQA Diamond, SuperGPQA,

AIME 2025; thinking on, default budgets), CUDA B200:

| Suite | Result |

|---|---|

| core (92 cases) | in progress: 8 / 8 correct so far (this table is updated when the run finishes) |

| probe (first 3 cases) | 3 / 3 |

ds4-eval scores are integration checks, not leaderboard numbers; compare

against the published full GLM 5.3 Q2 run on the same machine and suite.

An imatrix-guided variant (importance collected from this model's own routed

activations with ds4 --imatrix-dataset) is the natural next step and would

replace this file's weight-energy importance.

Field notes: security-domain probe on Metal (2026-09-17, this file)

A 6-question offensive/defensive security probe (SQL-injection methodology, C

code audit, ROP vs. Canary/NX/ASLR, AD lateral movement with blue-team

detection points, Python tooling, RSA broadcast attack) plus a

Chinese/English A/B on the same questions, run on the M2 Ultra resident setup

below. Single run per question, thinking on — enough to map the failure mode,

not a benchmark:

• Reasoning and structure survive the quant. The C audit found all five

planted bugs including the two subtle logic flaws; the RSA answer's math

(Håstad broadcast via CRT + integer cube root) and its reference code were

correct; defense playbooks were coherent in both languages.

• **Long-tail precise facts degrade — measurably more in Chinese than in

English.** The Chinese answers confabulated Windows event IDs, DRSUAPI

UUIDs and NTSTATUS codes; the English answers got the same facts right

(events 4624/4769/4776, etype 0x17, MITRE T1550.002/T1558.003) yet still

mangled tool names (krb5tgc for krb5tgs) and invented one detection

mechanism. **Verify any concrete ID, command or parameter before use, in

either language.** This matches the pruning A/B above (Chinese +7.3 %

perplexity vs. English +4.7 %, C-Eval as the largest score drop).

• Capacity beats precision on hard coding. On the same 13-task suite,

this 2-bit/563B build scored 10/13 vs 9/13 for a 4-bit MLX build of a much

smaller sibling model (~4× faster per task). The extra wins were the

hardest items (persistent segment tree, optimizer comparison) — evidence

that expert pruning + 2-bit costs long-tail facts (see above) far more

than code reasoning.

Field notes: cybersecurity probe, English (2026-09-17, this file)

Eight expert-level offensive/defensive questions (Kerberoasting detection

with event IDs and etype values, x86-64 gadget opcodes, DCSync IOCs, Flask

code audit, CBC bit-flipping/padding-oracle math, DNS-tunneling detection

design, Windows event-log chain reconstruction, CET/CFI/PAC bypass theory),

greedy-free sampling (temperature 1.0, top_p 0.95, min_p 0) via the

OpenAI-compatible server. Grades against ground truth:

| Task | Grade | Failure mode |

|---|---|---|

| Kerberoasting detection (4769, etype 0x17/0x11/0x12, PBKDF2-HMAC-SHA1) | A− | correct on all core facts; hedged where unsure |

| x86-64 gadget bytes + syscall numbers | A− | 9/10 correct; mis-encoded syscall as 0f 34 |

| DCSync IOCs (UUID / method / event IDs) | F | confabulated UUID, method name, event IDs — worse with self-"correction" |

| Flask audit (cmd-injection, traversal, pickle RCE, JWT none) | A− | all four found; working fixes; minor code garble |

| CBC bit-flip + padding oracle | A | byte-exact arithmetic, two-pass self-audit |

| DNS tunneling detection + rate math | B+ | sound engineering; RFC numbers hallucinated |

| Windows log chain triage | A− | full chain, PID vs event-ID trap avoided |

| CET / CFI / PAC bypass theory | B+ | concepts 5/5; MSR/mnemonic details garbled but honestly flagged |

• Two failure modes, unpredictable. On precision identifiers (UUIDs,

GUIDs, RFC numbers, MSR names, opcodes) the model sometimes confabulates

confidently (DCSync answer: invented interface UUID, invented RPC method

name) and sometimes explicitly refuses to guess and flags for

verification (the CET answer). Which mode appears is not predictable.

• Sampling matters more here than anywhere else. Greedy (temp 0) drove

the thinking trace into an endlessly repeating loop that consumed the

entire budget six times in eight runs, delivering nothing. Recommended

sampling for this build: temperature 1.0, top_p 0.95, min_p 0 — i.e.

the upstream GLM defaults with min-p disabled. With thinking on, budget

8–13k tokens; with think:false, tasks complete 3–5× faster and code

quality holds (10/13 on the coding benchmark).

• What is stable vs what is not. Reasoning chains, exploit-construction

logic, code, and defense architecture are reproducible across runs.

Long-tail identifiers (UUIDs, GUIDs, event names, opcodes, RFC numbers)

fail stochastically — different wrong values each run, ~1 in 10 facts per

identifier-dense answer. Treat every concrete identifier in its output as

unverified, in any language; prose, code, and arithmetic survive the quant.

• English thinking traces run 3–5× longer than Chinese ones (5–7k tokens

observed). With thinking on, budget ≥8–13k generation tokens for English

deep-dive prompts (--ctx 16384 -n 13000 worked) — a 3k-token cap was

consumed entirely by the reasoning trace twice, delivering no answer.

Field notes: code-execution benchmark on Metal (2026-09-17, this file)

A 13-task suite (number theory, combinatorics, segmented sieve; algorithm /

systems / data-structure / ML / graph coding; logic puzzles; physics and

statistics simulations) with automatic code execution, grading against known

answers, and one retry on runtime errors. ds4-server resident, thinking off,

temperature 1.0, top-p 1.0, ctx 16384. Single run per task; sample is small —

treat as a capability map, not a leaderboard:

• 10 / 13 PASS, keyword coverage 98 %. All three number-theory tasks

passed, including a segmented sieve to 10^12 and an O(n) totient sieve.

Harder coding passed: thread-safe sharded LRU with TTL under an 8-thread

stress test, a persistent segment tree with versioned queries, numpy-only

multi-head attention with RoPE.

• Failures are systematic, not flaky. One long task (Dijkstra + A* +

benchmarking harness) exhausted an 8k-token cap three times and needs ~16k;

one physics simulation mis-derived the potential-energy term (wrong total

energy from the start — a knowledge gap, not instability); one MCMC task

failed once and passed on a re-roll.

• Stability over a 4-hour server session (1,253 speed samples): decode

11.6–14.4 t/s, median 13.15, σ/median = 3.0 %. Zero crashes, zero restarts.

• Capacity beats precision on hard coding. On the same 13-task suite,

this 2-bit/563B build scored 10/13 vs 9/13 for a 4-bit MLX build of a much

smaller sibling model (~4× faster per task). The extra wins were the

hardest items (persistent segment tree, optimizer comparison) — evidence

that expert pruning + 2-bit costs long-tail facts (see above) far more

than code reasoning.

Per-task results (thinking off, temp 1.0; gen = generation wall time):

| Task | Category | Diff | Verdict | Gen | Note |

|---|---|---|---|---:|---|

| Euler totient Σ, last 6 digits | Math · Number Theory | 3/5 | PASS | 67 s | linear sieve, self-checks |

| Lattice paths vs anti-diagonal | Math · Combinatorics | 4/5 | PASS | 298 s | DP, n=20 exact |

| Segmented sieve [10¹², 10¹²+10⁶] | Math · Prime | 2/5 | PASS | 131 s | 36,249 primes, <60 s |

| Median of two sorted arrays | Coding · Algorithm | 3/5 | PASS | 120 s | O(log n), 6891× speedup measured |

| Thread-safe LRU + TTL | Coding · Systems | 4/5 | PASS | 448 s | 8-thread stress, 100k ops/thread |

| Persistent segment tree | Coding · Data Structures | 4/5 | PASS | 516 s | versioned k-th smallest, all queries correct |

| Multi-head attention + RoPE | Coding · ML | 4/5 | PASS | 178 s | numpy-only, causal mask verified |

| Dijkstra vs A*, 5k nodes | Coding · Graph | 3/5 | FAIL | 83 s | ran out of 8k budget once; re-rolls output incomplete code — needs ≥16k |

| Knights & Knaves solver | Logic · SAT | 3/5 | PASS | 155 s | exhaustive, correct count |

| Verify 3 math claims | Logic · Proof | 4/5 | PASS | 60 s | found both counterexamples (n=4; p=3) |

| Figure-8 three-body orbit | Science · Simulation | 3/5 | FAIL | 120 s | wrong PE term → E₀ off 2.3× (physics gap) |

| MH vs HMC + ESS | Science · Statistics | 4/5 | PASS | 238 s | mean/cov within tolerance |

| 4 optimizers on Rosenbrock | Science · Optimization | 3/5 | PASS | 150 s | GD/Momentum/Adam/L-BFGS-B |

Speed

| Machine | Backend | Prefill | Decode |

|---|---|---:|---:|

| 1× NVIDIA B200 180 GB, model resident | CUDA | 44 t/s on a 30-token prompt (fixed cost dominated; mmq tier for long prompts) | 21 t/s |

| Mac Studio M2 Ultra 192 GB, model resident, --ctx 8192 | Metal | 25–55 t/s on 22–157-token prompts (fixed cost dominated) | 12.3–13.3 t/s |

Mac runbook (192 GB, measured 2026-09-17, ds4 Metal build): quit heavy apps,

raise the Metal wired limit to 180 GiB (sudo sysctl iogpu.wired_limit_mb=184320;

the default 160 GiB fits the plan but leaves no headroom), then

DS4_GLM_MEMORY_GUARD_RESERVE_GB=16 ./ds4 -m GLM-5.3-SLIM-E192-IQ2_XXS.gguf --ctx 8192 -p "…"

Startup reports `KV 0.73 GiB + buffers 4.07 GiB + resident model 149.71 GiB =

154.50 GiB planned`; generation ran with zero swap used. Decode holds at

~13 t/s regardless of prompt. Short-prompt prefill is dominated by fixed

per-layer cost, so the per-token prefill rate rises with prompt length

(25 t/s at 22 tokens → 55 t/s at 157 tokens). With the file in the page cache,

model mapping takes ~2 s.

How it was built

python3 gguf-tools/glm53_full_quantize.py \
  --hf GLM-5.3-SLIM-E192 \
  --tokenizer-template GLM-5.3-UD-IQ2_XXS_RoutedIQ2XXS_blk78Q2K.gguf \
  --model-name GLM-5.3-SLIM-E192 \
  --repo-url https://huggingface.co/cloudyu/GLM-5.3-SLIM-E192 \
  --cuda --cuda-batch 32 \
  --out GLM-5.3-SLIM-E192-IQ2_XXS.gguf

• Source: the FP8 (block 128) safetensors of cloudyu/GLM-5.3-SLIM-E192,

dequantized exactly; tokenizer/chat template taken from the checkpoint,

validated against the published full GLM 5.3 GGUF.

• Routed experts quantized on a B200 with the fork's PyTorch/CUDA port of

DwarfStar's IQ2_XXS quantizer (gguf-tools/iq2xxs_cuda.py), which is

byte-identical to the C implementation for the same importance vector

(verified on 296,848 real blocks). 43,200 expert matrices in 29 minutes.

• Importance: DwarfStar's fallback importance[column] = Σ row[column]²

(no activation imatrix).

• Attention, shared experts, dense FFN, embeddings and output head: Q8_0 via

the C quantizer, as in the published GLM 5.3 Q2.

Limitations

• 2-bit routed experts. Expect a measurable drop versus the FP8 checkpoint

on knowledge-heavy and Chinese-exam tasks; coding and agent use are the

intended workloads. Measured on this file (see the Quality field notes):

reasoning and code survive; **long-tail precise facts (event IDs, UUIDs,

tool names, opcodes) are the first casualty**, more so in Chinese than in

English — treat concrete identifiers in its output as unverified.

• No MTP head. Do not pass --mtp.
• No imatrix in this build (see Quality).
• 192 GB Macs are borderline: 149.7 GiB weights + ~5 GiB graph leaves

little for macOS; DwarfStar's memory guard will ask for

DS4_GLM_MEMORY_GUARD_RESERVE_GB to be lowered and the context kept small.

Now measured on an M2 Ultra 192 GB (Metal, resident, --ctx 8192, wired

limit raised to 180 GiB): 12.3–13.3 t/s decode with zero swap — see

Speed. Larger contexts on 192 GB remain untested.

• SSD streaming starts cold on pruned variants (the built-in GLM 5.2 hot

seed uses the original expert numbering and is skipped).

• CUDA needs a single GPU with ≥ 158 GiB free (weights + 8 GiB) for the

resident fast path; smaller cards fall back to a host mapping that streams

experts over PCIe at ~1 t/s. Multi-GPU placement is untested with this file.

• general.source.revision in the GGUF metadata carries the quantizer's

default (the official GLM-5.3 revision); the SLIM checkpoint revision used

is e45b62eb3f5a22232f1e4980da255266ab933f31.

Verify the download

size    160760301792 bytes
sha256  68abcb6effe4e7a4379d92f68607dfbd1aca0a1fad5dabda78a96f1a4a0e4725

License and credits

• Weights: GLM-5.3 License (Z.AI), including the Model-as-a-Service clause;

this file is a derivative of zai-org/GLM-5.3 via cloudyu/GLM-5.3-SLIM-E192.

• Expert pruning: cloudyu (GLM-5.3-SLIM-E192).
• Inference engine and quant recipe: Salvatore Sanfilippo and the DwarfStar

contributors (antirez/ds4), building on llama.cpp / GGML.

• Fork, CUDA quantizer and this build: yuhai-china — https://github.com/yuhai-china/ds4-glm-slim.