What license applies to L-Alchemyst/GLM-5.3-GGUF?

License: other. Verify terms on Hugging Face before commercial use.

How do I run L-Alchemyst/GLM-5.3-GGUF locally?

Download a GGUF file from this page and load it in guIDE or llama.cpp. Pipeline task: text-generation.

Model Intelligence Sheet

L-Alchemyst/GLM-5.3-GGUF overview

Q: How much VRAM or disk space does L-Alchemyst/GLM-5.3-GGUF need?

Runs locally from ~3.02 GB disk (4 GB VRAM class GPUs with llama.cpp / guIDE).

ggufbase_model:zai-org/GLM-5.3-BF16base_model:quantized:zai-org/GLM-5.3-BF16license:otherendpoints_compatibleregion:usimatrixconversational

Runs locally from ~3.02 GB disk (4 GB VRAM class GPUs with llama.cpp / guIDE).

Downloads

9,660

Likes

Pipeline

—

Author

L-Alchemyst

Repository Files & Downloads

112 GGUF files detected

Direct downloads for local inference

File	Type	Quantization	Size	Link
IQ2_K/GLM-5.3-IQ2_K-00001-of-00027.gguf	GGUF	IQ2_K	9.32 GB	Download
IQ2_K/GLM-5.3-IQ2_K-00002-of-00027.gguf	GGUF	IQ2_K	9.06 GB	Download
IQ2_K/GLM-5.3-IQ2_K-00003-of-00027.gguf	GGUF	IQ2_K	9.05 GB	Download
IQ2_K/GLM-5.3-IQ2_K-00004-of-00027.gguf	GGUF	IQ2_K	8.16 GB	Download
IQ2_K/GLM-5.3-IQ2_K-00005-of-00027.gguf	GGUF	IQ2_K	9.06 GB	Download
IQ2_K/GLM-5.3-IQ2_K-00006-of-00027.gguf	GGUF	IQ2_K	9.06 GB	Download
IQ2_K/GLM-5.3-IQ2_K-00007-of-00027.gguf	GGUF	IQ2_K	9.05 GB	Download
IQ2_K/GLM-5.3-IQ2_K-00008-of-00027.gguf	GGUF	IQ2_K	9.06 GB	Download
IQ2_K/GLM-5.3-IQ2_K-00009-of-00027.gguf	GGUF	IQ2_K	9.06 GB	Download
IQ2_K/GLM-5.3-IQ2_K-00010-of-00027.gguf	GGUF	IQ2_K	9.05 GB	Download
IQ2_K/GLM-5.3-IQ2_K-00011-of-00027.gguf	GGUF	IQ2_K	9.06 GB	Download
IQ2_K/GLM-5.3-IQ2_K-00012-of-00027.gguf	GGUF	IQ2_K	9.06 GB	Download
IQ2_K/GLM-5.3-IQ2_K-00013-of-00027.gguf	GGUF	IQ2_K	9.05 GB	Download
IQ2_K/GLM-5.3-IQ2_K-00014-of-00027.gguf	GGUF	IQ2_K	9.06 GB	Download
IQ2_K/GLM-5.3-IQ2_K-00015-of-00027.gguf	GGUF	IQ2_K	9.06 GB	Download
IQ2_K/GLM-5.3-IQ2_K-00016-of-00027.gguf	GGUF	IQ2_K	9.05 GB	Download
IQ2_K/GLM-5.3-IQ2_K-00017-of-00027.gguf	GGUF	IQ2_K	9.06 GB	Download
IQ2_K/GLM-5.3-IQ2_K-00018-of-00027.gguf	GGUF	IQ2_K	9.06 GB	Download
IQ2_K/GLM-5.3-IQ2_K-00019-of-00027.gguf	GGUF	IQ2_K	9.06 GB	Download
IQ2_K/GLM-5.3-IQ2_K-00020-of-00027.gguf	GGUF	IQ2_K	9.06 GB	Download
IQ2_K/GLM-5.3-IQ2_K-00021-of-00027.gguf	GGUF	IQ2_K	9.06 GB	Download
IQ2_K/GLM-5.3-IQ2_K-00022-of-00027.gguf	GGUF	IQ2_K	9.05 GB	Download
IQ2_K/GLM-5.3-IQ2_K-00023-of-00027.gguf	GGUF	IQ2_K	9.06 GB	Download
IQ2_K/GLM-5.3-IQ2_K-00024-of-00027.gguf	GGUF	IQ2_K	9.06 GB	Download
IQ2_K/GLM-5.3-IQ2_K-00025-of-00027.gguf	GGUF	IQ2_K	8.53 GB	Download
IQ2_K/GLM-5.3-IQ2_K-00026-of-00027.gguf	GGUF	IQ2_K	8.12 GB	Download
IQ2_K/GLM-5.3-IQ2_K-00027-of-00027.gguf	GGUF	IQ2_K	3.02 GB	Download
IQ3_K/GLM-5.3-IQ3_K-00001-of-00039.gguf	GGUF	IQ3_K	8.89 GB	Download
IQ3_K/GLM-5.3-IQ3_K-00002-of-00039.gguf	GGUF	IQ3_K	8.32 GB	Download
IQ3_K/GLM-5.3-IQ3_K-00003-of-00039.gguf	GGUF	IQ3_K	8.32 GB	Download
IQ3_K/GLM-5.3-IQ3_K-00004-of-00039.gguf	GGUF	IQ3_K	8.32 GB	Download
IQ3_K/GLM-5.3-IQ3_K-00005-of-00039.gguf	GGUF	IQ3_K	8.32 GB	Download
IQ3_K/GLM-5.3-IQ3_K-00006-of-00039.gguf	GGUF	IQ3_K	8.61 GB	Download
IQ3_K/GLM-5.3-IQ3_K-00007-of-00039.gguf	GGUF	IQ3_K	8.32 GB	Download
IQ3_K/GLM-5.3-IQ3_K-00008-of-00039.gguf	GGUF	IQ3_K	8.32 GB	Download
IQ3_K/GLM-5.3-IQ3_K-00009-of-00039.gguf	GGUF	IQ3_K	8.32 GB	Download
IQ3_K/GLM-5.3-IQ3_K-00010-of-00039.gguf	GGUF	IQ3_K	8.32 GB	Download
IQ3_K/GLM-5.3-IQ3_K-00011-of-00039.gguf	GGUF	IQ3_K	8.32 GB	Download
IQ3_K/GLM-5.3-IQ3_K-00012-of-00039.gguf	GGUF	IQ3_K	8.32 GB	Download
IQ3_K/GLM-5.3-IQ3_K-00013-of-00039.gguf	GGUF	IQ3_K	8.32 GB	Download
IQ3_K/GLM-5.3-IQ3_K-00014-of-00039.gguf	GGUF	IQ3_K	8.32 GB	Download
IQ3_K/GLM-5.3-IQ3_K-00015-of-00039.gguf	GGUF	IQ3_K	8.32 GB	Download
IQ3_K/GLM-5.3-IQ3_K-00016-of-00039.gguf	GGUF	IQ3_K	8.32 GB	Download
IQ3_K/GLM-5.3-IQ3_K-00017-of-00039.gguf	GGUF	IQ3_K	8.32 GB	Download
IQ3_K/GLM-5.3-IQ3_K-00018-of-00039.gguf	GGUF	IQ3_K	8.32 GB	Download
IQ3_K/GLM-5.3-IQ3_K-00019-of-00039.gguf	GGUF	IQ3_K	8.32 GB	Download
IQ3_K/GLM-5.3-IQ3_K-00020-of-00039.gguf	GGUF	IQ3_K	8.32 GB	Download
IQ3_K/GLM-5.3-IQ3_K-00021-of-00039.gguf	GGUF	IQ3_K	8.32 GB	Download
IQ3_K/GLM-5.3-IQ3_K-00022-of-00039.gguf	GGUF	IQ3_K	8.32 GB	Download
IQ3_K/GLM-5.3-IQ3_K-00023-of-00039.gguf	GGUF	IQ3_K	8.32 GB	Download
IQ3_K/GLM-5.3-IQ3_K-00024-of-00039.gguf	GGUF	IQ3_K	8.32 GB	Download
IQ3_K/GLM-5.3-IQ3_K-00025-of-00039.gguf	GGUF	IQ3_K	8.32 GB	Download
IQ3_K/GLM-5.3-IQ3_K-00026-of-00039.gguf	GGUF	IQ3_K	8.32 GB	Download
IQ3_K/GLM-5.3-IQ3_K-00027-of-00039.gguf	GGUF	IQ3_K	8.32 GB	Download
IQ3_K/GLM-5.3-IQ3_K-00028-of-00039.gguf	GGUF	IQ3_K	8.32 GB	Download
IQ3_K/GLM-5.3-IQ3_K-00029-of-00039.gguf	GGUF	IQ3_K	8.32 GB	Download
IQ3_K/GLM-5.3-IQ3_K-00030-of-00039.gguf	GGUF	IQ3_K	8.32 GB	Download
IQ3_K/GLM-5.3-IQ3_K-00031-of-00039.gguf	GGUF	IQ3_K	8.32 GB	Download
IQ3_K/GLM-5.3-IQ3_K-00032-of-00039.gguf	GGUF	IQ3_K	8.32 GB	Download
IQ3_K/GLM-5.3-IQ3_K-00033-of-00039.gguf	GGUF	IQ3_K	8.32 GB	Download
IQ3_K/GLM-5.3-IQ3_K-00034-of-00039.gguf	GGUF	IQ3_K	8.32 GB	Download
IQ3_K/GLM-5.3-IQ3_K-00035-of-00039.gguf	GGUF	IQ3_K	8.32 GB	Download
IQ3_K/GLM-5.3-IQ3_K-00036-of-00039.gguf	GGUF	IQ3_K	8.32 GB	Download
IQ3_K/GLM-5.3-IQ3_K-00037-of-00039.gguf	GGUF	IQ3_K	8.02 GB	Download
IQ3_K/GLM-5.3-IQ3_K-00038-of-00039.gguf	GGUF	IQ3_K	9.25 GB	Download
IQ3_K/GLM-5.3-IQ3_K-00039-of-00039.gguf	GGUF	IQ3_K	4.16 GB	Download
IQ4_K/GLM-5.3-IQ4_K-00001-of-00046.gguf	GGUF	IQ4_K	8.72 GB	Download
IQ4_K/GLM-5.3-IQ4_K-00002-of-00046.gguf	GGUF	IQ4_K	8.65 GB	Download
IQ4_K/GLM-5.3-IQ4_K-00003-of-00046.gguf	GGUF	IQ4_K	8.64 GB	Download
IQ4_K/GLM-5.3-IQ4_K-00004-of-00046.gguf	GGUF	IQ4_K	8.84 GB	Download
IQ4_K/GLM-5.3-IQ4_K-00005-of-00046.gguf	GGUF	IQ4_K	8.83 GB	Download
IQ4_K/GLM-5.3-IQ4_K-00006-of-00046.gguf	GGUF	IQ4_K	8.64 GB	Download
IQ4_K/GLM-5.3-IQ4_K-00007-of-00046.gguf	GGUF	IQ4_K	9.16 GB	Download
IQ4_K/GLM-5.3-IQ4_K-00008-of-00046.gguf	GGUF	IQ4_K	8.84 GB	Download
IQ4_K/GLM-5.3-IQ4_K-00009-of-00046.gguf	GGUF	IQ4_K	8.64 GB	Download
IQ4_K/GLM-5.3-IQ4_K-00010-of-00046.gguf	GGUF	IQ4_K	8.83 GB	Download
IQ4_K/GLM-5.3-IQ4_K-00011-of-00046.gguf	GGUF	IQ4_K	8.84 GB	Download
IQ4_K/GLM-5.3-IQ4_K-00012-of-00046.gguf	GGUF	IQ4_K	8.64 GB	Download
IQ4_K/GLM-5.3-IQ4_K-00013-of-00046.gguf	GGUF	IQ4_K	8.56 GB	Download
IQ4_K/GLM-5.3-IQ4_K-00014-of-00046.gguf	GGUF	IQ4_K	8.84 GB	Download
IQ4_K/GLM-5.3-IQ4_K-00015-of-00046.gguf	GGUF	IQ4_K	8.64 GB	Download
IQ4_K/GLM-5.3-IQ4_K-00016-of-00046.gguf	GGUF	IQ4_K	8.84 GB	Download
IQ4_K/GLM-5.3-IQ4_K-00017-of-00046.gguf	GGUF	IQ4_K	8.83 GB	Download
IQ4_K/GLM-5.3-IQ4_K-00018-of-00046.gguf	GGUF	IQ4_K	8.64 GB	Download
IQ4_K/GLM-5.3-IQ4_K-00019-of-00046.gguf	GGUF	IQ4_K	8.75 GB	Download
IQ4_K/GLM-5.3-IQ4_K-00020-of-00046.gguf	GGUF	IQ4_K	8.65 GB	Download
IQ4_K/GLM-5.3-IQ4_K-00021-of-00046.gguf	GGUF	IQ4_K	8.64 GB	Download
IQ4_K/GLM-5.3-IQ4_K-00022-of-00046.gguf	GGUF	IQ4_K	8.84 GB	Download
IQ4_K/GLM-5.3-IQ4_K-00023-of-00046.gguf	GGUF	IQ4_K	8.83 GB	Download
IQ4_K/GLM-5.3-IQ4_K-00024-of-00046.gguf	GGUF	IQ4_K	8.64 GB	Download
IQ4_K/GLM-5.3-IQ4_K-00025-of-00046.gguf	GGUF	IQ4_K	8.83 GB	Download
IQ4_K/GLM-5.3-IQ4_K-00026-of-00046.gguf	GGUF	IQ4_K	8.56 GB	Download
IQ4_K/GLM-5.3-IQ4_K-00027-of-00046.gguf	GGUF	IQ4_K	8.64 GB	Download
IQ4_K/GLM-5.3-IQ4_K-00028-of-00046.gguf	GGUF	IQ4_K	8.83 GB	Download
IQ4_K/GLM-5.3-IQ4_K-00029-of-00046.gguf	GGUF	IQ4_K	8.84 GB	Download
IQ4_K/GLM-5.3-IQ4_K-00030-of-00046.gguf	GGUF	IQ4_K	8.64 GB	Download
IQ4_K/GLM-5.3-IQ4_K-00031-of-00046.gguf	GGUF	IQ4_K	8.83 GB	Download
IQ4_K/GLM-5.3-IQ4_K-00032-of-00046.gguf	GGUF	IQ4_K	8.84 GB	Download
IQ4_K/GLM-5.3-IQ4_K-00033-of-00046.gguf	GGUF	IQ4_K	8.37 GB	Download
IQ4_K/GLM-5.3-IQ4_K-00034-of-00046.gguf	GGUF	IQ4_K	8.83 GB	Download
IQ4_K/GLM-5.3-IQ4_K-00035-of-00046.gguf	GGUF	IQ4_K	8.84 GB	Download
IQ4_K/GLM-5.3-IQ4_K-00036-of-00046.gguf	GGUF	IQ4_K	8.64 GB	Download
IQ4_K/GLM-5.3-IQ4_K-00037-of-00046.gguf	GGUF	IQ4_K	8.84 GB	Download
IQ4_K/GLM-5.3-IQ4_K-00038-of-00046.gguf	GGUF	IQ4_K	8.83 GB	Download
IQ4_K/GLM-5.3-IQ4_K-00039-of-00046.gguf	GGUF	IQ4_K	8.46 GB	Download
IQ4_K/GLM-5.3-IQ4_K-00040-of-00046.gguf	GGUF	IQ4_K	8.75 GB	Download
IQ4_K/GLM-5.3-IQ4_K-00041-of-00046.gguf	GGUF	IQ4_K	8.83 GB	Download
IQ4_K/GLM-5.3-IQ4_K-00042-of-00046.gguf	GGUF	IQ4_K	8.64 GB	Download
IQ4_K/GLM-5.3-IQ4_K-00043-of-00046.gguf	GGUF	IQ4_K	8.75 GB	Download
IQ4_K/GLM-5.3-IQ4_K-00044-of-00046.gguf	GGUF	IQ4_K	8.42 GB	Download
IQ4_K/GLM-5.3-IQ4_K-00045-of-00046.gguf	GGUF	IQ4_K	9.19 GB	Download
IQ4_K/GLM-5.3-IQ4_K-00046-of-00046.gguf	GGUF	IQ4_K	8.38 GB	Download

Model Details

Model ID	L-Alchemyst/GLM-5.3-GGUF
Author	L-Alchemyst
Pipeline	—
License	other
Base model	zai-org/GLM-5.3-BF16
Last modified	2026-09-07T01:37:27.000Z

Model README

---

base_model: zai-org/GLM-5.3-BF16

license: other

---

<style>

.qz-wrap {

background: #0d1117;

color: #e6edf3;

font-family: 'Inter', system-ui, -apple-system, sans-serif;

font-size: 1rem;

line-height: 1.7;

padding: 2rem 2.5rem;

border-radius: 8px;

margin-bottom: 2rem;

}

.qz-wrap * { color: #e6edf3; }

.qz-wrap a { color: #58a6ff; text-decoration: none; }

.qz-wrap a:hover { text-decoration: underline; }

.qz-header {

font-size: 2.2rem;

font-weight: 700;

color: #ffffff;

border-bottom: 3px solid #2e7d5e;

padding-bottom: 0.5rem;

margin-bottom: 1rem;

}

.qz-subtitle {

font-size: 1.05rem;

color: #8b949e;

margin-top: 0.2rem;

margin-bottom: 0.4rem;

}

.qz-section {

font-size: 1.3rem;

font-weight: 600;

color: #ffffff;

margin-top: 2rem;

margin-bottom: 0.6rem;

}

.qz-sub {

font-size: 1.1rem;

font-weight: 600;

color: #7ab8d4;

margin-top: 1.4rem;

margin-bottom: 0.5rem;

}

.qz-wrap table.qz-table {

display: table;

width: 100%;

min-width: 100%;

max-width: 100%;

table-layout: fixed;

border-collapse: collapse;

border-spacing: 0;

margin-bottom: 1rem;

}

.qz-wrap table.qz-table th,

.qz-wrap table.qz-table td {

padding: 0.55rem 1rem;

text-align: left;

border-bottom: 1px solid #30363d;

box-sizing: border-box;

}

.qz-wrap table.qz-table th:first-child,

.qz-wrap table.qz-table td:first-child {

width: 30%;

}

.qz-wrap table.qz-table th:last-child,

.qz-wrap table.qz-table td:last-child {

width: 70%;

}

.qz-wrap table.qz-table th {

background: rgba(46, 125, 94, 0.08);

color: #5eb88a;

font-weight: 700;

text-transform: uppercase;

font-size: 0.72rem;

letter-spacing: 0.06em;

}

.qz-wrap table.qz-table tr:nth-child(even) td {

background: rgba(255, 255, 255, 0.02);

}

.qz-note {

font-size: 0.88rem;

color: #8b949e;

font-style: italic;

}

.qz-status {

font-size: 0.9rem;

color: #d69e2e;

font-weight: 500;

padding: 0.6rem 1rem;

border: 1px solid #d69e2e;

border-radius: 4px;

display: inline-block;

margin-bottom: 1rem;

}

.qz-recipe-toggle {

margin: 0.8rem 0 1rem 0;

border: 1px solid #0891b2;

border-radius: 6px;

overflow: hidden;

width: 100%;

}

.qz-recipe-toggle summary {

font-size: 0.9rem;

font-weight: 600;

color: #22d3ee;

padding: 0.6rem 1rem;

cursor: pointer;

user-select: none;

background: rgba(34, 211, 238, 0.06);

list-style: none;

}

.qz-recipe-toggle summary::before { content: '▸ '; }

.qz-recipe-toggle[open] summary::before { content: '▾ '; }

.qz-recipe-toggle summary:hover { background: rgba(34, 211, 238, 0.12); }

.qz-recipe-body {

font-family: 'JetBrains Mono', 'Fira Code', monospace;

font-size: 0.8rem;

padding: 0.8rem 1rem;

line-height: 1.7;

background: rgba(34, 211, 238, 0.03);

color: #e6edf3;

}

.qz-recipe-body .rq-section {

color: #22d3ee;

font-weight: 700;

margin-top: 0.6rem;

}

.qz-recipe-body .rq-line { display: block; }

.qz-recipe-body .rq-quant {

color: #67e8f9;

font-weight: 600;

}

.qz-code {

font-family: 'JetBrains Mono', 'Fira Code', monospace;

font-size: 0.82rem;

display: block;

padding: 1rem;

margin: 0.8rem 0;

overflow-x: auto;

border-left: 3px solid #0891b2;

background: rgba(34, 211, 238, 0.04);

color: #e6edf3;

line-height: 1.7;

white-space: pre;

}

.qz-chart-container {

text-align: center;

margin: 0.5rem 0 1.5rem 0;

}

.qz-chart-container img {

border: 1px solid #30363d;

border-radius: 4px;

max-width: 100%;

height: auto;

}

.qz-credit {

font-size: 0.9rem;

color: #8b949e;

margin-top: 2rem;

padding-top: 1rem;

border-top: 1px solid #30363d;

}

.qz-credit a { color: #5eb88a; }

</style>

<html>

<head>

</head>

<body>

<div class="qz-subtitle">Imatrix quants best-in-class for <a href="https://github.com/ikawrakow/ik_llama.cpp">ik_llama.cpp</a></div>

<div class="qz-subtitle">NOTE: ik_llama.cpp is highly versatile and runs mainline llama.cpp quants such as existing GGUFs from bartowski, unsloth, mradermacher, Aes Sedai etc</div>

<div class="qz-status">IQ2_K done! ✅ IQ3_K done! ✅ IQ4_K done! ✅. All quants done unless someone really wants a Q1. See the collection below.</div>

<div class="qz-section">General Specs</div>

<tr><th></th><th>Value</th></tr>

<tr><td>Architecture</td><td>GLM-DSA (MoE, 256 experts × 22B, 8 active)</td></tr>

<tr><td>Params</td><td>744B total / 40B active</td></tr>

<tr><td>Context</td><td>1,048,576</td></tr>

<tr><td>Imatrix</td><td><a href="https://huggingface.co/AesSedai">Aes Sedai</a> — converted for ik_llama.cpp</td></tr>

<tr><td>Calibration</td><td>calibration_datav5.txt</td></tr>

</table>

<div class="qz-section">PPL Analysis</div>

</div>

<div class="qz-section">Quant Collection</div>

<div class="qz-note">Evaluated on wiki.test.raw, 512 ctx, 565 chunks, against BF16 reference logits.</div>

<summary>Why the specific layer targeting? (Analysis notes)</summary>

Rather than relying solely on conventional depth-based heuristics, the quantization targets were selected using imatrix activation-energy analysis and residuals after detrending for the model's natural increase in activation energy with depth.

The analysis identified blk.3–11 and blk.76–77 as unusually low-energy relative to their expected depth, making them candidates for more aggressive <code>iq4_ks</code> quantization. This is intentionally non-conventional. Since the measured sensitivity profile for these layers was weaker than expected.

In contrast, the mid-depth MoE layers, particularly around blk.29–40, show substantially higher residual energy and were retained at <code>iq4_k</code>. The expert down projections were also treated more conservatively because they feed directly back into the residual stream.

Detrended layer ranking (down output energy):

Trend slope: d(log E)/d(layer) = 0.1545. Residual = actual/trend; lower residual ⇒ weaker than depth predicts.

blk.3 resid=0.016 up_flow=2.331e+06

blk.4 resid=0.075 up_flow=3.914e+06

blk.5 resid=0.110 up_flow=4.457e+06

blk.6 resid=0.209 up_flow=7.723e+06

blk.7 resid=0.289 up_flow=1.349e+07

blk.76 resid=0.292 up_flow=5.089e+09

blk.10 resid=0.325 up_flow=2.200e+07

blk.9 resid=0.349 up_flow=2.073e+07

blk.77 resid=0.350 up_flow=2.786e+09

blk.11 resid=0.355 up_flow=2.457e+07

blk.75 resid=0.364 up_flow=6.132e+09

blk.67 resid=0.377 up_flow=4.028e+09

...

blk.36 resid=4.166 up_flow=9.199e+08

blk.32 resid=4.225 up_flow=6.227e+08

blk.34 resid=4.276 up_flow=7.851e+08

blk.33 resid=4.401 up_flow=7.199e+08

blk.35 resid=4.430 up_flow=8.652e+08</div>

The resulting recipe is therefore sensitivity-guided rather than position-guided: quantize the measured low-sensitivity outliers slightly more, while preserving precision through the high-energy middle of the network.

</div>

</details>

<tr><th>Metric</th><th>Value</th></tr>

<tr><td>PPL (quant)</td><td>2.707673 ± 0.014047</td></tr>

</table>

<summary>Quantization recipe</summary>

# Attention

blk\..*\.attn_k_b\.weight=q8_0

blk\..*\.attn_v_b\.weight=q8_0

blk\..*\.attn_kv_a_mqa\.weight=q8_0

blk\..*\.attn_q_a\.weight=q8_0

blk\..*\.attn_q_b\.weight=q8_0

blk\..*\.attn_output\.weight=q8_0

# Router

blk\..*\.ffn_gate_inp\.weight=f32

# First 3 Dense Layers

blk\..*\.ffn_down\.weight=iq6_k

blk\..*\.ffn_(gate|up)\.weight=iq5_ks

# Shared Expert Layers

blk\..*\.ffn_down_shexp\.weight=iq6_k

blk\..*\.ffn_(gate|up)_shexp\.weight=iq6_k

# Routed Experts

blk\.(78)\.ffn_down_exps\.weight=q6_K

blk\.(78)\.ffn_(gate|up)_exps\.weight=q6_K

blk\.(3|4|5|6|7|8|9|10|11|76|77)\.ffn_down_exps\.weight=iq4_ks

blk\.(3|4|5|6|7|8|9|10|11|76|77)\.ffn_(gate|up)_exps\.weight=iq4_ks

blk\..*\.ffn_down_exps\.weight=iq4_k

blk\..*\.ffn_(gate|up)_exps\.weight=iq4_k

# Indexer

blk\..*\.indexer\.attn_q_b\.weight=q6_K

blk\..*\.indexer\.attn_k\.weight=q8_0

blk\..*\.indexer\.proj\.weight=q8_0

blk\..*\.indexer\.k_norm\.(weight|bias)=f32

# MTP block - eh_proj is the throughput-critical tensor per ik's finding

blk\..*\.nextn\.eh_proj\.weight=q4_0

blk\..*\.nextn\.enorm\.weight=f32

blk\..*\.nextn\.hnorm\.weight=f32

blk\..*\.nextn\.shared_head_norm\.weight=f32

token_embd\.weight=iq6_k

output\.weight=iq6_k

</div>

</details>

<div class="qz-note">Evaluated on wiki.test.raw, 512 ctx, 565 chunks, against BF16 reference logits.</div>

<tr><th>Metric</th><th>Value</th></tr>

<tr><td>PPL (quant)</td><td>2.820603 ± 0.014842</td></tr>

</table>

<summary>Quantization recipe</summary>

# Attention

blk\..*\.attn_k_b\.weight=q8_0

blk\..*\.attn_v_b\.weight=q8_0

blk\..*\.attn_kv_a_mqa\.weight=q8_0

blk\..*\.attn_q_a\.weight=iq6_k

blk\..*\.attn_q_b\.weight=iq6_k

blk\..*\.attn_output\.weight=iq6_k

# Router

blk\..*\.ffn_gate_inp\.weight=f32

# First 3 Dense Layers

blk\..*\.ffn_down\.weight=iq6_k

blk\..*\.ffn_(gate|up)\.weight=iq5_ks

# Shared Expert Layers

blk\..*\.ffn_down_shexp\.weight=iq6_k

blk\..*\.ffn_(gate|up)_shexp\.weight=iq6_k

# Routed Experts

blk\.(78)\.ffn_down_exps\.weight=q6_K

blk\.(78)\.ffn_(gate|up)_exps\.weight=q6_K

blk\..*\.ffn_down_exps\.weight=iq4_ks

blk\..*\.ffn_(gate|up)_exps\.weight=iq3_ks

# Indexer

blk\..*\.indexer\.k_norm\.(weight|bias)=f32

blk\..*\.indexer\.proj\.weight=q8_0

blk\..*\.indexer\.attn_k\.weight=q8_0

blk\..*\.indexer\.attn_q_b\.weight=q6_K

# MTP block - eh_proj is the throughput-critical tensor per ik's finding

blk\..*\.nextn\.eh_proj\.weight=q4_0

blk\..*\.nextn\.enorm\.weight=f32

blk\..*\.nextn\.hnorm\.weight=f32

blk\..*\.nextn\.shared_head_norm\.weight=f32

token_embd\.weight=iq5_k

output\.weight=iq6_k

</div>

</details>

<div class="qz-note">The Q2 to run on 256 GiB systems. Tight 256 GiB-friendly footprint at 2.695 BPW. GGUF upload in progress — see status above.</div>

<div class="qz-note">Evaluated on wiki.test.raw, 512 ctx, 565 chunks, against BF16 reference logits.</div>

<tr><th>Metric</th><th>Value</th></tr>

<tr><td>PPL (quant)</td><td>3.436789 ± 0.018731</td></tr>

</table>

<summary>Quantization recipe</summary>

# Attention

blk\..*\.attn_k_b\.weight=q8_0

blk\..*\.attn_v_b\.weight=q8_0

blk\..*\.attn_kv_a_mqa\.weight=q8_0

blk\..*\.attn_q_a\.weight=q8_0

blk\..*\.attn_q_b\.weight=q8_0

blk\..*\.attn_output\.weight=iq6_k

# Router

blk\..*\.ffn_gate_inp\.weight=f32

# First 3 Dense Layers

blk\..*\.ffn_down\.weight=iq6_k

blk\..*\.ffn_(gate|up)\.weight=iq5_ks

# Shared Expert Layers

blk\..*\.ffn_down_shexp\.weight=iq6_k

blk\..*\.ffn_(gate|up)_shexp\.weight=iq6_k

# Routed Experts

blk\.(78)\.ffn_down_exps\.weight=q6_K

blk\.(78)\.ffn_(gate|up)_exps\.weight=q6_K

blk\..*\.ffn_down_exps\.weight=iq3_ks

blk\..*\.ffn_(gate|up)_exps\.weight=iq2_ks

# Indexer

blk\..*\.indexer\.attn_q_b\.weight=q6_K

blk\..*\.indexer\.attn_k\.weight=q8_0

blk\..*\.indexer\.proj\.weight=q8_0

blk\..*\.indexer\.k_norm\.(weight|bias)=f32

# MTP block - eh_proj is the throughput-critical tensor per ik's finding

blk\..*\.nextn\.eh_proj\.weight=q4_0

blk\..*\.nextn\.enorm\.weight=f32

blk\..*\.nextn\.hnorm\.weight=f32

blk\..*\.nextn\.shared_head_norm\.weight=f32

token_embd\.weight=iq6_k

output\.weight=iq6_k

</div>

</details>

<div class="qz-section">Quick Start</div>

<div class="qz-note">Requires <a href="https://github.com/ikawrakow/ik_llama.cpp">ik_llama.cpp</a>. Standard llama.cpp will not run these quants.</div>

# Clone and build

git clone https://github.com/ikawrakow/ik_llama.cpp

cd ik_llama.cpp

cmake -B build -DCMAKE_BUILD_TYPE=Release -DGGML_CUDA=ON

cmake --build build --config Release -j $(nproc)

# Download

pip install huggingface_hub

hf download L-Alchemyst/GLM-5.3-GGUF --repo-type model --include "IQ3_K/*" --local-dir ./glm5.3-iq3k

# Hybrid CPU+GPU — adjust --n-cpu-moe and -cram to your RAM

./build/bin/llama-server \

--model GLM-5.3-IQ3_K.gguf \

-muge --merge-qkv \

--ctx-size 131072 \

-ctk f16 -mla 3 -amb 1536 \

-ngl 999 \

--n-cpu-moe 50 \

--parallel 1 \

--threads 96 --threads-batch 128 \

--host 127.0.0.1 --port 8080 \

--no-mmap \

-cram 8192 \

--jinja

<div class="qz-note">CPU-only: use <code>-ctk q8_0</code> and prefix with <code>numactl -N ${SOCKET} -m ${SOCKET}</code>.</div>

Credits

Imatrix & reference logits: <a href="https://huggingface.co/AesSedai">Aes Sedai</a>

Base model: <a href="https://huggingface.co/zai-org/GLM-5.3-BF16">zai-org/GLM-5.3-BF16</a>

Runtime: <a href="https://github.com/ikawrakow/ik_llama.cpp">ik_llama.cpp</a>

</div>

</body>

</html>

Run L-Alchemyst/GLM-5.3-GGUF with guIDE

Download guIDE — the AI-native code editor with local LLM inference and 69 built-in tools.

Download guIDE → · Browse 524k+ models · Compare models

Source: Hugging Face · Compare models