---

base_model: zai-org/GLM-5.2

base_model_relation: quantized

license: mit

tags:

• glm_moe_dsa
• conversational
• ik_llama.cpp

---

GLM-5.2-GGUF-2.788bpw

This is a 2.8 BPW quantized model for the GPU riches with more combined RAM + VRAM than common sense.

The quant aims to achieve best-in-class performance, by relying on SOTA quants from ik_llama.cpp:

• Routed experts tensors use the IQ2_KL quant (2.6875 BPW)
• Indexer tensors use the Q8_0 quant (8.5 BPW)
• All other tensors use the Q6_0 quant (6.5 BPW)

Coupled with the recent enhancements:

• MTP support with e.g. --spec-type mtp:n_max=4,p_min=0.0 (#1890)
• graph parallel support with -sm graph (#1821)
• DSA support with -dsa -fidx (#2045, #2098, #2109, and many others)

it should run at decent speed as well, with very little slowdown at long context.

(Note: For now, -sm graph and quantize KV cache e.g. -ctk q8_0 do not work together with -dsa -fidx. As a fun exercise, you can ask this quant to get them to work.)

Versions

There are 2 versions:

• GLM-5.2-GGUF-2.788bpw.gguf - Made with the imatrix from unsloth (thanks!)
• GLM-5.2-GGUF-2.788bpw-muzzy-imatrix.gguf - Made with the imatrix from muzzy (thanks!)

Comparison:

| version | imatrix | indexer | ppl |

| ---------------------------------------- | ------- | ------- | ------------------ |

| GLM-5.2-GGUF-2.788bpw.gguf | unsloth | Q8_0 | 3.2731 +/- 0.01793 |

| GLM-5.2-GGUF-2.788bpw-muzzy-imatrix.gguf | muzzy | Q8_0 | 3.2172 +/- 0.01751 |

The PPL results were gathered using llama-perplexity with -f wiki.test.raw -c 512.

Recommendation: GLM-5.2-GGUF-2.788bpw-muzzy-imatrix.gguf - The imatrix from muzzy, which took him 5 days to generate, is just amazing.

Size

Size from llama-server output:

llm_load_print_meta: model size       = 244.483 GiB (2.788 BPW)
llm_load_print_meta: repeating layers = 243.043 GiB (2.778 BPW, 751.427 B parameters)

Buffer size with -cmoe --no-mmap:

llm_load_tensors:        CPU buffer size = 233312.34 MiB
llm_load_tensors:      CUDA0 buffer size = 13724.12 MiB

<details>

<summary>Recipe</summary>

# Attention
blk\..*\.attn_k_b\.weight=q6_0
blk\..*\.attn_v_b\.weight=q6_0

blk\..*\.attn_kv_a_mqa\.weight=q6_0
blk\..*\.attn_q_a\.weight=q6_0
blk\..*\.attn_q_b\.weight=q6_0
blk\..*\.attn_output\.weight=q6_0

# First 3 Dense Layers
blk\..*\.ffn_down\.weight=q6_0
blk\..*\.ffn_(gate|up)\.weight=q6_0

# Shared Expert Layers
blk\..*\.ffn_down_shexp\.weight=q6_0
blk\..*\.ffn_(gate|up)_shexp\.weight=q6_0

# Routed Experts Layers
blk\..*\.ffn_(up|gate|down)_exps\.weight=iq2_kl

# Indexer
blk\..*\.indexer\.proj\.weight=q8_0
blk\..*\.indexer\.attn_k\.weight=q8_0
blk\..*\.indexer\.attn_q_b\.weight=q8_0

# NextN MTP Layer
blk\..*\.nextn\.embed_tokens\.weight=q6_0
blk\..*\.nextn\.shared_head_head\.weight=q6_0
blk\..*\.nextn\.eh_proj\.weight=q6_0

# Non-Repeating Layers
token_embd\.weight=q6_0
output\.weight=q6_0

</details>

Flags

Example that uses DSA with IndexShare:

--no-mmap -ngl 99 --fit \
-mla 1 -dsa -fidx \
-c 102400 \
-b 2048 -ub 2048 -wgt 1 \
-cram 0 -muge -cuda graphs=1 \
--jinja --parallel-tool-calls \
--chat-template-kwargs {"reasoning_effort": "high"} \
--spec-type mtp:n_max=4,p_min=0.5

Comparison with other quants

| quant | size | ppl |

| ---------------------------------------- | --------------------------- | ---------------------- |

| GLM-5.2-GGUF-2.788bpw-muzzy-imatrix.gguf | 244.483 GiB (2.788 BPW) | 3.2172 +/- 0.01751 |

| IQ2_KL from muzzy/GLM-5.2-GGUF | 261.988 GiB (2.985 BPW) | 3.1085 +/- 0.01690 |

| UD-IQ3_XXS from unsloth/GLM-5.2-GGUF | 262.334 GiB (2.989 BPW) | 3.2952 +/- 0.01824 |

The IQ2_KL from https://huggingface.co/muzzy/GLM-5.2-GGUF, using the same recipe as the IQ2_KL from https://huggingface.co/ubergarm/GLM-5.1-GGUF, is about 17.5 GiB larger than this, and has a better PPL. Recommended if there is enough RAM + VRAM.

On the other hand, the UD-IQ3_XXS from https://huggingface.co/unsloth/GLM-5.2-GGUF is even larger, while having a worse PPL. Unless one is stuck with the mainline llama.cpp, there is no good reason to use this unsloth quants.