---

base_model: zai-org/GLM-5.2

base_model_relation: quantized

license: mit

tags:

• glm_moe_dsa
• conversational
• ik_llama.cpp

---

GLM-5.2-GGUF-1.630bpw

This is a 1.6 BPW quantized model for the desperate.

The quant aims to achieve best-in-class performance, by relying on SOTA quants from ik_llama.cpp:

• Routed experts tensors use the IQ1_S_R4 quant (1.5 BPW)
• Indexer tensors use either the Q6_0 quant (6.5 BPW) or the Q8_0 quant (8.5 BPW)
• All other tensors use the Q6_0 quant (6.5 BPW)

Coupled with the recent enhancements:

• MTP support with e.g. --spec-type mtp:n_max=4,p_min=0.0 (#1890)
• graph parallel support with -sm graph (#1821)
• DSA support with -dsa -fidx (#2045, #2098, #2109, and many others)

it should run at decent speed as well, with very little slowdown at long context.

(Note: For now, -sm graph and quantize KV cache e.g. -ctk q8_0 do not work together with -dsa -fidx. As a fun exercise, you can ask this quant to get them to work.)

Versions

There are 2 versions:

• GLM-5.2-GGUF-1.630bpw.gguf - Made with the imatrix from unsloth (thanks!)
• GLM-5.2-GGUF-1.630bpw-muzzy-imatrix.gguf - Made with the imatrix from muzzy (thanks!)

Comparison:

| version | imatrix | indexer | ppl |

| ---------------------------------------- | ------- | ------- | ------------------ |

| GLM-5.2-GGUF-1.630bpw.gguf | unsloth | Q6_0 | 6.1409 +/- 0.03770 |

| (not uploaded) | unsloth | Q8_0 | 6.1409 +/- 0.03770 |

| GLM-5.2-GGUF-1.630bpw-muzzy-imatrix.gguf | muzzy | Q8_0 | 5.7371 +/- 0.03465 |

The PPL results were gathered using llama-perplexity with -f wiki.test.raw -c 512.

Recommendation: GLM-5.2-GGUF-1.630bpw-muzzy-imatrix.gguf - The imatrix from muzzy, which took him 5 days to generate, is just amazing. At very low bitrate, imatrix plays an even more important role to keep the quant coherent. From testing, this quant works well enough for agentic coding.

Size

Size from llama-server output:

llm_load_print_meta: model size       = 142.952 GiB (1.630 BPW)
llm_load_print_meta: repeating layers = 141.512 GiB (1.618 BPW, 751.427 B parameters)

Buffer size with -cmoe --no-mmap:

llm_load_tensors:        CPU buffer size = 130712.34 MiB
llm_load_tensors:      CUDA0 buffer size = 13724.12 MiB

<details>

<summary>Recipe</summary>

# Attention
blk\..*\.attn_k_b\.weight=q6_0
blk\..*\.attn_v_b\.weight=q6_0

blk\..*\.attn_kv_a_mqa\.weight=q6_0
blk\..*\.attn_q_a\.weight=q6_0
blk\..*\.attn_q_b\.weight=q6_0
blk\..*\.attn_output\.weight=q6_0

# First 3 Dense Layers
blk\..*\.ffn_down\.weight=q6_0
blk\..*\.ffn_(gate|up)\.weight=q6_0

# Shared Expert Layers
blk\..*\.ffn_down_shexp\.weight=q6_0
blk\..*\.ffn_(gate|up)_shexp\.weight=q6_0

# Routed Experts Layers
blk\..*\.ffn_(up|gate|down)_exps\.weight=iq1_s_r4

# Indexer
blk\..*\.indexer\.proj\.weight=q8_0
blk\..*\.indexer\.attn_k\.weight=q8_0
blk\..*\.indexer\.attn_q_b\.weight=q8_0

# NextN MTP Layer
blk\..*\.nextn\.embed_tokens\.weight=q6_0
blk\..*\.nextn\.shared_head_head\.weight=q6_0
blk\..*\.nextn\.eh_proj\.weight=q6_0

# Non-Repeating Layers
token_embd\.weight=q6_0
output\.weight=q6_0

</details>

Flags

Example that uses DSA with IndexShare:

--no-mmap -ngl 99 --fit \
-mla 1 -dsa -fidx \
-c 102400 \
-b 2048 -ub 2048 -wgt 1 \
-cram 0 -muge -cuda graphs=1 \
--jinja --parallel-tool-calls \
--chat-template-kwargs {"reasoning_effort": "high"} \
--spec-type mtp:n_max=4,p_min=0.5

For the very desperate with only 64 GiB of RAM and a GPU, can try to rely on mmap by adding --defer-experts --prefetch-experts --no-warmup and removing --no-mmap -muge, as well as setting the environment variable GGML_CUDA_NO_PINNED=1. From https://github.com/ikawrakow/ik_llama.cpp/pull/2101, this can increase PP by more than 2x! (result varies based on disk speed)

Comparison with other quants

| quant | size | ppl |

| ---------------------------------------- | --------------------------- | ---------------------- |

| GLM-5.2-GGUF-1.630bpw-muzzy-imatrix.gguf | 142.952 GiB (1.630 BPW) | 5.7371 +/- 0.03465 |

| IQ1_KT from muzzy/GLM-5.2-GGUF | 169.190 GiB (1.928 BPW) | 4.4567 +/- 0.02620 |

| UD-IQ1_S from unsloth/GLM-5.2-GGUF | 201.823 GiB (2.300 BPW) | 4.5930 +/- 0.02686 |

The IQ1_KT from https://huggingface.co/muzzy/GLM-5.2-GGUF, using the same recipe as the smol-IQ1_KT from https://huggingface.co/ubergarm/GLM-5.1-GGUF, is about 26 GiB larger, and has a much better PPL. Strongly recommended.

On the other hand, the UD-IQ1_S from https://huggingface.co/unsloth/GLM-5.2-GGUF is about 59 GiB larger than this quant, and its PPL is still worse than the IQ1_KT from muzzy. Unless one is stuck with the mainline llama.cpp, there is no good reason to use this unsloth quant.