steampunque/GLM-Z1-32B-0414-MP-GGUF overview

---

license: mit

base_model: zai-org/GLM-Z1-32B-0414

base_model_relation: quantized

tags:

• GLM Z1 32B 0414
• GGUF
• quantized
• 4-bit

---

Mixed Precision GGUF layer quantization of GLM-Z1-32B-0414 by zai-org

Original model: https://huggingface.co/zai-org/GLM-Z1-32B-0414

The hybrid quant employs different quantization levels on a per layer basis to enable

both high performance and small file size at the same time. An extended layer definition

E quant Q4_E_H for the model is defined as follows (updated 10/12/2026):

   LAYER_TYPES='[
   ["A","attn","Q","attn_q","K","attn_k","V","attn_v","O","attn_o","S","ssm","F","ffn","G","ffn_g","U","ffn_u","D","ffn_d"],
   ["MAP","VOD","0","QN_K","2","Q2_K","3","Q3_K","4","Q4_K","5","Q5_K","6","Q6_K","8","Q8_0","h","F16","f","F32"],
   [0 ,"Q5_K_886"],[1 ,"Q5_K_665"],[2 ,"Q4_K_655"],[3 ,"Q4_K_655"],
   [4 ,"Q4_K_644"],[5 ,"Q4_K_644"],[6 ,"Q4_K_644"],[7 ,"Q4_K_644"],
   [8 ,"Q4_K_544"],[9 ,"Q4_K_544"],[10,"Q4_K_544"],[11,"Q4_K_544"],
   [12,"Q4_K_544"],[13,"Q4_K_544"],[14,"Q4_K_544"],[15,"Q4_K_544"],
   [16,"Q4_K_544"],[17,"Q4_K_544"],[18,"Q4_K_544"],[19,"Q4_K_544"],
   [20,"Q4_K_544"],[21,"Q4_K_544"],[22,"Q4_K_544"],[23,"Q4_K_544"],
   [24,"Q4_K_544"],[25,"Q4_K_544"],[26,"Q4_K_544"],[27,"Q4_K_544"],
   [28,"Q4_K_544"],[29,"Q4_K_544"],[30,"Q4_K_544"],[31,"Q4_K_544"],
   [32,"Q4_K_544"],[33,"Q4_K_544"],[34,"Q4_K_544"],[35,"Q4_K_544"],
   [36,"Q4_K_544"],[37,"Q4_K_544"],[38,"Q4_K_544"],[39,"Q4_K_544"],
   [40,"Q4_K_544"],[41,"Q4_K_544"],[42,"Q4_K_544"],[43,"Q4_K_544"],
   [44,"Q4_K_544"],[45,"Q4_K_544"],[46,"Q4_K_544"],[47,"Q4_K_544"],
   [48,"Q4_K_544"],[49,"Q4_K_544"],[50,"Q4_K_544"],[51,"Q4_K_544"],
   [52,"Q4_K_555"],[53,"Q4_K_445"],[54,"Q4_K_655"],[55,"Q4_K_655"],
   [56,"Q4_K_655"],[57,"Q4_K_666"],[58,"Q4_K_666"],[59,"Q5_K_668"],
   [60,"Q6_K_886"]
   ]'
   FLAGS="--token-embedding-type Q4_K --output-tensor-type Q6_K --layer-types-high"

The layer quants were optimized with smallest quant across layers at Q4_K to minimize entropy loss and

provide extremely strong performance across a set of reasoning prompts using greedy sample. The final bpw results

in ~0.5G smaller file size than Q4_K_M.

Comparison:

Quant | size | PPL | Comment

---------|---------|------|-----------

Q4_K_M | 19.7e9 | 7.7 | default embedding/output

Q4_E_H | 19.3e9 | 7.7 | Hybrid E quant with Q4_K embedding Q6_K output

Usage:

This is a RL trained thinking model. The layer quants for this model were evaluated on a set of test/eval prompts using

greedy sampling. The quant/model show excellent performance in providing both minimal overthinking and/or getting stuck in infinite generations.

The model can be speculated using Qwen3 0.6B if the inference engine can support dynamic vocab translation between

draft and target models. Approx performance using a downstream speculator with llama.cpp on two 4070s (12G VRAM) with layers and

context fully in GPU and connected via RPC:

Q | QKV | ND | NKV | gen tps | Comment

------|-------|------|--------|------------|---------

Q4_E_H| F16 | 0 | 32k | 22 | No draft

Q4_E_H| F16 | 4 | 16k | 42 | Spec 4

Q4_E_H| Q8_0 | 4 | 18k | 41 | Spec 4

Benchmarks:

Math benchmarks for the model are given here: https://huggingface.co/spaces/steampunque/benchlm

Download the file from below:

| Link | Type | Size/e9 B | Notes |

|------|------|-----------|-------|

| GLM-Z1-32B-0414.Q4_E_H.gguf | Q4_E_H | 19.3e9 B | ~0.5B smaller than Q4_K_M |

A discussion thread about the hybrid layer quant approach can be found here on the llama.cpp git repository:

https://github.com/ggml-org/llama.cpp/discussions/13040