steampunque/GLM-Z1-9B-0414-MP-GGUF overview

---

license: mit

base_model: THUDM/GLM-Z1-9B-0414

base_model_relation: quantized

tags:

• GLM Z1 9B 0414
• GGUF
• quantized
• 4-bit

---

Mixed Precision GGUF layer quantization of GLM-Z1-9B-0414 by THUDM

Original model: https://huggingface.co/THUDM/GLM-Z1-9B-0414

The hybrid quant employs different quantization levels on a per layer basis to enable

both high performance and small file size at the same time. An extended layer definition

E quant Q4_E_H for the model is defined as follows (updated 10/12/2026):

   LAYER_TYPES='[
   ["A","attn","Q","attn_q","K","attn_k","V","attn_v","O","attn_o","S","ssm","F","ffn","G","ffn_g","U","ffn_u","D","ffn_d"],
   ["MAP","VOD","0","QN_K","2","Q2_K","3","Q3_K","4","Q4_K","5","Q5_K","6","Q6_K","8","Q8_0","h","F16","f","F32"],
   [0 ,"Q6_K_886"],[1 ,"Q5_K_666"],[2 ,"Q4_K_665"],[3 ,"Q4_K_655"],[4 ,"Q4_K_654"],[5 ,"Q4_K_654"],[6 ,"Q4_K_654"],[7 ,"Q4_K_654"],
   [8 ,"Q5_K_554"],[9 ,"Q4_K_554"],[10,"Q4_K_554"],[11,"Q4_K_554"],[12,"Q4_K_554"],[13,"Q4_K_554"],[14,"Q4_K_554"],[15,"Q4_K_554"],
   [16,"Q5_K_654"],[17,"Q4_K_654"],[18,"Q4_K_654"],[19,"Q4_K_654"],[20,"Q4_K_654"],[21,"Q4_K_654"],[22,"Q4_K_654"],[23,"Q4_K_654"],
   [24,"Q5_K_665"],[25,"Q4_K_665"],[26,"Q4_K_665"],[27,"Q4_K_665"],[28,"Q4_K_665"],[29,"Q4_K_665"],[30,"Q4_K_665"],[31,"Q4_K_665"],
   [32,"Q5_K_665"],[33,"Q5_K_665"],[34,"Q5_K_665"],[35,"Q5_K_665"],[36,"Q5_K_666"],[37,"Q5_K_666"],[38,"Q5_K_668"],[39,"Q6_K_886"]
   ]'
   FLAGS="--token-embedding-type Q6_K --output-tensor-type Q6_K --layer-types-high  --tensor-pad [[13696,13824],[27392,27648,2]] --override-kv glm4.feed_forward_length=int:13824"

The quant pads FFN dimension to even 256 so K quants can be used with it. The layer quants were optimized for strong performance

across a curated set of reasoning prompts with a minimum quant across layers at Q4_K to minimize entropy loss. It is sized to

~Q4_K_M bpw.

Comparison:

Quant | size | PPL | Comment

---------|---------|------|-----------

Q4_K_M | 6.2e9 | 14.9 | -

Q4_E_H | 6.2e9 | 15.2 | Hybrid quant with Q6_K embedding Q6_K output padded FFN tensors for K quants

Usage:

This is a RL trained thinking model. The layer quants for this model were optimized across a set of test/eval reasoning prompts.

The model can be speculated using Qwen3 0.6B if the inference engine can support dynamic vocab translation between

draft and target models. Approx performance using a downstream speculator with llama.cpp on a 4070 (12G VRAM) with layers and

context fully in GPU:

Q | QKV | ND | NKV | gen tps | Comment

------|-------|------|--------|------------|---------

Q4_E_H| F16 | 0 | 32k | 68 | No draft

Q4_E_H| F16 | 4 | 32k | 101 | Spec 4

Code:

Testing with some code problems show the model is extremely weak on code generation problems.

Benchmarks:

Math benchmarks for the model are given here: https://huggingface.co/spaces/steampunque/benchlm

Download the file from below:

| Link | Type | Size/e9 B | Notes |

|------|------|-----------|-------|

| GLM-Z1-9B-0414.Q4_E_H.gguf | Q4_E_H | 6.2e9 B | ~Q4_K_M size |

A discussion thread about the hybrid layer quant approach can be found here on the llama.cpp git repository:

https://github.com/ggml-org/llama.cpp/discussions/13040