---

base_model: zai-org/GLM-4.5-Air

tags:

• glm-4.5-air
• gguf
• iq2_xxs
• mtp
• text-generation-inference

language:

• en
• zh

---

GLM-4.5-Air-PS-IQ2_XXS-MTP GGUF

Compact GGUF quantization of zai-org/GLM-4.5-Air by PeasantSmith, quantized down from the full-size BF16 release. The only known quantized release that keeps the MTP (multi-token prediction) head intact for speculative decoding.

Specs

Format: GGUF

Base model: MoE, 106B total / 12B active params, 128 routed experts + 1 shared, top-8 routing, 46 layers (1 dense + 45 MoE), GQA (96 attention heads / 8 KV heads), 131,072 token context

Size: 45.49 GB / 42.4 GiB, 3.29 bits/param

Source: full-size BF16 GGUF (5 shards, 803 tensors), quantized with llama.cpp (commit c060ca9) using unsloth's published imatrix (502 entries) from the same release

MTP: 1 extra prediction layer embedded (blk.46.nextn.*), not present in other public quants of this model

Quantization Matrix

Source: full-size BF16 GGUF (5 shards, 803 tensors), quantized with llama.cpp (commit c060ca9)

using unsloth's published calibration importance matrix (imatrix, 502 entries) from the same full-size release.

| Tensor class | Type | Rationale |

|---|---|---|

| blk.1..45.ffn_gate_exps / ffn_up_exps | IQ2_XXS | 4096-col, imatrix-covered |

| blk.1..45.ffn_down_exps | Q4_0 | 1408-col (Q2_K impossible: 1408 % 256 = 128); down-exps precision on residual |

| blk.46.* (MTP block: gate/up/down) | Q4_0 | MTP draft head — no imatrix stats, quality immaterial |

| blk.46.nextn.* (MTP head) | Q4_K | embedded MTP, kept reasonably precise |

| attention / router / output | Q8_0 | lowest error per byte |

| token_embd / output | Q6_K | quality-critical |

| Layer 0 (dense) | Q8_0 (default) | no expert tensors |

Zero fallbacks; 90 IQ2_XXS tensors. SHA-256: 2f2a8c1234528ec00b0c73dfcb5d33395517e050cbbc7729a2249265bc16305c

Testing Results

Perplexity (512-token context, 67 chunks, calibration text): 5.5835 ± 0.1016. Measured on the calibration text itself, so treat as optimistic relative to held-out data.

| Test | Result |

|---|---|

| Factual QA (capital of France) | correct (Paris) |

| LRU implementation task | correct: O(1) doubly-linked-list design |

| Exact-value recall (magic value 9137) | exact |

MTP Head

• glm4moe.nextn_predict_layers = 1, MTP head embedded in-source at Q4_K (blk.46.nextn.*: embed_tokens, eh_proj, shared_head_head, norms)
• Usable via llama.cpp: llama-server --spec-type draft-mtp

Run it

`# Standard inference
llama-server -hf peasantsmith/GLM-4.5-Air-PS-IQ2_XXS-MTP-GGUF --temp 0.6 --top-p 0.9 -n 4096`

`# With the embedded MTP head, for speculative decoding
llama-server -hf peasantsmith/GLM-4.5-Air-PS-IQ2_XXS-MTP-GGUF --temp 0.6 --top-p 0.9 -n 4096 --spec-type draft-mtp`

llama-cli works the same way in place of llama-server for a terminal session instead of a server. Needs at least the 42GB file in memory, plus more for context.

Run with ollama

ollama run hf.co/peasantsmith/GLM-4.5-Air-PS-IQ2_XXS-MTP-GGUF:IQ2_XXS

The tag comes from the GGUF filename (last dash-separated segment). The embedded MTP/nextn head is available via llama.cpp (--spec-type draft-mtp); ollama runs the model as a standard MoE.

License

MIT, inherited from the base model.

Integrity

SHA-256: 2f2a8c1234528ec00b0c73dfcb5d33395517e050cbbc7729a2249265bc16305c File size: 45,490,244,672 bytes. Verify after download with sha256sum.

Additional information

The model name PS-IQ2_XXS-MTP is the PeasantSmith release signature. It identifies this quantized release and should not be confused with the upstream model's original naming.