---

library_name: gguf

tags:

• gguf
• quantized
• llama.cpp
• mixed-precision
• glm
• moe

license: other

license_name: glm-open-model-license

base_model: zai-org/GLM-5.2

base_model_relation: quantized

pipeline_tag: text-generation

---

GLM-5.2 — 307GB (GGUF)

Mixed-precision quantized version of zai-org/GLM-5.2 using a proprietary quantization method by baa.ai.

Per-tensor bit-width allocation via advanced sensitivity analysis and budget-constrained optimisation — no calibration data required. Built at the efficiency knee (best quality-per-GB).

Metrics

| Metric | Value |

|--------|-------|

| Size | 307 GB (7 shards) |

| Average bits | 3.50 |

| Format | llama.cpp (GGUF) |

| Architecture | MoE (256 experts, MLA + sparse attention) |

Routed experts Q3_K; attention Q4_K; DSA indexer / shared experts Q6_K–Q8_0; first/last-layer and protected tensors F16; output and token-embeddings Q6_K.

Requirements

llama.cpp build ≥ b9820 is required — GLM-5.2's sparse-attention shared-indexer layout is only handled by recent builds.

Usage

brew install llama.cpp

hf download baa-ai/GLM-5.2-RAM-307GB-GGUF --include "*.gguf" --local-dir ./glm-5.2-ram-307gb

# The model is split into 7 shards — point -m at the first; llama.cpp loads the rest automatically
llama-cli -m ./glm-5.2-ram-307gb/GLM-5.2-RAM-local-knee-00001-of-00007.gguf -p "Hello!" -n 256 -ngl 99

Or via llama-server for an OpenAI-compatible HTTP API:

llama-server -m ./glm-5.2-ram-307gb/GLM-5.2-RAM-local-knee-00001-of-00007.gguf --port 8080 -ngl 99 --ctx-size 8192

For fast inference use a host with ≥ ~310 GB RAM/VRAM; otherwise it runs via mmap paging. The chat template is embedded; GLM-5.2 supports a thinking mode. No importance matrix (imatrix) was used — per-tensor sensitivity allocation provides the primary quality signal.

---

Quantized by baa.ai

---

Black Sheep AI Products

Black Sheep AI builds private AI with the Bounded AI Architecture (BAA): three layers, each bound to something you control, on infrastructure you control. Nothing leaves your boundary.

Ram (Reasoning Layer) — Full-capability frontier models, shrunk to fit smaller GPUs with reasoning kept and certified. They rely strictly on your Data Modules for knowledge, so you swap the model as better ones ship.

Paddock (Knowledge Plane) — Binds your Data Modules directly to the reasoning layer: one per topic, period, person, permission or tenant. Permission, freeze, hand back or delete each one on its own.

Watchman (GRC Layer) — Governance, risk and compliance in a single, complete audit log: every module permission, every bind, every model's provenance. Five certificates before anything goes live.

Shepherd — Takes a frontier model, compresses it into a Ram reasoning model, certifies it with Watchman, binds it to your Paddock, and deploys the Bounded AI Architecture to your infrastructure: your cloud, a Mac Studio, or an air-gapped facility.

Learn more at baa.ai — Private AI for enterprises whose data cannot leave the building.