---

library_name: gguf

tags:

• gguf
• quantized
• llama.cpp
• mixed-precision
• glm
• moe

license: other

license_name: glm-open-model-license

base_model: zai-org/GLM-5.2

base_model_relation: quantized

pipeline_tag: text-generation

---

GLM-5.2 — 307GB (GGUF)

Mixed-precision quantized version of zai-org/GLM-5.2 using a proprietary quantization method by baa.ai.

Per-tensor bit-width allocation via advanced sensitivity analysis and budget-constrained optimisation — no calibration data required. Built at the efficiency knee (best quality-per-GB).

Metrics

| Metric | Value |

|--------|-------|

| Size | 307 GB (7 shards) |

| Average bits | 3.50 |

| Format | llama.cpp (GGUF) |

| Architecture | MoE (256 experts, MLA + sparse attention) |

Routed experts Q3_K; attention Q4_K; DSA indexer / shared experts Q6_K–Q8_0; first/last-layer and protected tensors F16; output and token-embeddings Q6_K.

Requirements

llama.cpp build ≥ b9820 is required — GLM-5.2's sparse-attention shared-indexer layout is only handled by recent builds.

Usage

brew install llama.cpp

hf download baa-ai/GLM-5.2-RAM-307GB-GGUF --include "*.gguf" --local-dir ./glm-5.2-ram-307gb

# The model is split into 7 shards — point -m at the first; llama.cpp loads the rest automatically
llama-cli -m ./glm-5.2-ram-307gb/GLM-5.2-RAM-local-knee-00001-of-00007.gguf -p "Hello!" -n 256 -ngl 99

Or via llama-server for an OpenAI-compatible HTTP API:

llama-server -m ./glm-5.2-ram-307gb/GLM-5.2-RAM-local-knee-00001-of-00007.gguf --port 8080 -ngl 99 --ctx-size 8192

For fast inference use a host with ≥ ~310 GB RAM/VRAM; otherwise it runs via mmap paging. The chat template is embedded; GLM-5.2 supports a thinking mode. No importance matrix (imatrix) was used — per-tensor sensitivity allocation provides the primary quality signal.

---

Quantized by baa.ai

---

Black Sheep AI Products

Shepherd — Private AI deployment platform that shrinks frontier models by 50-60% through RAM compression, enabling enterprises to run sophisticated AI on single GPU instances or Apple Silicon hardware. Deploy in your VPC with zero data leaving your infrastructure. Includes CI/CD pipeline integration, fleet deployment across Apple Silicon clusters, air-gapped and sovereign deployment support, and multi-format export (MLX, GGUF). Annual cloud costs from ~$2,700 — or run on a Mac Studio for electricity only.

Watchman — Capability audit and governance platform for compressed AI models. Know exactly what your quantized model can do before it goes live. Watchman predicts which capabilities survive compression in minutes — replacing weeks of benchmarking. Includes compliance-ready reporting for regulated industries, quality valley warnings for counterproductive memory allocations, instant regression diagnosis tracing issues to specific tensors, and 22 adversarial security probes scanning for injection, leakage, hallucination, and code vulnerabilities.

Learn more at baa.ai — Sovereign AI.