zenlm/zen-5-pro-gguf overview
Zen5 Pro High quality Zen5 tier that fits on a single 128 GB unified memory machine. Sparse MoE with 284B total / 37B active parameters per token, 1M context, …
Runs locally from ~3.55 GB disk (4 GB VRAM class GPUs with llama.cpp / guIDE).
Repository Files & Downloads
| File | Type | Quantization | Size | Link |
|---|---|---|---|---|
| DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix.gguf | GGUF | IQ2XXS | 80.76 GB | Download |
| DeepSeek-V4-Flash-Layers37-42Q4KExperts-OtherExpertLayersIQ2XXSGateUp-Q2KDown-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix-fixed.gguf | GGUF | Q2KDOWN | 90.89 GB | Download |
| DeepSeek-V4-Flash-MTP-Q4K-Q8_0-F32.gguf | GGUF | Q4K | 3.55 GB | Download |
| DeepSeek-V4-Flash-Q4KExperts-F16HC-F16Compressor-F16Indexer-Q8Attn-Q8Shared-Q8Out-chat-v2-imatrix.gguf | GGUF | Q4KEXPERTS | 153.33 GB | Download |
Model Details
| Model ID | zenlm/zen-5-pro-gguf |
|---|---|
| Author | zenlm |
| Pipeline | text-generation |
| License | apache-2.0 |
| Base model | — |
| Last modified | 2026-06-26T08:58:05.000Z |
Model README
---
license: apache-2.0
library_name: gguf
pipeline_tag: text-generation
language:
- en
- zh
tags:
- zen5
- zen5-pro
- zenlm
- hanzo
- gguf
- imatrix
- iq2_xxs
- moe
- 1m-context
- mtp
- zen-5
---
Zen5 Pro
High-quality Zen5 tier that fits on a single 128 GB unified-memory machine. Sparse MoE with 284B total / 37B active parameters per token, 1M context, asymmetric routed-MoE quantization (routed IQ2_XXS up/gate, Q2_K down; shared experts, attention projections, routing logits and the LM head left at higher precision).
Runs on a single 128 GB Apple Silicon (M3/M4 Max), DGX Spark (GB10), or H100 80 GB with the zen5-engine.
Part of the canonical Zen5 ladder:
| SKU | Hardware fit | This repo |
|---|---|---|
| zen5-flash | anything | zen-5-flash-gguf |
| zen5-mini | 32 GB | zen-5-mini-gguf |
| zen5 (default) | 24 GB+ VRAM | zen-5-gguf |
| zen5-pro | 128 GB single-machine | ← you are here |
| zen5-max | 512 GB / 8x H100 | zen-5-max-gguf |
Files
| File pattern | Size | Notes |
|---|---|---|
| -IQ2XXS-w2Q2K--chat-v2-imatrix.gguf | 81 GB | Recommended — IQ2_XXS routed-MoE with imatrix calibration, fits 128 GB |
| -Layers37-42Q4KExperts--imatrix-fixed.gguf | 91 GB | Mixed-quant — higher quality at the boundary layers, fits 128 GB |
| -Q4KExperts-F16HC--imatrix.gguf | 153 GB | Q4-imatrix — for 256 GB+ unified-memory machines |
| *-MTP-Q4K-Q8_0-F32.gguf | 3.5 GB | Optional speculative-decoding draft heads (use with --mtp) |
| -IQ2XXS-w2Q2K--chat-v2.gguf (non-imatrix) | 81 GB | Legacy Q2 — prefer the -imatrix version above |
| -Q4KExperts-F16HC--chat-v2.gguf (non-imatrix) | 153 GB | Legacy Q4 — prefer the -imatrix version above |
Run
Hosted via the Hanzo gateway (api.hanzo.ai) as zen5-pro.
Local with the zen5-engine:
git clone https://github.com/zenlm/zen5-engine
cd zen5-engine && make # macOS Metal
# or: make cuda-spark # DGX Spark GB10
# or: make cuda-generic # generic CUDA box
./download_model.sh q2-imatrix # pulls this repo's recommended GGUF
./zen5 -p "Hello"
./zen5-server --ctx 100000 --kv-disk-dir /tmp/zen5-kv --kv-disk-space-mb 8192Performance (Metal, --ctx 32768 --nothink)
| Machine | Prompt | Prefill | Generation |
|---|---|---:|---:|
| MacBook Pro M3 Max 128 GB | short | 58.5 t/s | 26.7 t/s |
| MacBook Pro M3 Max 128 GB | 11709 tok | 250.1 t/s | 21.5 t/s |
| Mac Studio M3 Ultra 512 GB | short | 84.4 t/s | 36.9 t/s |
| Mac Studio M3 Ultra 512 GB | 12018 tok (Q4) | 448.8 t/s | 26.6 t/s |
| DGX Spark GB10 128 GB | 7047 tok | 343.8 t/s | 13.7 t/s |
Run zenlm/zen-5-pro-gguf with guIDE
Download guIDE — the AI-native code editor with local LLM inference and 69 built-in tools.
Source: Hugging Face · Compare models