zenlm/zen5-flash-gguf overview
Zen5 Flash Smallest and fastest tier in the Zen5 https://zenlm.org family. A dense 4B parameter instruct model with sub 100ms time to first token at 32K contex…
Repository Files & Downloads
| File | Type | Quantization | Size | Link |
|---|---|---|---|---|
| Browse files on Hugging Face | ||||
Model Details
| Model ID | zenlm/zen5-flash-gguf |
|---|---|
| Author | zenlm |
| Pipeline | text-generation |
| License | apache-2.0 |
| Base model | Qwen/Qwen3-4B |
| Last modified | 2026-07-14T09:02:50.000Z |
Model README
---
license: apache-2.0
base_model: Qwen/Qwen3-4B
library_name: transformers
pipeline_tag: text-generation
language:
- en
- zh
tags:
- zen5
- zen5-flash
- zenlm
- hanzo
- safetensors
- instruct
- agentic
- zen-5
---
Zen5 Flash
Smallest and fastest tier in the Zen5 family. A dense 4B-parameter instruct model with sub-100ms time-to-first-token at 32K context, tuned for high-throughput routing and simple agent loops.
Repackaged from Qwen/Qwen3-4B (apache-2.0, Alibaba Qwen) — redistributed as safetensors from the abliterated huihui-ai/Qwen3-4B-abliterated variant. Not trained from scratch — a permissively-licensed redistribution for the OSS-clean Zen model line.
Part of the canonical Zen5 ladder:
| SKU | Hardware fit | This repo |
|---|---|---|
| zen5-flash | anything (4 GB VRAM) | ← you are here |
| zen5-mini | hosted only | zen-5-mini-gguf |
| zen5 (default) | 24 GB+ VRAM | zen-5-gguf |
| zen5-pro | Mac M4 Max / DGX Spark / H100 80GB | zen-5-pro-gguf |
| zen5-max | Mac Studio M3 Ultra 512GB / 8x H100 | zen-5-max-gguf |
Files
| File | Format |
|---|---|
| model-00001-of-00002.safetensors + model-00002-of-00002.safetensors | sharded safetensors |
| tokenizer.json, tokenizer_config.json, special_tokens_map.json | tokenizer |
| config.json, generation_config.json | model config |
| chat_template.jinja | chat template |
Run
Hosted via the Hanzo gateway (api.hanzo.ai) as zen5-flash — see https://docs.hanzo.ai/zen.
Local with the zen5-engine or any transformers-compatible runtime:
from transformers import AutoModelForCausalLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("zenlm/zen-5-flash-gguf")
model = AutoModelForCausalLM.from_pretrained("zenlm/zen-5-flash-gguf", device_map="auto")
License
apache-2.0. Upstream: Qwen/Qwen3-4B by Alibaba Qwen; abliterated variant by huihui-ai. This repository redistributes a derivative under the same license.
Run zenlm/zen5-flash-gguf with guIDE
Download guIDE — the AI-native code editor with local LLM inference and 69 built-in tools.
Source: Hugging Face · Compare models