pottokao/Qwen3-4B-Text-Encoder-DHQ-GGUF overview
Qwen3 4B text encoder GGUF, quantized by what the DiT sees DHQ Two 2.9 GB GGUFs of the Qwen3 4B text encoder, quantized with DHQ , a per tensor mixed precision…
Runs locally from ~2.71 GB disk (4 GB VRAM class GPUs with llama.cpp / guIDE).
Repository Files & Downloads
Model Details
Model README
---
license: apache-2.0
base_model:
- Qwen/Qwen3-4B
- BennyDaBall/Z-Image-Engineer-V6
tags:
- gguf
- text-encoder
- z-image
- flux2-klein
- comfyui
- quantization
---
Qwen3-4B text encoder GGUF, quantized by what the DiT sees (DHQ)
Two 2.9 GB GGUFs of the Qwen3-4B text encoder, quantized with DHQ, a per-tensor mixed-precision allocation that is scored by the diffusion model rather than by next-token KL.
| File | Source | Size | For |
|---|---|---|---|
| qwen3_4b_dhq_2p9.gguf | Qwen/Qwen3-4B | 2.91 GB | Z-Image and FLUX.2 [klein] 4B text encoder |
| zimage_engineer_v6_dhq_2p9.gguf | BennyDaBall/Z-Image-Engineer-V6 | 2.91 GB | Z-Image text encoder that also writes prompts (one file for both jobs) |
About the DHQ series
DHQ files are quantized for use as a diffusion model's text encoder: each tensor's precision is chosen by how
much its quantization error changes what the target DiT reads (measured through the DiT's own text-input stage),
not by next-token loss. They stay normal LLM GGUFs, so the same file can still write or rewrite prompts: where a
model is also used as a prompt enhancer (Z-Image-Engineer-V6, the heretic Qwen3-VL-8B), every file is also checked
for that (the PE column) and the tables show where it gets worse.
DHQ repos: Qwen-Image-2.1 ·
Boogu ·
Qwen3-4B (Z-Image / FLUX.2 klein 4B) ·
We also built Dual-Head Dragon (ComfyUI-DualHead-Dragon), a ComfyUI node that runs one GGUF as both the text encoder and the prompt
enhancer (llama.cpp inside). It is public but still in testing, including its LoRA support
(separate LoRAs for the encoding and the rewriting head) and DFlash speculative decoding for faster rewriting.
The files here work without it (see How to load).
Why
A text encoder feeding a DiT is judged by the hidden states the DiT reads (Z-Image reads layer 34; klein reads layers 8/17/26), not by the next token. Standard quants (llama.cpp Q4_K_M, Unsloth Dynamic) spend bits based on LLM output. DHQ measures each tensor's damage through the DiT's input layer and spends bits where the DiT feels it.
Rules for these files: every tensor is at least Q4_K; the size budget is 2.9 GB. Mix: Q4_K / Q5_K / Q6_K / Q8_0 per tensor. The full per-layer table (q/k/v/o/gate/up/down for every layer) is in GRIDS.md.
Measured error (real GGUFs, lower is better)
Metric: relative error of the text-encoder output after the DiT's input layer, compared with BF16, averaged over 18 prompts (12 long, 6 short). All files run through llama.cpp's real kernels.
Qwen3-4B
| File | Size | Z-Image | klein 4B |
|---|---|---|---|
| Q8_0 | 4.28 GB | 2.1% | 1.4% |
| DHQ 2.9 (this repo) | 2.91 GB | 6.0% | 4.0% |
| Unsloth UD-Q5_K_XL type map, same imatrix | 2.90 GB | 6.8% | 5.0% |
| Q4_K_M (imatrix) | 2.50 GB | 11.5% | 8.3% |
Z-Image-Engineer-V6
| File | Size | Z-Image |
|---|---|---|
| DHQ 2.9 (this repo) | 2.91 GB | 4.5% |
| Q4_K_M | 2.50 GB | 10.6% |
The V6 file uses an importance matrix computed on V6 itself (same calibration text as the Qwen3-4B file). On 36 held-out prompts never used to pick the mix it scores 4.4%.
Formats
Both files use only K-quant types (Q4_K / Q5_K / Q6_K / Q8_0 per tensor), so they decode fast everywhere
(GPU, CPU, Metal, Vulkan, and ComfyUI-GGUF on the GPU). There are no IQ files in this repo.
Usage
ComfyUI (needs ComfyUI-GGUF): put the file in models/text_encoders/, load it with CLIPLoaderGGUF, type lumina2 for Z-Image or flux2 for FLUX.2 [klein] 4B. Everything else is the stock workflow.
llama.cpp / LM Studio: both are standard qwen3 GGUFs. The V6 file can write Z-Image prompts the same way the original V6 does.
Reproducibility
recipes/ has one recipe per file: source repo and revision, llama.cpp commit, importance matrix (sha256), and the
type of every tensor as built. Rebuilding from a recipe gives a bit-identical file (checked).
sha256
1be4aa21207e48ac7a06a9b5a6656d3aa8b612769a3cbcf9b6960c173860ab42 qwen3_4b_dhq_2p9.gguf
81125bf274d28b0481984ac0da2bacbf9d7505e58d34962c0e6b0da886d48762 zimage_engineer_v6_dhq_2p9.ggufLicense
Apache-2.0, same as both source models. Credit to the Qwen team for Qwen3-4B and to BennyDaBall for Z-Image-Engineer-V6.
Run pottokao/Qwen3-4B-Text-Encoder-DHQ-GGUF with guIDE
Download guIDE — the AI-native code editor with local LLM inference and 69 built-in tools.
Source: Hugging Face · Compare models