---

license: apache-2.0

base_model:

• Qwen/Qwen3-VL-8B-Instruct

base_model_relation: quantized

pipeline_tag: image-text-to-text

tags:

• gguf
• llama.cpp
• qwen-image
• text-encoder
• qwen3-vl

---

Qwen-Image-2.1 Text Encoder: GGUF

GGUF builds of the text encoder shipped in

Qwen/Qwen-Image-2.1 (text_encoder/). That encoder is stock

Qwen/Qwen3-VL-8B-Instruct: all 750 tensors match in name, shape

and dtype, and every sampled tensor is byte-identical. So these files also work as a general Qwen3-VL-8B-Instruct

GGUF.

| File | Quant | Size |

|---|---|---:|

| Qwen-Image-2.1-Text-Encoder-BF16.gguf | BF16 (lossless) | 16.39 GB |

| Qwen-Image-2.1-Text-Encoder-Q8_0.gguf | Q8_0 | 8.71 GB |

| Qwen-Image-2.1-Text-Encoder-Q6_K.gguf | Q6_K, imatrix, see below | 7.06 GB |

| Qwen-Image-2.1-Text-Encoder-Q4_K_M.gguf | Q4_K_M, imatrix, see below | 5.67 GB |

| Qwen-Image-2.1-Text-Encoder-mmproj-F16.gguf | vision projector, F16 | 1.16 GB |

The mmproj file carries the vision tower, needed for image-conditioned (edit) prompts.

Q6_K and Q4_K_M are not stock recipes. Plain Q6_K visibly garbled rendered text, so both use:

• an importance matrix computed on ~1500 real image prompts wrapped in the pipeline's own encoder template

(llama-imatrix --parse-special);

• the last 4 decoder layers (32-35) kept at Q8_0, because the pipeline reads the last layer's output directly;
• token embeddings at Q8_0.

Recommendation: Q8_0 if it fits. Q6_K is the smallest build that rendered text correctly in testing.

Q4_K_M is usable, but at the same seed it changes composition and can misspell rendered text.

How they were tested

For a diffusion text encoder, what matters is what the image model reads: the **last decoder layer's hidden

states before the final norm**, from the pipeline's own prompt templates. Chat quality and perplexity don't measure

that. Each quant was dequantized back into the HF model and run through diffusers' QwenImage21Pipeline.encode_prompt

on held-out prompts (12 text-to-image, 8 image-edit on 2 held-out images), then compared token by token with the bf16

original. Same-seed 1024px images were also rendered with each encoder.

| Build | Mean token cosine vs bf16 | Relative L2 error | Rendered text (same seed) |

|---|---|---|---|

| BF16 | 1.00000 (exact) | 0 | identical image |

| Q8_0 | 0.9938 | 0.070 (text-only 0.044, edit 0.110) | correct, same layout |

| Q6_K (imatrix recipe) | 0.9863 | 0.112 (text-only 0.079, edit 0.160) | correct, same layout |

| Q4_K_M (imatrix recipe) | 0.9319 | 0.295 (text-only 0.253, edit 0.358) | one of two titles misspelled ("224 Hours"), layout changed |

For comparison, stock-recipe Q6_K measured 0.979 / 0.145 and garbled a book title; stock Q4_K_M measured

0.872 / 0.394. Image-edit prompts (through the vision tower) always err more than text-only ones.

Q8_0, Q6_K and Q4_K_M were also run in llama-server with the mmproj. Each answered text questions correctly and described

a held-out photo accurately.

Use

llama-server -m Qwen-Image-2.1-Text-Encoder-Q8_0.gguf --mmproj Qwen-Image-2.1-Text-Encoder-mmproj-F16.gguf --jinja -ngl 99

Converted with llama.cpp's convert_hf_to_gguf.py (bd4f514) and quantized with llama-quantize (Q8_0 without imatrix; Q6_K / Q4_K_M as described above).

License: Apache-2.0, the license of Qwen3-VL-8B-Instruct. (The rest of Qwen-Image-2.1 is under the Qwen Research

License; this repository contains only the encoder.)