---

license: other

license_name: qwen-research

license_link: LICENSE

library_name: gguf

base_model:

- Qwen/Qwen-Image-2.1-PE-T2I

- Qwen/Qwen-Image-2.1-PE-I2I

- pottokao/Qwen-Image-2.1-PE-T2I-Heretic

- darrellbest/Qwen-Image-2.1-PE-I2I-Heretic

base_model_relation: quantized

tags:

- gguf

- llama.cpp

- qwen3.5

- mtp

- speculative-decoding

- prompt-enhancer

- qwen-image

---

Qwen-Image 2.1 Prompt Enhancer GGUF (with MTP)

GGUF builds of the Qwen-Image 2.1 prompt enhancers (PE), with the Qwen3.5 multi-token prediction (MTP) head included, so llama.cpp can use speculative decoding (--spec-type draft-mtp).

These files are made for the llama.cpp backend of ComfyUI-Qwen-Image-2.1-PromptEnhancer-MTP, which downloads them automatically. They also work with a plain llama-server (see Usage).

The PE models don't ship an MTP head. It is grafted from the base model they were fine-tuned from, Qwen/Qwen3.5-9B (the same 15 mtp.* tensors the ComfyUI node uses). Community GGUFs of these models have no MTP head and no Q8_0.

Files

| File | Model | Quant | Size |

|---|---|---|---|

| qwen3.5_9b_qwen_image_2.1_pe_t2i.mtp.Q8_0.gguf | t2i | Q8_0 | 9.79 GB |

| qwen3.5_9b_qwen_image_2.1_pe_t2i.mtp.Q4_K_M.gguf | t2i | Q4_K_M | 5.99 GB |

| qwen3.5_9b_qwen_image_2.1_pe_i2i.mtp.Q8_0.gguf | i2i | Q8_0 | 9.79 GB |

| qwen3.5_9b_qwen_image_2.1_pe_i2i.mtp.Q4_K_M.gguf | i2i | Q4_K_M | 5.99 GB |

| qwen3.5_9b_qwen_image_2.1_pe_t2i_heretic.mtp.Q8_0.gguf | t2i - heretic | Q8_0 | 9.79 GB |

| qwen3.5_9b_qwen_image_2.1_pe_t2i_heretic.mtp.Q4_K_M.gguf | t2i - heretic | Q4_K_M | 5.99 GB |

| qwen3.5_9b_qwen_image_2.1_pe_i2i_heretic.mtp.Q8_0.gguf | i2i - heretic | Q8_0 | 9.79 GB |

| qwen3.5_9b_qwen_image_2.1_pe_i2i_heretic.mtp.Q4_K_M.gguf | i2i - heretic | Q4_K_M | 5.99 GB |

| qwen3.5_9b_qwen_image_2.1_pe_i2i.mmproj.bf16.gguf | vision encoder for both i2i models | BF16 | 0.92 GB |

• Q8_0: for GPUs with 16 GB of VRAM or more. Its output is practically identical to bf16 (see Quality).
• Q4_K_M: for 8–12 GB GPUs. It drifts more from bf16 but stays well within the quality bar, and it is also faster (it reads fewer bytes per token).
• Vision file: the i2i models need it for images. There is one file, converted from Qwen/Qwen-Image-2.1-PE-I2I; the heretic i2i model uses the same vision encoder (heretic only changes the language model).
• *.build.json: the source, llama.cpp build, quantize arguments and mean KLD of each model's build.

Sources

| Model | Source (bf16) | Revision |

|---|---|---|

| t2i | Qwen/Qwen-Image-2.1-PE-T2I | f3ed7985c788ad75b3ab7223e0c4c51e2a43545b |

| i2i | Qwen/Qwen-Image-2.1-PE-I2I | 72927bc08afc99b7888ceb7d7d51a12db3700bbd |

| t2i - heretic | pottokao/Qwen-Image-2.1-PE-T2I-Heretic | 96e92081f46cd0ca2ac9679432c5b8b02c12cfdb |

| i2i - heretic | darrellbest/Qwen-Image-2.1-PE-I2I-Heretic | 0402e068524500b29633ccc4f85b90779fe73092 |

| MTP head (all models) | Qwen/Qwen3.5-9B | c202236235762e1c871ad0ccb60c8ee5ba337b9a |

The heretic models are community abliterated fine-tunes that refuse less.

How they were built

llama.cpp build b11160 (commit 70c4e1582), per model:

1. Download the bf16 source.
2. Add the base model's mtp.* tensors as an extra shard (model-mtp.safetensors), list them in model.safetensors.index.json, and set text_config.mtp_num_hidden_layers to 1 (the PE configs have it as null, which the converter rejects).
3. convert_hf_to_gguf.py --outtype bf16. For i2i, also --mmproj --outtype bf16 for the vision file.
4. Build an importance matrix from the bf16 GGUF with llama-imatrix (calibration text below).
5. Quantize:

- Q8_0: plain llama-quantize … Q8_0.

- Q4_K_M: llama-quantize --imatrix <model>.imatrix.gguf --tensor-type ssm_alpha=q8_0 --tensor-type ssm_beta=q8_0 --tensor-type attn_gate=q6_K --tensor-type ssm_out=q6_K … Q4_K_M

1. Measure the KL divergence of each quant against the bf16 model and reject it above the bar (Q8_0 0.01, Q4_K_M 0.05).

The build script is tools/build_gguf.py in the node's repository.

Why this Q4_K_M recipe

Plain Q4_K_M drifts too far on this model: a mean KLD of 0.146, three times the bar. Qwen3.5 is a hybrid model (24 of its 32 layers are linear attention), and those layers' gate and output projections are sensitive to 4-bit. Tested on the t2i model:

| Q4_K_M variant | Mean KLD | Median KLD | 99% KLD | Same top token | Size |

|---|---|---|---|---|---|

| plain | 0.1460 | 0.0324 | 2.24 | 87.8% | 5.78 GB |

| + importance matrix | 0.0717 | 0.0130 | 0.68 | 91.8% | 5.78 GB |

| + importance matrix, ssm_alpha/ssm_beta at q8_0 | 0.0740 | 0.0132 | – | 92.1% | 5.78 GB |

| ssm_alpha/ssm_beta at q8_0 only | 0.1460 | 0.0309 | – | 88.0% | 5.78 GB |

| + importance matrix, ssm_alpha/ssm_beta q8_0, attn_gate/ssm_out q6_K (used) | 0.0398 | 0.0093 | 0.41 | 93.0% | 5.99 GB |

The importance matrix halves the drift; keeping the linear-attention gates and outputs at 6-bit halves it again, for 0.2 GB.

MTP layer precision

The MTP layer (blk.32) only drafts tokens; the main model checks every draft, so its precision affects speed, never the output. In Q4_K_M it is 4-bit. The importance matrix has no data for it, since calibration only runs the main model. Keeping it at 8-bit made no measurable difference (t2i, 3 prompts × 2 seeds, RTX 4090 Laptop):

| File | Draft acceptance | Tokens/s |

|---|---|---|

| Q8_0 | 35.1% | 60.4 |

| Q4_K_M, MTP layer 4-bit (used) | 33.7% | 76.4 |

| Q4_K_M, MTP layer 8-bit | 34.3% | 75.5 |

Quality (KL divergence)

Each quant is compared with its own bf16 model using llama-perplexity --kl-divergence (context 4096, flash attention). Lower is closer to bf16.

Evaluation text: the official t2i PE system prompt followed by 12 rewritten prompts written by the t2i PE itself (thinking and JSON answer, 6 from llama.cpp and 6 from ComfyUI), about 6k scored tokens. It is in the prompt enhancer's own format, so it measures drift where it matters. The i2i models are measured on the same text-only file.

Calibration text (importance matrix): 16 answers from the t2i Q8_0 model to other ideas, none from the evaluation text, about 35k tokens. It is used for all four models.

Mean KLD, all models

| Model | Q8_0 | Q4_K_M |

|---|---|---|

| t2i | 0.0054 | 0.0398 |

| i2i | 0.0019 | 0.0150 |

| t2i - heretic | 0.0060 | 0.0414 |

| i2i - heretic | 0.0017 | 0.0156 |

Details

| File | Mean KLD | Median KLD | 99% KLD | Same top token | RMS Δp |

|---|---|---|---|---|---|

| t2i Q8_0 | 0.0054 | 0.00055 | 0.034 | 98.2% | 1.66% |

| t2i Q4_K_M | 0.0398 | 0.0093 | 0.41 | 93.0% | 5.09% |

| i2i Q8_0 | 0.0019 | 0.00049 | 0.0052 | 98.5% | 0.77% |

| i2i Q4_K_M | 0.0150 | 0.0080 | 0.084 | 94.5% | 2.98% |

| For reference: the ComfyUI node's int8 convrot t2i model | 0.0052 | 0.00107 | – | 97.5% | 2.12% |

Q8_0 is on par with the int8 model the ComfyUI backend uses: closer on typical tokens (lower median), with an equal mean. The int8 figures are for its weights only, dequantized to F16.

8-bit KV cache

The KV cache can be stored at 8 bits (-ctk q8_0 -ctv q8_0, the node's kv_cache: q8_0), saving memory on small GPUs. Only 8 of Qwen3.5's 32 layers keep a KV cache, so the saving is modest (about 240 MiB at a 16k context), and so is the cost (i2i model):

| Quant | KV f16 | KV q8_0 |

|---|---|---|

| Q8_0 | 0.0019 | 0.0023 |

| Q4_K_M | 0.0150 | 0.0150 |

Speed

Measured through the ComfyUI node on an RTX 4090 Laptop (16 GB), 6 generations per configuration (3 prompts × 2 seeds for t2i; one two-image edit × 6 seeds for i2i) after a warmup. All runs used the official PE settings: Qwen's system prompts and sampling, thinking on, and the official max length (t2i 16256, i2i 24000). f16 KV cache; Q8_0 unless noted.

Speed is compared in tokens generated per second (tok/s), not time, since answers differ in length from run to run. Tok/s is the answer's tokens divided by the time the PE took, including everything the node does: unloading ComfyUI's models, waking llama-server, generating, and waiting for it to free its VRAM again. Each step adds one change, starting from the node's ComfyUI backend (int8 convrot) without MTP.

t2i

| Step | Tok/s | Step gain | Total vs baseline |

|---|---|---|---|

| ComfyUI, MTP off (baseline) | 23.5 | – | 1.00× |

| + MTP | 33.9 | 1.44× | 1.44× |

| + llama.cpp (these GGUFs, Q8_0) | 46.8 | 1.38× | 1.99× |

| + Q4_K_M instead of Q8_0 | 63.2 | 1.35× | 2.69× |

i2i (two input images)

| Step | Tok/s | Step gain | Total vs baseline |

|---|---|---|---|

| ComfyUI, MTP off (baseline) | 19.4 | – | 1.00× |

| + MTP | 32.6 | 1.68× | 1.68× |

| + llama.cpp (these GGUFs, Q8_0) | 67.8 | 2.08× | 3.49× |

| + Q4_K_M instead of Q8_0 | 85.2 | 1.26× | 4.39× |

• llama.cpp without MTP: 42.3 tok/s (t2i, 1.80× the baseline) and 46.6 tok/s (i2i, 2.40×).
• llama.cpp's speed doesn't depend on the reserved length: through the node, 8192 gave the same speed as the official lengths for both t2i and i2i (standalone decode: 67.4 tok/s with MTP, 52.3 without, at any context size).
• MTP draft acceptance is about 35–41% (t2i) and higher on i2i. MTP decoding keeps one CPU core busy, so heavy CPU load elsewhere slows it more than decoding without MTP.
• Q4_K_M is faster than Q8_0 because it reads fewer bytes per token, at the quality cost shown under Quality.

At the node's default max length (8192)

The ComfyUI backend is faster at 8192 than at the official lengths, which narrows llama.cpp's lead. Totals are relative to ComfyUI without MTP at 8192.

t2i

| Step | Tok/s | Step gain | Total vs baseline |

|---|---|---|---|

| ComfyUI, MTP off (baseline) | 30.8 | – | 1.00× |

| + MTP | 41.1 | 1.33× | 1.33× |

| + llama.cpp (these GGUFs, Q8_0) | 47.0 | 1.14× | 1.53× |

| + Q4_K_M instead of Q8_0 | 62.6 | 1.33× | 2.03× |

i2i (two input images)

| Step | Tok/s | Step gain | Total vs baseline |

|---|---|---|---|

| ComfyUI, MTP off (baseline) | 27.9 | – | 1.00× |

| + MTP | 45.8 | 1.64× | 1.64× |

| + llama.cpp (these GGUFs, Q8_0) | 67.7 | 1.48× | 2.43× |

| + Q4_K_M instead of Q8_0 | 85.4 | 1.26× | 3.06× |

All 60 answers at 8192 parsed and none reached the limit; the longest was 5,437 tokens (i2i).

Usage

ComfyUI

Install ComfyUI-Qwen-Image-2.1-PromptEnhancer-MTP and set the loader's backend to llama.cpp. The node downloads llama-server and the files it needs into ComfyUI/models/LLM/Qwen-Image-2.1-PE/. It needs an NVIDIA GPU on Windows or Linux (x64).

By default the node frees llama-server's VRAM after each prompt, so the rest of the workflow gets the GPU. On a GPU with room for both the PE and your other models (typically 24–32 GB or more), turn off the loader's unload_model to keep llama-server and ComfyUI's models loaded between prompts: each prompt then skips reloading the PE, about 6 s faster for t2i on an RTX 4090 Laptop. On a GPU that's too small, the rest of the workflow slows down instead, since ComfyUI can't free llama-server's memory.

unload_model exists only for the llama.cpp backend. The node's ComfyUI backend doesn't need it: ComfyUI's dynamic VRAM moves its own models between RAM and VRAM as they're needed, and in testing, keeping them loaded made no difference to its speed.

llama-server

llama-server -m qwen3.5_9b_qwen_image_2.1_pe_t2i.mtp.Q8_0.gguf -ngl 99 -fa on -c 24576 --spec-type draft-mtp
# i2i: add --mmproj qwen3.5_9b_qwen_image_2.1_pe_i2i.mmproj.bf16.gguf

Use the official system prompt from the source repository (system_prompt.txt) and the official sampling: temperature 1.0, top_k 20, top_p 0.95, min_p 0, presence penalty 1.5 (t2i) or 0 (i2i), with thinking on. The answer ends with a JSON object with rewritten_prompt (plus wh_ratio, and ratio_follow for i2i). MTP needs llama.cpp b11160 or newer.

License

These files are derived from the Qwen-Image 2.1 prompt enhancers and Qwen3.5-9B and are distributed under the Qwen Research License (see LICENSE), for non-commercial use only. See NOTICE.

Changes from the original models: the MTP head of Qwen/Qwen3.5-9B was added, mtp_num_hidden_layers was set to 1 in the config, the weights were converted to GGUF and quantized to Q8_0 and Q4_K_M, and the i2i vision encoder was converted to a separate BF16 GGUF.