---

license: apache-2.0

license_link: https://huggingface.co/Qwen/Qwen3.5-4B/blob/main/LICENSE

base_model:

• darrellbest/Qwen3.5-4B-Heretic

base_model_relation: quantized

pipeline_tag: image-text-to-text

tags:

• gguf
• llama.cpp
• heretic
• ara
• abliterated
• uncensored
• qwen3.5

---

Qwen3.5-4B-Heretic — GGUF

GGUF builds of darrellbest/Qwen3.5-4B-Heretic. Qwen/Qwen3.5-4B with its refusal behaviour removed by Heretic using full-weight Arbitrary-Rank Ablation: 6/100 refusals (original: 99/100) at KL divergence 0.0220. See the main repository for how it was made and measured.

| File | Quant | Size |

|---|---|---:|

| Qwen3.5-4B-Heretic-BF16.gguf | BF16 (lossless) | 8.67 GB |

| Qwen3.5-4B-Heretic-Q8_0.gguf | Q8_0 | 4.61 GB |

| Qwen3.5-4B-Heretic-Q4_K_M.gguf | Q4_K_M | 2.78 GB |

| Qwen3.5-4B-Heretic-mmproj-F16.gguf | vision projector (F16) | 0.67 GB |

Converted from the bf16 safetensors with llama.cpp's convert_hf_to_gguf.py and quantized with llama-quantize

(no imatrix). The mmproj file carries the vision encoder; load it alongside any of the three for image input.

Checked

Every file was loaded in llama-server with the mmproj and in Ollama. All three answered ordinary prompts correctly (BF16 and Q8_0 reproduce each other word for word), described a test image (a red circle and a blue square) correctly, and in Ollama answered 17 x 23 = 391 with thinking off. Thinking-mode reasoning, 4 arithmetic and word problems x 10 seeds with Qwen's recommended sampling: BF16 39/40, Q8_0 40/40, Q4_K_M 40/40 finished and correct (the original model in vLLM: 40/40).

Use

llama-server -m Qwen3.5-4B-Heretic-Q8_0.gguf --mmproj Qwen3.5-4B-Heretic-mmproj-F16.gguf --jinja -ngl 99

Thinking is on by default; pass "chat_template_kwargs": {"enable_thinking": false} (llama.cpp) or think: false

(Ollama) to turn it off per request.

> Reduced safety guardrails by design. You are responsible for what you do with it.

The family

| Repository | Format | Size | Use it with |

|---|---|---|---|

| Qwen3.5-4B-Heretic | bf16 safetensors | 9.35 GB | transformers, vLLM, SGLang |

| Qwen3.5-4B-Heretic-GGUF | GGUF BF16 / Q8_0 / Q4_K_M + vision mmproj | 8.67 / 4.61 / 2.78 GB + 0.67 GB | llama.cpp, Ollama |

| Qwen3.5-4B-Heretic-FP8 | FP8 W8A8, compressed-tensors | 6.79 GB | vLLM |

| Qwen3.5-4B-Heretic-NVFP4 | NVFP4, compressed-tensors | 5.67 GB | vLLM on Blackwell |