---

license: mit

base_model: deepseek-ai/DeepSeek-V4-Flash-0731

tags:

- uncensored

- abliterated

- heretic

- lora

- gguf

- deepseek

---

DeepSeek-V4-Flash-0731 Heretic v2 — LoRA adapters

> ⚠️ Content warning: These adapters have had the base model's refusal

> behavior surgically suppressed. The resulting model will comply with

> requests the base model refuses, including requests that are harmful,

> unethical, offensive, or illegal. It has reduced safety guardrails. See

> Responsible use below — **you are solely

> responsible for what you do with it.**

This is a set of rank-1 LoRA adapters that decensor / "abliterate"

DeepSeek-V4-Flash-0731

(284B total / 13B active MoE, MIT license), produced with heretic-gguf —

a GGUF-native port of Heretic's

Optuna-optimized directional ablation, which runs the whole search directly

on quantized GGUF weights via llama.cpp.

v2 comes out of a fourth study (followup4) that re-scored every trial

on the full 140-prompt eval set. It improves on the

v1 adapter

at every point of the refusal/KL tradeoff. Four adapters are included,

all Pareto-optimal, so you can pick your own point on the ladder:

| File | Refusals (140 harmful prompts) | KL divergence |

|---|---|---|

| ds4-flash-heretic-f4-t265-lora.gguf | 8.57% (12/140) | 0.0651 |

| ds4-flash-heretic-f4-t256-lora.gguf | 7.14% (10/140) | 0.0845 |

| ds4-flash-heretic-f4-t273-lora.gguf | 5.00% (7/140) | 0.0927 |

| ds4-flash-heretic-f4-t264-lora.gguf | 3.57% (5/140) | 0.1399 |

Lower KL = less collateral damage to normal behavior; fewer refusals =

stronger decensoring. t265 is the conservative pick (v1-equivalent drift,

nearly half the refusals of v1's 20/140); t264 is the aggressive one.

This repository contains only the adapters. You need the base model

separately (any GGUF quant of DeepSeek-V4-Flash-0731 works — the adapter is

applied in f32/f16 compute regardless of the base quant; it was tuned and

evaluated against UD-Q8_K_XL). The LoRA form is the lossless option: the

base weights are never modified or requantized, and the download is ~50 MB

per adapter instead of ~160 GB. A merged, ready-to-run GGUF of the v1

configuration is at

MoriNoNushi/DeepSeek-V4-Flash-0731-heretic-GGUF.

**heretic-gguf is available at

github.com/MoriNoNushi/heretic-gguf** —

the full tool, so the method can be applied to other GGUF models.

Results

Measured on a held-out eval set of 140 harmful prompts (100 from

mlabonne/harmful_behaviors test + 40 custom) and 100 harmless prompts

(mlabonne/harmless_alpaca test), greedy decoding, 100-token responses,

against the UD-Q8_K_XL base:

| | Refusal rate (harmful) | KL divergence (harmless) |

|---|---|---|

| Base model | 99.29% (139/140) | 0 (by definition) |

| v1 adapter (followup3 t25) | 14.29% (20/140) | 0.0569 |

| v2 t265 | 8.57% (12/140) | 0.0651 |

| v2 t256 | 7.14% (10/140) | 0.0845 |

| v2 t273 | 5.00% (7/140) | 0.0927 |

| v2 t264 | 3.57% (5/140) | 0.1399 |

Refusals are counted by refusal-keyword matching (English + Chinese markers);

KL divergence is measured on first-token logits on harmless prompts. Unlike

v1 — which was selected on a cheap 70-prompt Stage A ranking and only

re-scored on the full set afterwards — **every trial in the v2 study was

scored on the full eval set from the start**, so these numbers are exactly

what the optimizer saw.

Usage

llama-server \
    -m DeepSeek-V4-Flash-0731-UD-Q8_K_XL-00001-of-00005.gguf \
    --lora ds4-flash-heretic-f4-t265-lora.gguf \
    --jinja

Add your usual offload/context flags (-ngl 999, -c, tensor splits,

etc.) — nothing model-specific is required, and no special sampling

parameters are needed. Simply omitting --lora restores the base model

exactly.

How it was made

• Method: directional ablation ("abliteration") — the refusal direction

in residual space (difference of means over 480 harmful / 480 harmless

prompts, orthogonalized against the harmless mean) is projected out of the

attention output and MoE down-projection weights. Strengths, layer kernel,

and direction index were tuned by multi-objective Optuna TPE (minimize

refusal rate and KL jointly) across a base study and four seeded follow-up

studies — ~2,100 trials total, the last ~280 of them scored on the full

eval set directly.

• Configurations (study followup4; all global direction scope,

direction index ~20.5–21.2 of 43 layers; per-expert strengths scaled by

measured harmful/harmless routing frequency):

- t265: attn max weight 4.96 @ layer 31; routed MLP 0.55; shared 1.46.

- t256: attn 3.46 @ 29; routed MLP off; shared 1.62.

- t273: attn 5.00 @ 27; routed MLP 0.61; shared 1.50.

- t264: attn 4.70 @ 28; routed MLP 0.58; shared 1.54.

• Why a LoRA: heretic-gguf expresses ablation as a rank-1 LoRA overlay,

the same math stock Heretic writes into PEFT adapters. Shipping the adapter

avoids requantizing the 162 GB base entirely — bit-identical base weights,

instant to apply. Each adapter embeds its full provenance (study, trial,

parameters, scores, commit hashes) as adapter.heretic.* GGUF metadata

keys; inspect with strings <file> | grep adapter.heretic.

Responsible use & disclaimer

• **These adapters can make the base model generate content that is

offensive, disturbing, hateful, sexually explicit, violent, or otherwise

objectionable, including detailed instructions for harmful or illegal

acts.** That is the direct and intended consequence of removing refusal

behavior.

• The ablation suppresses refusals, not the base model's knowledge —

outputs on dangerous topics may be wrong, hallucinated, or incoherent.

Nothing the model says should be treated as accurate, safe, or legal

advice.

• **Do not deploy models using these adapters in any production system,

public-facing service, or multi-user setting.** They are intended for

personal research, red-teaming, and evaluation purposes.

• **You, the user, are solely responsible for any output the model produces

and for any consequences of using these adapters.** The authors of this

release, of heretic-gguf, of Heretic, of Unsloth, and of DeepSeek accept

no liability whatsoever. Using these adapters to produce illegal content

or to harm others is your choice and your legal exposure — ensure your use

complies with all applicable laws in your jurisdiction.

• By downloading or using these adapters you acknowledge the above.

License

The base model is MIT-licensed (see the

base repo);

these adapters inherit those terms. The heretic-gguf tooling used to produce

them is AGPL-3.0-or-later.