---

license: mit

base_model: deepseek-ai/DeepSeek-V4-Flash-0731

tags:

- uncensored

- abliterated

- heretic

- lora

- gguf

- deepseek

---

DeepSeek-V4-Flash-0731 Heretic v2 — LoRA adapters

> ⚠️ Content warning: These adapters have had the base model's refusal

> behavior surgically suppressed. The resulting model will comply with

> requests the base model refuses, including requests that are harmful,

> unethical, offensive, or illegal. It has reduced safety guardrails. See

> Responsible use below — **you are solely

> responsible for what you do with it.**

This is a set of rank-1 LoRA adapters that decensor / "abliterate"

DeepSeek-V4-Flash-0731

(284B total / 13B active MoE, MIT license), produced with heretic-gguf —

a GGUF-native port of Heretic's

Optuna-optimized directional ablation, which runs the whole search directly

on quantized GGUF weights via llama.cpp.

v2 comes out of a fourth study (followup4) that re-scored every trial

on the full 140-prompt eval set (no cheap two-stage ranking — earlier

releases were selected on an optimistic cheap subset). It improves on the

v1 adapter

at every point of the refusal/KL tradeoff. Four adapters are included,

all Pareto-optimal, so you can pick your own point on the ladder:

| File | Refusals (140 harmful prompts) | KL divergence |

|---|---|---|

| ds4-flash-heretic-f4-t265-lora.gguf | 8.57% (12/140) | 0.0651 |

| ds4-flash-heretic-f4-t256-lora.gguf | 7.14% (10/140) | 0.0845 |

| ds4-flash-heretic-f4-t273-lora.gguf | 5.00% (7/140) | 0.0927 |

| ds4-flash-heretic-f4-t264-lora.gguf | 3.57% (5/140) | 0.1399 |

Each adapter also ships as a -r2 variant

(ds4-flash-heretic-f4-tNNN-lora-r2.gguf) — same scores, padded for faster

inference; see below.

Lower KL = less collateral damage to normal behavior; fewer refusals =

stronger decensoring. t265 is the conservative pick (v1-equivalent drift,

nearly half the refusals of v1's 20/140); t264 is the aggressive one.

This repository contains only the adapters. You need the base model

separately (any GGUF quant of DeepSeek-V4-Flash-0731 works — the adapter is

applied in f32/f16 compute regardless of the base quant; it was tuned and

evaluated against UD-Q8_K_XL). The LoRA form is the lossless option: the

base weights are never modified or requantized, and the download is ~50 MB

per adapter instead of ~160 GB. A merged, ready-to-run GGUF of the v1

configuration is at

MoriNoNushi/DeepSeek-V4-Flash-0731-heretic-GGUF.

> 2026-08 update — generation-speed fix: running these adapters on stock

> llama.cpp costs 25-40% generation speed (two llama.cpp issues, not the

> ablation itself). The repo now includes **-r2 variants of all four

> adapters** (zero-padded to rank 2, mathematically identical, same scores)

> plus a small llama.cpp patch that together recover most of the loss — and

> fix a hard crash on ROCm builds. See

> Generation speed & the -r2 adapters.

**heretic-gguf is available at

github.com/MoriNoNushi/heretic-gguf** —

the full tool, so the method can be applied to other GGUF models.

In initial hands-on testing the abliterated model shows **excellent general

capabilities** — responses remain coherent, detailed on normal tasks — and

it has not refused a prompt during personal testing. (Anecdotal, not a

benchmark; see the measured numbers below.)

Results

Measured on a held-out eval set of 140 harmful prompts (100 from

mlabonne/harmful_behaviors test + 40 custom) and 100 harmless prompts

(mlabonne/harmless_alpaca test), greedy decoding, 100-token responses,

against the UD-Q8_K_XL base:

| | Refusal rate (harmful) | KL divergence (harmless) |

|---|---|---|

| Base model | 99.29% (139/140) | 0 (by definition) |

| v1 adapter (followup3 t25) | 14.29% (20/140) | 0.0569 |

| v2 t265 | 8.57% (12/140) | 0.0651 |

| v2 t256 | 7.14% (10/140) | 0.0845 |

| v2 t273 | 5.00% (7/140) | 0.0927 |

| v2 t264 | 3.57% (5/140) | 0.1399 |

Refusals are counted by refusal-keyword matching (English + Chinese markers);

KL divergence is measured on first-token logits on harmless prompts. Unlike

v1 — which was selected on a cheap 70-prompt Stage A ranking and only

re-scored on the full set afterwards — **every trial in the v2 study was

scored on the full eval set from the start**, so these numbers are exactly

what the optimizer saw.

Usage

llama-server \
    -m DeepSeek-V4-Flash-0731-UD-Q8_K_XL-00001-of-00005.gguf \
    --lora ds4-flash-heretic-f4-t265-lora-r2.gguf \
    --jinja

Add your usual offload/context flags (-ngl 999, -c, tensor splits,

etc.) — nothing model-specific is required, and no special sampling

parameters are needed. Simply omitting --lora restores the base model

exactly.

Generation speed & the -r2 adapters

TL;DR: on stock llama.cpp the expert-covering adapters (all except

t256, which has routed MLP off) slow generation down by ~25-40%. Use the

-r2 files together with the patch below to get most of that back. For zero

overhead, use a merged GGUF instead of a runtime LoRA.

Cause (not the ablation — two llama.cpp issues)

1. LoRA adapter buffers are not tagged as weights in llama.cpp, so the

scheduler does not pin LoRA matmuls to the device of the corresponding

base tensor. With hybrid offloading (-ot ...=CPU), the expert-LoRA ops

get pulled onto the GPU anyway, dragging per-token host→device copies and

stream synchronizations with them.

1. NVIDIA has no fast kernel for non-quantized mul_mat_id. The mmvf

vector path is AMD-only, and the rank-1 LoRA shapes miss every other fast

path — so each expert-LoRA matmul takes a fallback that synchronizes the

stream twice per call and disables CUDA graphs for the whole decode

graph. (t256 has no routed-expert tensors, which is why it alone runs

at full speed.) On ROCm the same shapes are worse than slow: the AMD

mmvf path is entered unconditionally and its ncols % 2 == 0 assert

crashes on rank-1 lora_b — this is fixed by the -r2 files alone.

As of 2026-08 upstream master only fixes the blanket CUDA-graph disable

(PR #26802); both issues above are still present.

The -r2 adapters

ds4-flash-heretic-f4-tNNN-lora-r2.gguf are the same four adapters with the

LoRA rank zero-padded from 1 to 2 (alpha adjusted accordingly — the

applied delta is bit-for-bit the same math, verified to produce identical

generations). The even inner dimension keeps every LoRA matmul on the fast

CUDA kernel paths and fixes the ROCm crash. Scores are unchanged; use them

exactly like the originals.

The llama.cpp patch

llama_cpp_lora_speed.patch (in this repo,

~50 lines) does two things: tags adapter buffers as weights so LoRA ops

follow the base tensor's device placement, and enables the mmvf path for

non-quantized mul_mat_id on NVIDIA (with the CUDA-graph check refined to

match, same approach as upstream #26802). Apply and rebuild:

cd llama.cpp
git checkout c5a4a0bb832fcdb44487996150b3141490fdff69   # the commit the patch was made against
git apply llama_cpp_lora_speed.patch
cmake --build build --target llama-server llama-cli -j

It applies cleanly on the pinned commit above; on other commits expect to

re-validate (the touched code moves around). test-backend-ops -o MUL_MAT_ID

passes 790/790 with the patch.

Measured

DeepSeek-V4-Flash UD-Q8_K_XL, 4× GPU + partial CPU expert offload

(-ts 32,24,24,24, experts of layers 21+ on CPU), greedy decode:

| config | stock llama.cpp | + patch |

|---|---|---|

| base model | 11.3 t/s | 14.0 t/s |

| attn-only adapter (t256-like) | 11.0 t/s | 12.9 t/s |

| expert adapter, rank 1 | 8.5 t/s | 11.8 t/s |

| expert adapter, rank 2 (-r2) | 8.1 t/s | 11.6 t/s |

Qwen3.6-35B-A3B fully on one GPU with a worst-case 40-layer synthetic expert

adapter: base 244.6 t/s → rank 1 stock 154.0 t/s (-37%) → rank 2 + patch

199.3 t/s (-18%). Residual overhead is proportional to how many layers the

adapter touches; the v2 adapters touch ~15 of 43 layers. Run-to-run variance

is ~1 t/s.

If you want zero runtime overhead, merge the adapter into the weights

instead (heretic-gguf export --mode merged), e.g. the merged v1 release

linked above.

How it was made

• Method: directional ablation ("abliteration") — the refusal direction

in residual space (difference of means over 480 harmful / 480 harmless

prompts, orthogonalized against the harmless mean) is projected out of the

attention output and MoE down-projection weights. Strengths, layer kernel,

and direction index were tuned by multi-objective Optuna TPE (minimize

refusal rate and KL jointly) across a base study and four seeded follow-up

studies — ~2,100 trials total, the last ~280 of them scored on the full

eval set directly.

• Configurations (study followup4; all global direction scope,

direction index ~20.5–21.2 of 43 layers; per-expert strengths scaled by

measured harmful/harmless routing frequency):

- t265: attn max weight 4.96 @ layer 31; routed MLP 0.55; shared 1.46.

- t256: attn 3.46 @ 29; routed MLP off; shared 1.62.

- t273: attn 5.00 @ 27; routed MLP 0.61; shared 1.50.

- t264: attn 4.70 @ 28; routed MLP 0.58; shared 1.54.

• Why a LoRA: heretic-gguf expresses ablation as a rank-1 LoRA overlay,

the same math stock Heretic writes into PEFT adapters. Shipping the adapter

avoids requantizing the 162 GB base entirely — bit-identical base weights,

instant to apply. Each adapter embeds its full provenance (study, trial,

parameters, scores, commit hashes) as adapter.heretic.* GGUF metadata

keys; inspect with strings <file> | grep adapter.heretic.

Responsible use & disclaimer

• **These adapters can make the base model generate content that is

offensive, disturbing, hateful, sexually explicit, violent, or otherwise

objectionable, including detailed instructions for harmful or illegal

acts.** That is the direct and intended consequence of removing refusal

behavior.

• The ablation suppresses refusals, not the base model's knowledge —

outputs on dangerous topics may be wrong, hallucinated, or incoherent.

Nothing the model says should be treated as accurate, safe, or legal

advice.

• **Do not deploy models using these adapters in any production system,

public-facing service, or multi-user setting.** They are intended for

personal research, red-teaming, and evaluation purposes.

• **You, the user, are solely responsible for any output the model produces

and for any consequences of using these adapters.** The authors of this

release, of heretic-gguf, of Heretic, of Unsloth, and of DeepSeek accept

no liability whatsoever. Using these adapters to produce illegal content

or to harm others is your choice and your legal exposure — ensure your use

complies with all applicable laws in your jurisdiction.

• By downloading or using these adapters you acknowledge the above.

License

The base model is MIT-licensed (see the

base repo);

these adapters inherit those terms. The heretic-gguf tooling used to produce

them is AGPL-3.0-or-later.