---

license: mit

base_model:

• inclusionAI/Ling-3.1-flash

base_model_relation: finetune

quantized_by: AtomicChat

pipeline_tag: text-generation

library_name: gguf

tags:

• atomic-chat
• ling
• inclusionai
• bailingmoe3
• moe
• gguf
• imatrix
• llama.cpp
• lora
• abliteration
• refusal-direction

---

How to Run Ling 3.1 flash Without Refusals Locally

<p style="margin-top: 0; margin-bottom: 0;">

<em>inclusionAI's refusal behaviour projected out of the released weights, baked into GGUF builds and measured against the original.</em>

</p>

<div style="display: flex; gap: 8px; align-items: center; margin-top: 10px; margin-bottom: 10px;">

<a href="https://atomic.chat/?utm_source=huggingface&utm_medium=referral&utm_campaign=hf_ling_3_1_flash_abliterate&utm_content=btn_atomic"><img src="https://huggingface.co/AtomicChat/Qwen3.8-27B-GGUF/resolve/main/btn_atomic.png" width="162" alt="Atomic Chat"></a>

<a href="https://discord.gg/8wGSsvmg4V"><img src="https://huggingface.co/AtomicChat/Qwen3.8-27B-GGUF/resolve/main/btn_discord.png" width="119" alt="Discord"></a>

<a href="https://github.com/AtomicBot-ai/Atomic-Chat"><img src="https://huggingface.co/AtomicChat/Qwen3.8-27B-GGUF/resolve/main/btn_github.png" width="115" alt="GitHub"></a>

</div>

<ul style="margin: 0 0 12px 0;">

<li>On held-out JailbreakBench prompts refusals fall from 96.3% to 0%, and with thinking on from 10 of 10 to none. MMLU stays where it was: 85.0% before, 85.8% after (inside the noise). Tool calls and long context are unaffected; no empty or looping replies.</li>

<li>The refusal sits in the attention outputs as much as in the experts: editing only the experts and the shared expert left most English refusals in place, so the edit covers every matrix that writes into the residual stream.</li>

<li>Same layout and importance matrix as our <a href="https://huggingface.co/AtomicChat/Ling-3.1-flash-GGUF">Ling 3.1 flash quants</a>; each file is the size of the original build with the same name.</li>

</ul>

<hr style="margin: 0 0 16px 0;">

The builds

| Build | On disk | Mean KLD | Same top-1 |

|---|---:|---:|---:|

| AD-Q4_K_M | 354.4 GB | 0.0154 | 96.22% |

| AD-IQ2_M | 218.1 GB | 0.0623 | 92.56% |

| AD-Q5_K_M | 400.2 GB | 0.0125 | 96.62% |

KLD and top-1 are against the original, unmodified BF16 model on our

held-out neutral text (144 chunks of 4096 tokens), so they hold the edit and the

quantization together. For scale, our plain AD-Q4_K_M of the original model is

0.0111 and 96.86%: the edit itself adds about 0.004.

Running it

hf download AtomicChat/Ling-3.1-flash-Uncensored-GGUF --include "AD-Q4_K_M/*" --local-dir .
llama-server -m AD-Q4_K_M/Ling-3.1-flash-Uncensored-AD-Q4_K_M-00001-of-00009.gguf -ngl 99 -c 32768 --jinja

The chat template is inside every file. Thinking is on by default; put

detailed thinking off in the system prompt to turn it off. Memory: the file

plus about 0.5 GB per conversation and 10 KB per token of context.

How it was made

1. Direction. Difference of means of the residual stream at the last prompt

token, thinking off, chat template applied: 544 harmful against 544 harmless

prompts, on the released weights. The

component along the harmless mean is removed. One direction, measured at the

input of block 43 of 72.

1. Edit. W' = W - 1.25 r rᵀW on all 213 matrices that write into the

residual stream: the attention output of every block (63 linear, 9 latent),

the down projections of the 4 dense blocks, of the shared expert and of all

512 experts in the 68 MoE blocks, and the token embeddings. The router and

the multi-token-prediction head are untouched.

1. Choice. The row, the set of edited matrices and the scale were picked in

a screen of variants on validation prompts (writer sets, rows, scales,

one direction per block, English-only directions), with edits of the same

shape along a random direction as the control.

1. Bake. The same edit applied to the BF16 weights in f32 and rounded once,

then quantized with the importance matrix and layout of our

original quants.

What changed, and what did not

Original against this edit on the released weights (Q8_0 with the edit as an

adapter). The refusal sets are held-out prompts never used to pick anything;

MMLU, the same 500 questions throughout.

| | Original | This edit | Random direction, same size |

|---|---:|---:|---:|

| JailbreakBench harmful behaviours (81), refused | 96.3% | 0.0% | 95.1% |

| XSTest safe prompts (250), refused | 2.8% | 0.4% | 2.8% |

| With thinking (10 prompts, 3072 tokens), refused | 10 of 10 | 0 of 10 | - |

| Thinking left unclosed at 3072 tokens | 0 of 10 | 0 of 10 | - |

| Empty or looping replies | 0 | 0 | 0 |

| MMLU, 500 questions | 85.0% | 85.8% (McNemar p = 0.48) | 85.0% |

| Tool calls (20) | 20/20 valid | 20/20 valid | - |

| Needle at 30k tokens (3 depths) | 3/3 | 3/3 | - |

The baked files themselves, on the same JailbreakBench and XSTest prompts:

AD-Q4_K_M refused 0.0% and 0.0%, AD-IQ2_M 2.5% and 0.0%.

Validation prompts (104 harmful, 104 harmless), replies of up to 64 tokens, the

check of the recipe on the released weights: 98.1% → 2.9% refused, none of the

harmless prompts, a random direction 97.1%.

Refusal is counted by the opening of the reply, so it is indicative, not a

judge.

Limitations

• MMLU on 500 questions and thinking on 10 prompts: enough to see a large

change, not a small one.

• The refusal count reads the opening of each reply; a soft refusal phrased as

an answer would be missed.

• Refusals are measured on English prompts.

Responsible use

This model has its learned refusal behaviour removed, so it will answer

requests the original declines. Nothing about it makes the output safe, correct

or lawful, and any deployment needs its own access controls and policy

enforcement.

License and credits

MIT, as the original. Base model Ling 3.1 flash by

inclusionAI. Direction, edit,

quantization and evaluation by AtomicChat. The method follows Arditi et

al., Refusal in Language Models Is Mediated by a Single Direction (2024).