---

license: other

license_name: openmdw-1.1

license_link: https://openmdw.ai/license/1-1/

base_model:

• nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16

base_model_relation: finetune

quantized_by: AtomicChat

pipeline_tag: image-text-to-text

library_name: gguf

tags:

• atomic-chat
• atomic-dynamic
• nvidia
• nemotron
• nemotron_h_moe
• moe
• gguf
• imatrix
• llama.cpp
• abliteration
• refusal-direction

---

How to Run NVIDIA Nemotron 3.5 Super VL 120B-A12B Without Refusals Locally

<p style="margin-top: 0; margin-bottom: 0;">

<em>NVIDIA's refusal behaviour projected out of the released weights, baked into Atomic Dynamic GGUF builds and measured against the original.</em>

</p>

<div style="display: flex; gap: 8px; align-items: center; margin-top: 10px; margin-bottom: 10px;">

<a href="https://atomic.chat/?utm_source=huggingface&utm_medium=referral&utm_campaign=hf_nemotron_3_5_super_uncensored&utm_content=btn_atomic"><img src="https://huggingface.co/AtomicChat/Qwen3.8-27B-GGUF/resolve/main/btn_atomic.png" width="162" alt="Atomic Chat"></a>

<a href="https://discord.gg/8wGSsvmg4V"><img src="https://huggingface.co/AtomicChat/Qwen3.8-27B-GGUF/resolve/main/btn_discord.png" width="119" alt="Discord"></a>

<a href="https://github.com/AtomicBot-ai/Atomic-Chat"><img src="https://huggingface.co/AtomicChat/Qwen3.8-27B-GGUF/resolve/main/btn_github.png" width="115" alt="GitHub"></a>

</div>

<ul style="margin: 0 0 12px 0;">

<li>On held-out JailbreakBench prompts refusals fall from 95.1% to 0%, on Russian red-teaming prompts from 76% to 2%, and with thinking on from 30 of 30 to 2 of 30. MMLU stays where it was: 85.05% before, 85.10% after (2000 questions, McNemar p = 1.0). Tool calls and long context are unaffected; no empty or looping replies.</li>

<li>The refusal is written into the residual stream by every kind of block: the edit covers the Mamba-2 outputs, the attention outputs, the shared experts and the latent projection through which all 512 routed experts write.</li>

<li>Same Atomic Dynamic layouts and importance matrix as our <a href="https://huggingface.co/AtomicChat/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-GGUF">Nemotron 3.5 Super quants</a>; each file is the size of the original build with the same name.</li>

</ul>

<hr style="margin: 0 0 16px 0;">

The builds

| Build | On disk | Mean KLD | Same top-1 |

|---|---:|---:|---:|

| AD-Q4_K_S | 86.9 GB | 0.0125 | 95.30% |

| AD-IQ3_XXS | 65.3 GB | 0.0397 | 91.90% |

| AD-Q2_0_L | 55.8 GB | 0.0763 | 88.94% |

KLD and top-1 are against the original, unmodified BF16 model on our held-out

neutral text (24 chunks of 4096 tokens), so they hold the edit and the quantization

together. For scale, our plain builds of the original model with the same layouts

are 0.0082, 0.0352 and 0.0722 (on 95 chunks): the edit itself adds about 0.004.

The baked files themselves, on the held-out prompts: AD-Q4_K_S refused 0 of 81

English, 3 of 100 Russian and 1 of 250 XSTest prompts; AD-IQ3_XXS 0, 2 and 1;

AD-Q2_0_L 0, 5 and 0.

The vision projector (mmproj-*.gguf) is the original model's, unchanged; image

input was not tested on these files.

Running it

hf download AtomicChat/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-Uncensored-GGUF \
  --include "NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-Uncensored-AD-IQ3_XXS/*" --local-dir .
llama-server -m NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-Uncensored-AD-IQ3_XXS/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-Uncensored-AD-IQ3_XXS-00001-of-00043.gguf \
  -ngl 99 -c 32768 --jinja

The chat template is inside every file. Upstream llama.cpp with nemotron_h_moe

support runs them; no patch is needed.

How it was made

1. Direction. Difference of means of the residual stream at the last prompt

token, thinking off, chat template applied: 416 harmful against 416 harmless

English prompts and the same in Russian, on the released weights. The

component along the harmless mean is removed. One direction, measured at the

input of block 66 of 88.

1. Edit. W' = W - r rᵀW on the 128 matrices that write into the residual

stream: the output projection of all 40 Mamba-2 blocks and of the 8 attention

blocks, the down projection of the 40 shared experts, and the 40 latent up

projections through which every routed expert writes. The router, the token

embeddings and the multi-token-prediction head are untouched.

1. Choice. The recipe was set from a rehearsal on a smaller model of the same

family and confirmed here against an alternative (block 50) and a random

direction of the same size as the control, on validation prompts and then on

every check below.

1. Bake. The same edit applied to the BF16 weights in f32 and rounded once,

then quantized with the importance matrix and the layouts of our

original quants.

What changed, and what did not

Original against this edit on the released weights (BF16, the edit applied as an

adapter). The refusal sets are held-out prompts never used to pick anything.

| | Original | This edit | Random direction, same size |

|---|---:|---:|---:|

| JailbreakBench harmful behaviours (81), refused | 95.1% | 0.0% | 93.8% |

| Russian red-teaming prompts (100, aya), refused | 76.0% | 2.0% | 75.0% |

| XSTest safe prompts (250), refused | 5.2% | 0.4% | 4.0% |

| With thinking (30 prompts, 3072 tokens), refused | 30 of 30 | 2 of 30 | - |

| Thinking left unclosed at 3072 tokens | 0 of 30 | 0 of 30 | - |

| Empty or looping replies | 0 | 0 | 0 |

| MMLU, 2000 questions | 85.05% | 85.10% (McNemar p = 1.0) | 85.10% |

| Tool calls (20) | 20/20 valid | 20/20 valid | - |

| Needle at 30k tokens (3 depths) | 3/3 | 3/3 | - |

Validation prompts (104 harmful, 104 harmless per language), replies of up to 64

tokens, the check of the recipe on the released weights: 1.9% of English and 3.0%

of Russian harmful prompts refused.

Refusal is counted by the opening of the reply, so it is indicative, not a

judge.

Limitations

• With thinking on, 2 of 30 prompts were still refused; our gate allows fewer.

We publish because every other check passed and the difference from zero is

inside the noise of 30 prompts, but the thinking mode is not fully cleared.

• The refusal count reads the opening of each reply; a soft refusal phrased as

an answer would be missed.

• MMLU and the thinking check use fixed question sets; they show a large change,

not a small one.

• We publish the baked files only. The same edit as a LoRA adapter needs a

llama.cpp patch on this architecture (upstream applies no LoRA to the latent

projections), so it is not offered here.

Responsible use

This model has its learned refusal behaviour removed, so it will answer

requests the original declines. Nothing about it makes the output safe, correct

or lawful, and any deployment needs its own access controls and policy

enforcement.

License and credits

OpenMDW-1.1, as the original. Base model Nemotron 3.5 Super VL 120B-A12B by

NVIDIA. Direction, edit, quantization and

evaluation by AtomicChat. The method follows Arditi et al., *Refusal in

Language Models Is Mediated by a Single Direction* (2024).