---

license: apache-2.0

base_model:

• openbmb/MiniCPM-V-4.7-35B-A3B

base_model_relation: finetune

quantized_by: AtomicChat

pipeline_tag: image-text-to-text

library_name: gguf

tags:

• atomic-chat
• minicpm
• minicpm-v
• openbmb
• moe
• gguf
• imatrix
• llama.cpp
• lora
• abliteration
• refusal-direction

---

How to Run MiniCPM-V 4.7 35B-A3B Without Refusals Locally

<p style="margin-top: 0; margin-bottom: 0;">

<em>One refusal direction projected out of OpenBMB's original weights, shipped two ways: three baked GGUF builds, and a 106 MB LoRA adapter for the MiniCPM-V 4.7 quants you already have. The <a href="https://huggingface.co/datasets/AtomicChat/MiniCPM-V-4.7-35B-A3B-Uncensored-GGUF-metrics">measurements</a> behind this card are public.</em>

</p>

<div style="display: flex; gap: 8px; align-items: center; margin-top: 10px; margin-bottom: 10px;">

<a href="https://atomic.chat/?utm_source=huggingface&utm_medium=referral&utm_campaign=hf_minicpm_v_4_7_35b_a3b_abliterate&utm_content=btn_atomic"><img src="https://huggingface.co/AtomicChat/Qwen3.8-27B-GGUF/resolve/main/btn_atomic.png" width="162" alt="Atomic Chat"></a>

<a href="https://discord.gg/8wGSsvmg4V"><img src="https://huggingface.co/AtomicChat/Qwen3.8-27B-GGUF/resolve/main/btn_discord.png" width="119" alt="Discord"></a>

<a href="https://github.com/AtomicBot-ai/Atomic-Chat"><img src="https://huggingface.co/AtomicChat/Qwen3.8-27B-GGUF/resolve/main/btn_github.png" width="115" alt="GitHub"></a>

</div>

<ul style="margin: 0 0 12px 0;">

<li>On held-out prompts, refusals fall from 96.3% to 0% in English and from 70% to 0% in Russian, and from 13 of 30 to none with thinking on. MMLU moves by 0.25 points, inside the noise.</li>

<li>The 20.1 GB build matches the original's token choice 94.5% of the time; our quant of the unmodified model at the same size does 94.9%.</li>

<li>Already running our <a href="https://huggingface.co/AtomicChat/MiniCPM-V-4.7-35B-A3B-GGUF">MiniCPM-V 4.7 quants</a>? Add the adapter with <code>--lora-scaled</code> instead of downloading a new model.</li>

</ul>

<hr style="margin: 0 0 16px 0;">

The builds

| Build | On disk | Mean KLD | Same top-1 | Refused: EN / RU / XSTest |

|---|---|---|---|---|

| AD-Q4_K-IQ4_XS | 20.1 GB | 0.0232 | 94.47% | 0% / 0% / 0% |

| AD-Q5_K-Q4_K | 22.1 GB | 0.0191 | 94.84% | 0% / 0% / 0% |

| AD-Q6_K | 29.1 GB | 0.0109 | 96.15% | 0% / 0% / 0% |

AD-Q4_K-IQ4_XS is the one for a 24 GB card, with room for a long context; take

AD-Q6_K if it fits.

KLD and top-1 are against the original, unmodified BF16 model, so they hold

the ablation and the quantization together. For scale: the ablation alone, on

BF16 weights, is 0.0090; our plain Q8_0 of the original is 0.0051. The same

layouts of the unmodified model, from our quants:

| Layout | On disk | Unmodified: KLD / top-1 | This repo: KLD / top-1 |

|---|---|---|---|

| AD-Q4_K-IQ4_XS | 20.1 GB | 0.0197 / 94.94% | 0.0232 / 94.47% |

| AD-Q5_K-Q4_K | 22.1 GB | 0.0150 / 95.47% | 0.0191 / 94.84% |

| AD-Q6_K | 29.1 GB | 0.0061 / 97.18% | 0.0109 / 96.15% |

The names follow our original quants:

AD-<ffn_down>-<ffn_up> is what the routed experts got; attention, the shared

expert, the embedding and the output head stay at q8_0.

What changed, and what did not

Original against ablated, both BF16, on prompts never used to pick anything:

| | Original | Ablated |

|---|---|---|

| JBB harmful behaviours (EN, 81), refused | 96.3% | 0% |

| Aya red-teaming (RU, 100), refused | 70.0% | 0% |

| XSTest safe prompts (250), refused | 7.6% | 0% |

| Thinking on (30 prompts), refused | 43% | 0% |

| Thinking on, traces left unclosed at 3072 tokens | 16 of 30 | 0 of 30 |

| MMLU, 2000 questions | 84.25% | 84.50% (+0.25, McNemar p = 0.52) |

| Tool calls (20) | 20/20 valid | 20/20 valid |

| Needle in a long context (3 depths) | 3/3 | 3/3 |

| Mean KLD to the original, neutral / code | - | 0.0090 / 0.0095 |

Refusal is counted by the opening of the reply, so it is indicative, not a

judge; empty or degenerate replies count as damage, and there were none. The

baked builds were checked the same way on the same test prompts.

The adapter

lora/MiniCPM-V-4.7-35B-A3B-Uncensored-LoRA.gguf: rank 1, 106 MB, 120

sites. It applies to any GGUF of the original MiniCPM-V-4.7-35B-A3B weights,

including our published quants. Use scale 1.0:

llama-server -m MiniCPM-V-4.7-35B-A3B-AD-Q4_K-IQ4_XS.gguf -ngl 99 --jinja \
  --lora-scaled lora/MiniCPM-V-4.7-35B-A3B-Uncensored-LoRA.gguf:1.0

The adapter was measured on the BF16 weights (the table above); on a quant it

adds the same edit on top of that quant's own error.

Get started

• llama.cpp at or after 69f201a (MiniCPM-V 4.7 support, ggml-org/llama.cpp#29416).

Download one build and point -m at its first shard:

hf download AtomicChat/MiniCPM-V-4.7-35B-A3B-Uncensored-GGUF \
  --include "MiniCPM-V-4.7-35B-A3B-Uncensored-AD-Q4_K-IQ4_XS/*" --local-dir .

llama-server \
  -m MiniCPM-V-4.7-35B-A3B-Uncensored-AD-Q4_K-IQ4_XS/MiniCPM-V-4.7-35B-A3B-Uncensored-AD-Q4_K-IQ4_XS-00001-of-00011.gguf \
  --mmproj mmproj-MiniCPM-V-4.7-35B-A3B-Uncensored-F16.gguf \
  -ngl 99 -c 32768 --jinja

With --jinja the model's own chat template starts every answer in thinking

mode; turn it off per request with "chat_template_kwargs": {"enable_thinking": false}.

How it was made

1. Direction. Difference of means of the residual stream at the last prompt

token, chat template applied, thinking off: 416 harmful against 416 harmless

English prompts, taken entering block 28 of 40, with the harmless-mean

component removed.

1. Edit. W' = W - r rᵀW on the matrices that write into the residual: 40

attention and linear-attention outputs, the down projections of all 256

experts in all 40 MoE blocks, and 40 shared experts. The token embedding is

left alone: the vision projector copies rows of it.

1. Choice. Directions from four depths at four strengths, then narrower

writer sets, one direction per block, English only or English + Russian,

and a random direction as the control, all on validation prompts; the

numbers on this card come from held-out test prompts. The screen's own

pick (one direction per block, scale 0.75) drifted a little less on the

first token but still refused 5 of 30 prompts with thinking on, which this

template does by default, so this release uses the English direction at

scale 1.0, which refused none.

1. Bake. The same edit applied to the BF16 weights in f32 and rounded once;

every one of the 120 writers checked to keep none of its component along

r, within 7.6e-3 after bf16 rounding. Then quantized with the importance

matrix of our original quants,

which a 300-chunk matrix of the ablated model agrees with (cosine 0.999 or more).

Everything was measured against one reference, one corpus, one machine: the

original BF16 model's own logits over a held-out neutral set, 87 chunks at 4096

context, llama.cpp 8bb3cf0d, 2x RTX PRO 6000 (sm_120). The unmodified quants

above were measured on H100s against their own reference from the same weights;

our Q8_0 scores 0.0051 on both.

Limitations

• The refusal count reads the opening of each reply; a soft refusal phrased as

an answer would be missed.

• Vision was not tested after the edit. The mmproj files here are the

original's projector, unmodified.

• Not run yet on a Mac, in LM Studio or in Ollama.
• The direction comes from English prompts at one token position. Russian

refusals went with it; other languages were not measured.

Responsible use

This model has a learned refusal direction removed, so it will answer requests

the original declines. Nothing about it makes the output safe, correct or

lawful, and any deployment needs its own access controls and policy

enforcement.

Credits

Direction estimation, adapter build, quantization and evaluation by

AtomicChat. Base model MiniCPM-V 4.7 35B-A3B by

OpenBMB, under Apache-2.0. The method follows

Arditi et al., Refusal in Language Models Is Mediated by a Single Direction (2024).