---

base_model: prism-ml/Ternary-Bonsai-2-27B-gguf

base_model_relation: adapter

library_name: gguf

license: apache-2.0

license_link: https://huggingface.co/prism-ml/Ternary-Bonsai-2-27B-gguf/blob/main/LICENSE

pipeline_tag: text-generation

tags:

- gguf

- llama.cpp

- lora

- abliteration

- refusal-direction

- ternary

- bonsai

- prismml

- atomic-chat

---

How to Run Ternary Bonsai 2 27B Without Refusals Locally

<p style="margin-top: 0; margin-bottom: 0;">

<em>A 9.7 MB adapter, not a checkpoint. The 5.95 GB ternary pack stays byte-identical and the refusal direction is removed inside the compute graph, at a strength you set per request. The <a href="https://huggingface.co/datasets/AtomicChat/Ternary-Bonsai-2-27B-Abliterate-LoRA-GGUF-metrics">measurements</a> behind this card are public.</em>

</p>

<div style="display: flex; gap: 8px; align-items: center; margin-top: 10px; margin-bottom: 10px;">

<a href="https://atomic.chat/?utm_source=huggingface&utm_medium=referral&utm_campaign=hf_ternary_bonsai_2_27b_abliterate&utm_content=btn_atomic"><img src="https://huggingface.co/AtomicChat/Ternary-Bonsai-2-27B-Abliterate-LoRA-GGUF/resolve/main/btn_atomic.png" width="162" alt="Atomic Chat"></a>

<a href="https://discord.gg/8wGSsvmg4V"><img src="https://huggingface.co/AtomicChat/Ternary-Bonsai-2-27B-Abliterate-LoRA-GGUF/resolve/main/btn_discord.png" width="119" alt="Discord"></a>

<a href="https://github.com/AtomicBot-ai/Atomic-Chat"><img src="https://huggingface.co/AtomicChat/Ternary-Bonsai-2-27B-Abliterate-LoRA-GGUF/resolve/main/btn_github.png" width="115" alt="GitHub"></a>

</div>

<ul style="margin: 0 0 12px 0;">

<li>Load it with <code>--lora-scaled adapter.gguf:1.5</code>. Scale 0 gives the published model back, token for token, so one process serves both.</li>

<li>It needs PrismML's llama.cpp fork - stock llama.cpp and Ollama cannot open the base pack at all.</li>

<li>Every number on this card was measured inside the running model, not inferred from the adapter file.</li>

</ul>

<hr style="margin: 0 0 16px 0;">

Ternary-Bonsai-2-27B-Abliterate-LoRA-GGUF

A rank-1 LoRA adapter that removes a refusal direction from

prism-ml/Ternary-Bonsai-2-27B-gguf

at inference time, built and measured by AtomicChat. The base pack stays byte-identical;

nothing is requantized. The direction was estimated on the ternary pack itself, not on

the bf16 model it was trained from.

Why an adapter and not a model

W' = W - r (r^T W) is a dense matrix, but these weights are ternary with an FP16 scale

per 128. The edit is about 1.4% of ||W|| against a grid step of 1.7-2.4x a typical

weight, so baking it back into the pack rounds it away. llama.cpp builds a LoRA into the

graph as two extra matmuls and never merges it into the base weights, which is what makes

a rank-1 adapter an exact carrier of the edit at 1.75 bits/weight.

You need PrismML's llama.cpp fork

Stock llama.cpp cannot open the base pack at all: PTQ1_0 (143) and PQ2_0 (142) are

private ggml type ids, and the weights live in a Hadamard-rotated basis. Use

PrismML-Eng/llama.cpp. Built and verified

against tag prism-b10709-9a9394a.

Get the files

hf download prism-ml/Ternary-Bonsai-2-27B-gguf Ternary-Bonsai-2-27B-PTQ1_0.gguf --local-dir .   # or PQ2_0
hf download AtomicChat/Ternary-Bonsai-2-27B-Abliterate-LoRA-GGUF \
    bonsai2-27b-abliterate-own42-lora.gguf --local-dir .

Run it:

llama-cli -m Ternary-Bonsai-2-27B-PTQ1_0.gguf \
    --lora-scaled bonsai2-27b-abliterate-own42-lora.gguf:1.5

Or serve it, with the strength as a per-request knob:

llama-server -m Ternary-Bonsai-2-27B-PTQ1_0.gguf -ngl 99 --jinja \
    --lora bonsai2-27b-abliterate-own42-lora.gguf --lora-init-without-apply

curl localhost:8080/v1/chat/completions -H 'Content-Type: application/json' -d '{
  "messages": [{"role": "user", "content": "..."}],
  "lora": [{"id": 0, "scale": 1.5}]
}'

Use scale 1.5. At 1.0 the projection is exact but 14% of AdvBench-style prompts still

refuse; at 2.0 the model starts emitting empty replies. See the table below. Scale 0

reproduces the published model token for token, so one process can serve both behaviours.

Files

| file | sites | what |

|---|---|---|

| bonsai2-27b-abliterate-own42-lora.gguf | 129 | main adapter, direction from residual row 42 |

| bonsai2-27b-abliterate-own38-lora.gguf | 129 | same, row 38 |

| bonsai2-27b-abliterate-own42-layers15-63-lora.gguf | 98 | row 42, blocks 15-63 only, no embedding |

| directions/refusal_dir_own_projected.safetensors | - | [65, 5120] directions, the ones the adapters use |

| directions/refusal_dir_own_plain.safetensors | - | same before the harmless-mean correction (see caveat) |

| eval/*.json | - | raw eval summaries |

129 sites = 64 ffn_down + 48 ssm_out + 16 attn_output + token_embd. Each adapter

carries its provenance in its GGUF metadata (prism.abliterate.*), including the sha256

of the base pack it was derived from.

How the direction was estimated

Difference of means of the residual stream at the last prompt token (chat template

applied, thinking off), 416 harmful prompts from mlabonne/harmful_behaviors against 416

harmless prompts from mlabonne/harmless_alpaca. The held-out test splits were never used

for estimation. Row L of the direction file is the residual stream entering block L;

row 0 is the embedding output.

The naive difference of means over-refuses. It carries a component along the mean

harmless activation (cosine about -0.3), so zeroing the projection moves a harmless prompt

roughly 3.5 sigma toward the harmful cluster. Measured: the uncorrected row-38 direction

made the model answer "I cannot provide information about the fastest methods for illegal

activities" to a question about the fastest land mammal. The shipped directions subtract

that component, which puts an average harmless prompt back where it was (measured on the

held-out split). The uncorrected file is included only for reproduction.

Data provenance

No prompt set is redistributed here - only the recipe, so anyone can rebuild the same

splits from the sources.

| source | role | derived from | license |

|---|---|---|---|

| mlabonne/harmful_behaviors | 416 train / 104 held-out test | AdvBench (Zou et al. 2023, llm-attacks, MIT) | not stated on the dataset |

| mlabonne/harmless_alpaca | 416 train / 104 held-out test, sampled | tatsu-lab/alpaca | CC BY-NC 4.0 on the source |

| cais/mmlu | 500 questions over 57 subjects | original | MIT |

Splits are rebuilt by make_directions.py with sampling seed 0 and mmlu_run.py with

sampling seed 0. numpy.random.Generator does not guarantee an identical stream across

numpy releases, so verify you rebuilt the same subsets against these checksums (produced

with numpy 2.5.3):

038bd43f018f66617129f7bf80c5514f... test_harmful.jsonl

e640e1637beb3283985487412b684cea... test_harmless.jsonl

9859cc69902a2b5707afe54346034e8f... train_harmful.jsonl

087080f508babfd93553662fbaffca84... train_harmless.jsonl

Model outputs on the harmful split are not published; eval/ carries aggregates only.

Evaluation

104 harmful + 104 harmless held-out prompts, greedy, 64-token budget, thinking off, same

seed and system prompt everywhere. Refusal is a rule-based opening-phrase match, which is

indicative, not a judge. An empty or degenerate reply is counted as damage, not as

compliance.

| config | harmful: refused | empty/degenerate | harmless: refused |

|---|---|---|---|

| base | 99% | 0 | 0% |

| row 42, scale 1 | 14% | 0 | 0% |

| row 42, scale 1.5 | 0% | 0 | 0% |

| row 42, scale 2 | 0% | 5 empty | 0% |

| row 38, scale 1 | 34% | 0 | 0% |

| row 38, scale 1.5 | 0% | 0 | 0% |

| row 42, blocks 15-63, scale 1 | 14% | 0 | 0% |

Restricting the intervention to blocks 15-63 is consistently worse than covering every

site, so the lower blocks and the embedding carry part of the behaviour.

MMLU, 500 questions stratified over 57 subjects, answer forced to a single letter by a

root ::= [A-D] grammar, thinking off. Answer-only, so the absolute number sits below

PrismML's published thinking-mode result; what matters here is the difference.

| config | accuracy | flipped to wrong | flipped to right |

|---|---|---|---|

| base | 73.2% +- 2.0 | - | - |

| scale 1 | 72.6% | 12 | 9 |

| scale 1.5 | 73.0% | 15 | 14 |

| scale 2 | 72.6% | 26 | 23 |

Flips are symmetric, so there is no systematic loss of knowledge even at scale 2. The

damage at scale 2 shows up in open-ended generation on harmful prompts, not in knowledge.

Both packs

The same adapter file applies to PTQ1_0 and PQ2_0 unchanged, and was run on both.

The two packs carry identical tensor names, shapes and prism.hadamard.* metadata; both

decoders match the F16 pack bit for bit; an adapter rebuilt from PQ2_0 is bit-identical to

the shipped one, tensor for tensor. On PQ2_0 the leak along r falls to 3.6-4.0e-6 at

every writer and stays below 1.6e-5 in the residual stream, and all 416 greedy replies of

the refusal evaluation - base and scale 1.5 - are byte-identical to the PTQ1_0 run. The

packing changes how the trits are stored, not what they are.

A note on head order in ssm_out

The fork rotates the activation only for the base matmul and hands the LoRA branch the

raw activation, so A belongs in the unfolded input basis:

writers: A = P^T (s * H(r^T W_fold)), B = -r

token_embd: A = W_fold @ H(s * r), B = -r

P is the tiled to grouped V-head permutation that the fork applies to ssm_out when

prism.hadamard.gdn_v_grouped is set. An adapter whose ssm_out factors are left in the

checkpoint's (grouped) head order does not ablate those 48 sites. Verified by tapping the

residual writes inside the fork: with this adapter the component along r falls from

about 1e-2 to 3.8e-6 at every site, and the residual stream leak stays below 1.5e-5 across

all 64 blocks.

Limitations

• The F16 pack was not run. It carries the same tensor names, values and rotation metadata,

so the adapter should apply, but only PTQ1_0 and PQ2_0 were exercised.

• The refusal metric matches opening phrases only. A reply that answers with a disclaimer

counts as compliance.

• Capability was checked with MMLU alone. No math, code or agentic benchmark.
• The direction comes from one prompt-pair source at one token position. Transfer to other

phrasings and languages is unmeasured.

Responsible use

This adapter removes a learned refusal direction, so the model will answer requests the

published model declines. It is an inference-control and research artifact. Nothing about

it makes the resulting output safe, correct or lawful, and any deployment needs its own

access controls and policy enforcement.

Credits

Direction estimation, adapter build and evaluation by nik.bogatyrev.

Base model Ternary Bonsai 2 27B by prism-ml,

Apache-2.0, redistributed here in no form - this repository ships only the adapter.

The refusal-direction method follows Arditi et al., *Refusal in Language Models Is

Mediated by a Single Direction* (2024). The rank-1 LoRA formulation for this pack, and the

residual-writer inventory it targets, follow

OrcaRouter's OrcaBonsai-27B-Uncensored,

Apache-2.0. The directions shipped here are estimated independently on the ternary pack.