---

license: other

license_name: nvidia-open-model-license

license_link: https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-open-model-license/

language:

• en
• zh

library_name: gguf

pipeline_tag: text-generation

base_model: nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16

base_model_relation: quantized

tags:

• gguf
• llama.cpp
• nemotron-h
• mamba2
• moe
• abliterated
• uncensored
• crack
• reasoning
• mtp

---

<p align="center">

<img src="dealign_logo.png" alt="Dealign.ai" width="180"/>

<br/><strong><a href="https://dealign.ai">Dealign.ai</a></strong>

</p>

Nemotron-3.5-Lightning-30B-A3B-CRACK-GGUF

CRACK-abliterated NVIDIA Nemotron 3.5 Lightning 30B-A3B — GGUF quants for llama.cpp. Three

quantizations (Q8_0 / Q4_K_M / Q2_K) in one repository, each with the native MTP (Multi-Token

Prediction) draft head folded in for speculative decoding. Refusal behavior removed while preserving

the model's knowledge, reasoning (thinking), and multilingual ability.

> Research artifact with reduced safety guardrails. Use responsibly and lawfully.

Model

• Base: nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B — hybrid Mamba-2 SSM + MoE + attention

(52 layers: 23 Mamba-2 / 23 MoE / 6 attention), 128 routed experts (~3B active), 262K context,

reasoning (thinking) ON by default, Multi-Token-Prediction head.

• CRACK: early decision-zone abliteration with per-layer refusal directions. Knowledge preserved;

the native MTP draft head and reasoning are kept intact.

Quantizations

| File | Size | Notes |

|---|---|---|

| Nemotron-CRACK-Q8_0.gguf | 35.0 GB | near-lossless reference |

| Nemotron-CRACK-Q4_K_M.gguf | 25.4 GB | balanced (recommended) |

| Nemotron-CRACK-Q2_K.gguf | 18.7 GB | smallest; importance-matrix (imatrix) calibrated |

All three include the folded native MTP block (blk.52.nextn.*) for speculative decoding.

Benchmarks

MMLU is logit-mode accuracy (base vs. CRACK — measures knowledge retention). HarmBench is

answer-channel compliance on harm behaviors, counting only coherent responses (gibberish/degenerate

outputs do not count as compliant).

| Metric | Base | This model |

|---|---|---|

| MMLU (logit mode, 1000Q) | 78.3% | 74.2% (Δ -4.1 pp) |

| HarmBench-240 compliance | refuses | 100% (240/240, 0 refusals, 0 gibberish) |

Knowledge is largely retained (overall Δ ~-4pp; STEM/Humanities most affected). Refusals are removed on

the answer channel while outputs remain coherent and on-task.

HarmBench compliance by category (CRACK)

Standard HarmBench 240-behavior test set, answer-channel compliance, coherent-only (gibberish excluded):

| Category | Comply | Rate |

|---|---|---|

| chemical biological | 42/42 | 100.0% |

| cybercrime intrusion | 52/52 | 100.0% |

| harassment bullying | 21/21 | 100.0% |

| harmful | 18/18 | 100.0% |

| illegal | 53/53 | 100.0% |

| misinformation disinformation | 54/54 | 100.0% |

| All | 240/240 | 100.0% |

MMLU by topic (base vs CRACK, same vLLM harness, 1000Q logit-mode)

| Category | Base | CRACK | Δ |

|---|---|---|---|

| STEM | 74.9% | 68.2% | -6.7 |

| Humanities | 74.1% | 67.6% | -6.5 |

| Social Sciences | 85.4% | 83.1% | -2.3 |

| Other | 80.8% | 80.8% | 0.0 |

| All | 78.3% | 74.2% | -4.1 |

Usage (llama.cpp)

# recommended sampling: temperature 1.0, top_p 0.95 (NVIDIA defaults). Reasoning (thinking) is ON by default.
llama-cli -m Nemotron-CRACK-Q4_K_M.gguf --temp 1.0 --top-p 0.95 -p "<your prompt>"

# or serve (OpenAI-compatible):
llama-server -m Nemotron-CRACK-Q4_K_M.gguf --jinja --temp 1.0 --top-p 0.95 -c 8192

Reasoning

The model emits a <think>…</think> reasoning block followed by the answer. Thinking is **ON by

default**; disable it with the chat template's enable_thinking=false for direct answers.

Native MTP (speculative decoding)

The MTP draft head is folded into every quant, so llama.cpp can use it for self-speculative decoding

where supported — no separate draft model required.

Safety

Refusal behavior has been removed; this model will follow instructions it would normally decline.

Intended for research and authorized red-teaming. You are responsible for your use.

Attribution

Base model © NVIDIA, released under the NVIDIA Open Model License. This is a derivative quantization.