---

license: other

license_name: openmdw-1.1

base_model: nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16

pipeline_tag: text-generation

library_name: gguf

tags:

• gguf
• llama.cpp
• uncensored
• abliterated
• moe
• mamba

language:

• en
• zh

---

Nemotron-3.5-Lightning-30B-A3B-Uncensored-xCloud (GGUF)

繁體中文 ｜ English below

由云碩科技（xCloudinfo）以 nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16

為基礎，移除其過度拒絕傾向後所產生的語言模型，並轉為 llama.cpp 可用的 GGUF 量化格式。

本模型在云碩自有 AI 算力資源池（xCloud 算力中心）上完成處理與量化。

這是什麼

• 基礎模型：Nemotron 3.5 Lightning 30B-A3B（混合架構：Mamba-2 + MoE + Attention，MoE 總量 30B、激活約 3B，

含 Multi-Token Prediction，上下文最長 1M，授權 OpenMDW-1.1，可商用）。

• 處理方式：方向消融（directional ablation / abliteration），非重新訓練。依 Arditi et al. (2024)，

將「拒絕方向」從殘差寫入矩陣（attention 的 o_proj 與所有 MoE 專家的 down_proj，共 2,973 個矩陣）正交化移除，

強度 0.8。Mamba(SSM) 的 out_proj 與 MTP 草稿頭保持原狀，以維持混合架構的連貫性。

- 註：混合／SSM 模型收「拒絕方向」時必須單一裝置載入，跨多卡切分會破壞 SSM 遞迴而得到壞方向；本模型於單機統一記憶體上處理。

版本對照

| 量化 | 檔案大小 | 說明 |

|---|---|---|

| Q8_0 | 33 GB | 近乎無損 |

| Q6_K | 33 GB | 高品質（見下方註）|

| Q5_K_M | 26 GB | 品質與體積平衡 |

| Q4_K_M | 24 GB | 一般部署建議 |

| IQ4_XS | 18 GB | 以 importance matrix 量化 |

| IQ2_M | 18 GB | 最小可用，以 importance matrix 量化（見下方註）|

另附 imatrix.dat（量化用的 importance matrix）。

> 註（MoE 量化特性）：本模型 hidden 維度為 2688，非 256 的整數倍，部分專家張量在量化時會回退到較高位元，

> 因此 Q8_0 與 Q6_K、IQ4_XS 與 IQ2_M 的體積相近。各版本皆可正常使用；追求最小體積請選 IQ4_XS 或 IQ2_M，

> 追求品質請選 Q8_0 或 Q6_K。

使用方式（llama.cpp）

llama-server -m Nemotron-3.5-Lightning-30B-A3B-Uncensored-xCloud-Q4_K_M.gguf \
  --jinja -ngl 99 -c 8192

• 思考預算：本模型的 reasoning（thinking）預設開啟，且屬「預算問題」而非開關問題。要求事實正確的回答時，

請給足 max_tokens（建議 ≥ 2000），思考才收斂、答案才會出現；避免使用 medium_effort（思考更長且不收斂）。

• 注入當前日期：在 system prompt 明確給出今天日期，否則模型的年份會預設為較早的年份。

授權與責任

• 授權：OpenMDW-1.1（沿用基礎模型；Linux Foundation 制定，可商用，散布時保留授權書與著作權聲明，對產出無限制）。
• 本模型移除了安全對齊層的拒絕行為，可能對敏感或雙用途請求直接作答。使用者須自行負責合法、合規、合乎倫理地使用本模型。

云碩不對本模型的輸出或其後續使用承擔責任。

• 本模型為內部研發／技術驗證用途。

---

<a name="english"></a>

Nemotron-3.5-Lightning-30B-A3B-Uncensored-xCloud (GGUF) — English

A language model produced by xCloudinfo, based on

nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16,

with its over-refusal behaviour removed, converted to llama.cpp GGUF quantizations. Processing and quantization

were performed on xCloudinfo's own AI compute pool.

What this is

• Base model: Nemotron 3.5 Lightning 30B-A3B (hybrid: Mamba-2 + MoE + Attention, 30B total / ~3B active,

with Multi-Token Prediction, up to 1M context, OpenMDW-1.1 license, commercial use permitted).

• Method: directional ablation (abliteration), not retraining. Following Arditi et al. (2024), the refusal

direction is orthogonalized out of the residual-writing matrices (attention o_proj and every MoE expert's

down_proj; 2,973 matrices) at strength 0.8. The **Mamba (SSM) out_proj and the MTP draft head are left

intact** to preserve the coherence of the hybrid architecture.

- Note: for hybrid/SSM models the refusal direction must be collected with the model on a single device;

splitting across GPUs corrupts the SSM recurrence and yields a bad direction. This model was processed on a

single unified-memory device.

Versions

| Quant | Size | Notes |

|---|---|---|

| Q8_0 | 33 GB | near-lossless |

| Q6_K | 33 GB | high quality (see note) |

| Q5_K_M | 26 GB | quality/size balance |

| Q4_K_M | 24 GB | recommended for deployment |

| IQ4_XS | 18 GB | importance-matrix quantized |

| IQ2_M | 18 GB | smallest usable, importance-matrix quantized (see note) |

Also included: imatrix.dat.

> Note (MoE quantization): this model's hidden size is 2688, not a multiple of 256, so some expert tensors fall

> back to higher bit-widths during quantization. As a result Q8_0/Q6_K and IQ4_XS/IQ2_M are close in size. All

> variants work normally; pick IQ4_XS/IQ2_M for the smallest footprint, Q8_0/Q6_K for the highest quality.

Usage (llama.cpp)

llama-server -m Nemotron-3.5-Lightning-30B-A3B-Uncensored-xCloud-Q4_K_M.gguf \
  --jinja -ngl 99 -c 8192

• Thinking budget: reasoning is on by default and is a budget matter, not an on/off switch. For factually

correct answers, give enough max_tokens (>= 2000) so the thinking converges and the answer appears; avoid

medium_effort (longer, non-converging thinking).

• Inject the current date in the system prompt; otherwise the model defaults the year to an earlier value.

License and responsibility

• License: OpenMDW-1.1 (inherited from the base model; Linux Foundation, commercial use permitted, keep the

license and copyright notice when redistributing, no restrictions on outputs).

• This model has had its safety-alignment refusal behaviour removed and may respond directly to sensitive or

dual-use requests. Users are solely responsible for using it lawfully, in compliance, and ethically.

xCloudinfo assumes no responsibility for the outputs or their downstream use.

• Released for internal research and technical validation.