DrMoriarty0/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-GGUF overview
mAPEX Quantized Models Source Data The source data for quantization was taken from bartowski/NVIDIA Nemotron 3.5 Lightning 30B A3B GGUF https://huggingface.co/…
Runs locally from ~52.8 MB disk (4 GB VRAM class GPUs with llama.cpp / guIDE).
Repository Files & Downloads
| File | Type | Quantization | Size | Link |
|---|---|---|---|---|
| NVIDIA-Nemotron-3.5-Lightning-30B-A3B-Tier15-s.gguf | GGUF | GGUF | 18.00 GB | Download |
| NVIDIA-Nemotron-3.5-Lightning-30B-A3B-Tier15.gguf | GGUF | GGUF | 18.00 GB | Download |
| NVIDIA-Nemotron-3.5-Lightning-30B-A3B-bf16.gguf | GGUF | BF16 | 61.33 GB | Download |
| NVIDIA-Nemotron-3.5-Lightning-30B-A3B-imatrix.gguf | GGUF | GGUF | 52.8 MB | Download |
Model Details
| Model ID | DrMoriarty0/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-GGUF |
|---|---|
| Author | DrMoriarty0 |
| Pipeline | — |
| License | openmdw-1.1 |
| Base model | nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16 |
| Last modified | 2026-10-02T15:51:36.000Z |
Model README
---
license: openmdw-1.1
base_model:
- nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16
---
mAPEX Quantized Models
Source Data
The source data for quantization was taken from bartowski/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-GGUF.
Original source file: NVIDIA-Nemotron-3.5-Lightning-30B-A3B-bf16/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-bf16-00001-of-00002.gguf
Quantization Method
All quants were produced using the modified APEX quantization scheme.
mAPEX (modified Automated Precision EXpert allocation) assigns per-layer, per-tensor precision
for MoE models using llama.cpp's --tensor-type-file.
For more information, see: <https://github.com/DrMoriarty/apex-quant/>
Quantized Models
> Note: TierN-s quants are typically slightly larger than their regular
> counterparts, but provide roughly 10–20% faster inference (highly dependent
> on the model and GPU).
| Name | Size (GB) | Comments |
|------|-----------|----------|
| NVIDIA-Nemotron-3.5-Lightning-30B-A3B-Tier15.gguf | 18.00 | |
| NVIDIA-Nemotron-3.5-Lightning-30B-A3B-Tier15-s.gguf | 18.00 | |
Run DrMoriarty0/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-GGUF with guIDE
Download guIDE — the AI-native code editor with local LLM inference and 69 built-in tools.
Source: Hugging Face · Compare models