---

license: other

license_name: nvidia-open-model-agreement

license_link: https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-open-model-agreement/

base_model: nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-BF16

tags:

• nemotron
• nemotron-3
• nano
• omni
• reasoning
• mamba2
• moe
• hybrid
• gguf
• llama.cpp
• ollama
• text-only
• q4_k_m
• 24gb
• cpu-offload

language:

• en

pipeline_tag: text-generation

library_name: gguf

base_model_relation: quantized

---

Nemotron-3-Nano-Omni-30B-A3B-Reasoning — Text-Only GGUF

Text-only GGUF conversion of nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-BF16, released by NVIDIA on 2026-04-28.

This conversion strips the vision (CRADIO v4-H) and audio (Parakeet) encoders and packages only the language model core for use with llama.cpp and Ollama. The text core is a Mamba2-Transformer hybrid Mixture-of-Experts (30B total, 3B active) with reasoning tuning.

What this is — and what it isn't

• ✅ The full text-reasoning capability of Nemotron-3-Nano-Omni, in a single text-only GGUF.
• ✅ Identical text weights to the omni release (extracted from language_model.* tensors, prefix stripped, vision/audio dropped).
• ❌ Not multimodal. The GGUF cannot accept images, audio, or video. For full omni capability, use NVIDIA's official BF16 / FP8 / NVFP4 weights with a transformers-compatible runtime — multimodal heads in llama.cpp would require an upstream PR adding support for CRADIO + Parakeet.
• ⚠ Distinct from nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16. That's NVIDIA's text-only sister model. Our weights come from the omni variant's text encoder, which was co-trained with the multimodal heads. Behavior may differ slightly from the standalone text-only release.

Architecture

| Field | Value |

|---|---|

| Architecture | Mamba2-Transformer hybrid MoE (NemotronHForCausalLM) |

| Hybrid pattern | MEMEMEMEMEMEMEMEMEMEMEMEMEMEMEMEMEMEMEMEMEMEME |

| Parameters | ~30B total, ~3B active per token |

| Hidden size | 2688 |

| Layers | 52 |

| Mamba heads | 64 |

| Attention heads | 32 (head_dim 128) |

| Routed experts | 128 |

| Shared experts | 1 |

| Top-k routing | 6 |

| Vocab | 131,072 |

| Context | 32K (per chat_template.jinja) |

Quants

This is a 30B-A3B MoE — only 3B params active per token. MoE architectures hold quality well at lower bit widths because routing isolates each token's compute to a small fraction of the model. The Q4_K_M default is the accessible end here; sub-Q4 quants would be wasted disk for quality lost on a model this sparse.

| Quant | Size | Use case |

|---|---|---|

| Q4_K_M | ~17 GB | recommended default — accessible end, runs on consumer hardware |

| Q5_K_M | ~21 GB | bump quality if you have headroom |

| Q6_K | ~25 GB | near-lossless reasoning |

| Q8_0 | ~32 GB | reference quality |

| F16 | ~60 GB | full precision (uploaded on request — useful for further quantization) |

(Sizes approximate — actual sizes confirmed once conversion completes. 30B MoE means total params, not active — disk size scales with total.)

Usage — Ollama

# Pull a quant
huggingface-cli download Hob-forge/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-GGUF \
    Nemotron-3-Nano-Omni-30B-A3B-Reasoning-text-only.Q4_K_M.gguf \
    --local-dir ./nemotron-omni

# Build local Ollama model
cd ./nemotron-omni
cat > Modelfile <<EOF
FROM ./Nemotron-3-Nano-Omni-30B-A3B-Reasoning-text-only.Q4_K_M.gguf
PARAMETER num_ctx 32768
PARAMETER temperature 0.6
PARAMETER top_p 0.95
PARAMETER repeat_penalty 1.05
PARAMETER stop "<|im_end|>"
PARAMETER stop "<|endoftext|>"
EOF
ollama create nemotron-3-omni-text:Q4_K_M -f Modelfile

# Use
ollama run nemotron-3-omni-text:Q4_K_M

A Modelfile is included in this repo.

Reasoning toggle

The chat template supports thinking-mode toggles via tokens in user messages:

• /think — enable thinking
• /no_think — suppress thinking

You can also pass "think": false at the top level of /api/chat (NOT inside options) to suppress thinking via the API.

Usage — llama.cpp

./build/bin/llama-cli \
    -m Nemotron-3-Nano-Omni-30B-A3B-Reasoning-text-only.Q4_K_M.gguf \
    -c 32768 \
    -p "Explain MoE routing in three sentences." \
    -n 256 \
    --temp 0.6

License

Use is governed by the NVIDIA Open Model Agreement. Commercial use is permitted under the agreement's terms. This conversion is a derivative work — same license applies.

Conversion details

• Source: nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-BF16 downloaded 2026-04-28
• Tools: llama.cpp (commit 1a635cde0) convert_hf_to_gguf.py with the existing NemotronHForCausalLM registration (no patches required to the converter)
• Steps: extracted language_model.* tensors → stripped prefix → wrote clean NemotronHForCausalLM config → ran converter → quantized

Limitations & caveats

• Text-only. Drop the omni weights if you need vision/audio.
• Hybrid Mamba2-Attention layers. Some llama.cpp/Ollama features that assume pure-attention models may behave unexpectedly (e.g. context shifting). Standard generation works fine.
• Brand-new architecture. Released the same day as this conversion. Expect rough edges; please open issues at the discussions tab.
• Quant quality on hybrid models. Mamba2 layers may be more quant-sensitive than pure attention. If you see degraded reasoning at Q4_K_M, try Q5_K_M or Q6_K.

Acknowledgments

• NVIDIA for the open release of Nemotron-3-Nano-Omni
• The llama.cpp team for NemotronHForCausalLM support
• This conversion produced for the Zenith swarm — autonomous engineering collective project

Citation

If you use this GGUF, please cite NVIDIA's original release:

@misc{nvidia2026nemotron3nanoomni,
  title  = {Nemotron-3-Nano-Omni-30B-A3B-Reasoning},
  author = {NVIDIA},
  year   = {2026},
  month  = {April},
  url    = {https://huggingface.co/nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-BF16}
}