---

license: other

license_name: openmdw-1.1

license_link: https://developer.nvidia.com/open-model-license

base_model: nvidia/NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-BF16

tags:

• gguf
• llama.cpp
• nemotron
• nemotron-h
• mixture-of-experts
• mamba
• mtp
• quantized

---

NVIDIA Nemotron Labs 3 Puzzle 75B-A9B GGUF

Community GGUF conversion of nvidia/NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-BF16.

Important llama.cpp compatibility notice

This model was converted with the still-unmerged ggml-org/llama.cpp PR #25444, pinned to commit af49ef5cd990d039dbf360dd3a9f3b5dafdd1726, plus a narrowly scoped converter compatibility fix for the official BF16 checkpoint's model.layers.* tensor prefix and bounded writeback for large lazy tensors on the high-RAM Colab runtime. Until equivalent support is merged into mainline llama.cpp, use a build containing PR #25444 to load these files.

PR #25444 adds NemotronHPuzzleForCausalLM / nemotron_h_puzzle support, heterogeneous per-layer MoE settings, and the model's two-block MTP draft head. This repository is not an official NVIDIA or llama.cpp release.

Files

• NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-BF16.gguf — BF16 master GGUF
• NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-Q2_K.gguf
• NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-Q3_K_S.gguf
• NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-Q3_K_M.gguf
• NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-Q3_K_L.gguf
• NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-IQ4_XS.gguf
• NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-Q4_K_S.gguf
• NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-Q4_K_M.gguf
• NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-Q5_K_S.gguf
• NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-Q5_K_M.gguf
• NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-Q6_K.gguf
• NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-Q8_0.gguf
• NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-Q4_0.gguf
• NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-Q4_1.gguf
• NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-Q5_0.gguf
• NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-Q5_1.gguf

No importance matrix was used for any quantization in this repository.

Shape-fallback warning

Puzzle uses heterogeneous expert widths from 1280 to 2688. Some expert down-projection rows, especially width 2688, are not divisible by the 256-element block size used by K-quants. The pinned llama.cpp quantizer therefore applies its compatible fallback automatically on those tensors: Q2_K/Q3_K to Q4_0, IQ4_XS to IQ4_NL, Q4_K to Q5_0, Q5_K to Q5_1, and Q6_K to Q8_0. Consequently, named K-quant files are mixed recipes rather than a single tensor type everywhere. Required norms, scales, and other small tensors may remain F32.

Artifact integrity and quantizer compatibility are checked automatically, but successful conversion does not prove task-quality parity with BF16. Evaluate very small formats before practical use.

Provenance

• Source revision: 8fe5546888e9bd03fdbf52d808adebdfca901b52
• Source safetensors: 156,596,801,168 bytes across 31 model shards plus mtp.safetensors
• llama.cpp PR: #25444
• PR commit: af49ef5cd990d039dbf360dd3a9f3b5dafdd1726
• Patched converter SHA-256: 1aaa36ac789fc6eceebefe19d4d80c3c9dc56185a4a3e956411bc0478ee46afc
• Patched large-write GGUF writer SHA-256: 350db0132703b3b4025ee61e344b7d7400b9c3a2692b87dd6ca186d82423fa30
• Conversion: pinned safetensors in hybrid RAM/disk storage, then local convert_hf_to_gguf.py --outtype bf16
• Quantization: pinned llama-quantize, without --imatrix
• License: the source model is governed by the OpenMDW License Agreement 1.1

Follow the prompting, safety, license, and usage guidance in the original NVIDIA model card.