---

license: other

license_name: openmdw1.1-license

license_link: https://openmdw.ai/license/1-1/

base_model: nvidia/Cosmos3-Nano

pipeline_tag: image-text-to-text

library_name: llama.cpp

tags:

• gguf
• nvidia
• cosmos
• cosmos3
• physical-ai
• qwen3-vl
• vision
• llama.cpp
• vlm
• mmproj
• q4_k_m
• 8gb
• 12gb

base_model_relation: quantized

---

Cosmos3-Nano-GGUF (understanding branch)

The first GGUF of nvidia/Cosmos3-Nano — the reasoning/VQA core of NVIDIA's omnimodal Physical-AI world model, extracted and converted so it runs anywhere llama.cpp runs (llama.cpp, LM Studio, Ollama, Jan, …).

What this is (and isn't)

Cosmos3-Nano is a joint omnimodal transformer: every layer contains an understanding path (a standard Qwen3-VL-8B-class VLM) and a generation path (MMDiT-style joint attention + parallel _moe_gen MLPs) driving video/audio/action generation through a diffusion stack.

This repo contains the understanding path + vision encoder only, re-keyed to the standard Qwen3VLForConditionalGeneration layout and converted with mainline convert_hf_to_gguf.py:

• ✅ Image + multi-image understanding, VQA, physical-world reasoning, chat, tool calling (ChatML), 256K context (mrope, interleaved)
• ❌ Video/audio/action generation — those weights (*_moe_gen, joint-attention adds, action/audio/diffusion heads, ≈14 GB) are dropped; they require the VAE + scheduler pipeline and cannot run in llama.cpp

Architecture of what you get: 36 layers, 4096 hidden, GQA 32/8, head_dim 128, qk-norm, 262,144 max context — plus the 27-block deepstack vision encoder (deepstack indexes 8/16/24) as an mmproj.

Files

| File | Quant | Size |

|---|---|---|

| Cosmos3-Nano-VLM.Q4_K_M.gguf | Q4_K_M | ~5.0 GB |

| Cosmos3-Nano-VLM.Q5_K_M.gguf | Q5_K_M | ~5.9 GB |

| Cosmos3-Nano-VLM.Q6_K.gguf | Q6_K | ~6.8 GB |

| Cosmos3-Nano-VLM.Q8_0.gguf | Q8_0 | ~8.8 GB |

| Cosmos3-Nano-VLM.bf16.gguf | bf16 | ~16.5 GB |

| mmproj-Cosmos3-Nano-VLM.f16.gguf | f16 | vision encoder |

Run it

# server with vision
llama-server -m Cosmos3-Nano-VLM.Q4_K_M.gguf \
  --mmproj mmproj-Cosmos3-Nano-VLM.f16.gguf -c 32768

# one-shot VQA
llama-mtmd-cli -m Cosmos3-Nano-VLM.Q4_K_M.gguf \
  --mmproj mmproj-Cosmos3-Nano-VLM.f16.gguf \
  --image photo.jpg -p "What will happen next in this scene?" -c 8192

Always pass an explicit -c; without it the context allocation can eat all VRAM and generation crawls.

Verified (RTX 5070 12GB, Q4_K_M + f16 mmproj, -c 8192): correct grounded scene description plus sensible physical next-event prediction on a held-out image; 213 tok/s prefill, 105 tok/s generation.

Provenance

Extraction is a pure tensor re-keying — no weights were modified. The understanding path uses diffusers-style attention naming in the source (to_q/to_k/to_v/to_out, norm_q/norm_k); these map 1:1 onto Qwen3-VL's q_proj/k_proj/v_proj/o_proj/q_norm/k_norm. The extraction script is included in this repo (extract_vlm.py) for full reproducibility.

Original model: nvidia/Cosmos3-Nano, released under the OpenMDW-1.1 license, which this repo inherits. All credit for the model itself to the NVIDIA Cosmos team — see their technical report.