---

license: apache-2.0

base_model: allenai/Molmo2-8B

tags:

- gguf

- llama.cpp

- allenai

- molmo

- vision

- image-text-to-text

pipeline_tag: image-text-to-text

---

Molmo2-8B GGUF

GGUF conversion of allenai/Molmo2-8B, published by Mu2 Solutions. This is the first GGUF release of Molmo2-8B.

Molmo2-8B is Allen Institute for AI's fully open multimodal model (Apache 2.0): SigLIP vision encoder + Qwen3-8B text backbone with a pooling cross-attention projector.

License

Apache 2.0 — same as the source model. No additional restrictions.

Files

| File | Quant | Size |

|---|---|---|

| Molmo2-8B-text-F16.gguf | F16 (text) | 16 GB |

| Molmo2-8B-text-Q8_0.gguf | Q8_0 (text) | 8.2 GB |

| Molmo2-8B-text-Q4_K_M.gguf | Q4_K_M (text) | 4.7 GB |

| Molmo2-8B-mmproj-F16.gguf | F16 (vision projector) | 900 MB |

Q4_K_M is the recommended default. Pair any text quant with the mmproj for vision.

Conversion

• Source: HF safetensors (8 shards) via the llama.cpp HF-to-GGUF converter (b9859-era, custom Molmo2 vision converter)
• Text architecture: qwen3 (rope freq base 1,000,000 read from source config)
• mmproj architecture: clip (312 tensors: SigLIP ViT + pooling cross-attention + SwiGLU projector)
• Chat template embedded from source chat_template.jinja

Verification

• Text: verified. Raw decoder completion produces coherent, correct output (e.g. "The capital of France is Paris..."). Rope, tokenizer, and weights confirmed against source config.
• Vision encode: verified. Image encoding through the vision graph completes correctly (sane embedding stats, ~1.3s on CPU).
• Vision chat generation: KNOWN ISSUE — experimental fork only. This conversion was produced with Mu2's experimental llama.cpp fork (official upstream llama.cpp does not yet support the Molmo2 architecture). In that fork, vision encoding works but generation can leak image/control tokens (<im_start>, <im_patch>, ...) into the output. This is a runtime limitation of the experimental tooling, not a weights problem — text inference is clean. Work on the runtime is ongoing; a corrected mmproj will be published here when ready.

Usage

# Text-only chat (works with standard llama.cpp once Molmo2 arch support lands,
# or with the Mu2 experimental fork today)
llama-server -m Molmo2-8B-text-Q4_K_M.gguf --jinja -c 8192

# Vision (Mu2 experimental mtmd tooling; expect the generation quirk above)
llama-mtmd-cli -m Molmo2-8B-text-Q4_K_M.gguf \
  --mmproj Molmo2-8B-mmproj-F16.gguf \
  --image image.jpg -p "Describe this image."

Credit

• Model: Allen Institute for AI (Ai2) — Apache 2.0
• Conversion: Mu2 Solutions — promoting truly open-source AI, one GGUF at a time