---

license: apache-2.0

language:

- en

pipeline_tag: image-text-to-text

library_name: gguf

tags:

- granite

- granite-vision

- multimodal

- vision-language-model

- gguf

- llama.cpp

- local-inference

- quantized

base_model:

- ibm-granite/granite-vision-3.3-2b

---

Granite-Vision-3.3-2B-Q_M_GGUF-F16

GGUF export of Granite Vision 3.3 2B for local multimodal inference with llama.cpp.

This repository provides a quantized GGUF version of Granite Vision together with the multimodal projector required for image understanding.

Files

This repository contains the following files:

• granite-vision-3.3-2b-Q5_K_M.gguf — main quantized Granite Vision model
• mmproj-model-f16.gguf — multimodal projector file required for image input support

Description

Granite Vision 3.3 2B is a vision-language model (VLM) that can process:

• text prompts
• image + text prompts
• image description / captioning
• basic visual question answering (VQA)
• OCR-like reasoning depending on the image and prompt quality

This GGUF version is intended to be used locally with llama.cpp and llama-server.

Requirements

• Windows / Linux / macOS
• a recent llama.cpp build with multimodal support
• llama-server or llama-cli
• the two files in this repository:

- granite-vision-3.3-2b-Q5_K_M.gguf

- mmproj-model-f16.gguf

Example with llama-server

Example command:

llama-server \
  -m granite-vision-3.3-2b-Q5_K_M.gguf \
  --mmproj mmproj-model-f16.gguf \
  --host 127.0.0.1 \
  --port 8080 \
  --threads 12 \
  -c 4096