---

license: apache-2.0

language:

• en
• es
• fr
• de
• it
• pt

library_name: gguf

pipeline_tag: image-text-to-text

base_model:

• SmallAICreator/GRAFT-1B
• google/siglip-so400m-patch14-384
• markendo/llava-extract-qwen3-1.7B

tags:

• gguf
• vision
• multimodal
• llava
• siglip
• llama.cpp
• on-device

---

GRAFT-1B-Vision-GGUF

GRAFT-1B can now see images, with zero vision training.

This repo pairs the GRAFT-1B language

model (SysPrompt release, Q6_K) with a SigLIP-so400m image encoder and a small

2-layer connector, packaged as a llama.cpp mmproj file. Nothing was trained to make

this work. GRAFT-1B was shrunk from Qwen3-1.7B, and a connector originally trained for

Qwen3-1.7B carried over to GRAFT-1B directly.

Made by SmallAICreator / UltraLabs.

| File | What it is | Size |

| --- | --- | ---: |

| GRAFT-1B-SysPrompt-Q6_K.gguf | The GRAFT-1B language model (same file as the main GRAFT-1B repo) | 971 MB |

| mmproj-GRAFT-1B-siglip-so400m-Q8_0.gguf | Vision encoder + connector (SigLIP-so400m/14 @ 384px, 2-layer MLP into GRAFT's 2048-wide input), 8-bit so it fits on phones | 587 MB |

Quick start

llama.cpp

# one image, one question
llama-mtmd-cli -m GRAFT-1B-SysPrompt-Q6_K.gguf \
  --mmproj mmproj-GRAFT-1B-siglip-so400m-Q8_0.gguf \
  --image photo.jpg -p "Describe this image." --temp 0.4

# chat UI with image upload at http://localhost:8080
llama-server -m GRAFT-1B-SysPrompt-Q6_K.gguf \
  --mmproj mmproj-GRAFT-1B-siglip-so400m-Q8_0.gguf -c 4096

PocketPal (Android / iOS)

Models → + → Add from Hugging Face → search GRAFT-1B-Vision-GGUF → download

the GGUF. PocketPal fetches the projector automatically; turn on Vision on the model

card, then attach a photo in chat.

Example outputs

Real, unedited outputs (temperature 0.2, no training of any part):

> Image: two cats lying on a pink blanket

>

> GRAFT-1B: The image is a close-up of two cats, one on a pink and white pillowcase,

> the other curled up on a soft, cozy blanket. The contrast between their fur tones and

> the surrounding fabric creates a warm, intimate scene.

> Image: a living room with a person, chairs, plants and a table

>

> GRAFT-1B: The image is a close-up of a living room, with a focus on the furniture

> and decor. The person is sitting in a chair or standing by a sofa, looking at the

> colorful arrangement of plants, flowers, and other decorative items.

On a phone

Running fully on-device on a Pixel 6a in PocketPal (this repo's files, Q8_0 vision

file), with a photo of a backyard deck:

> User: (photo) What's this image?

>

> GRAFT-1B: The image shows a wooden deck or patio with a wooden plank or beam

> running across it, with a wooden fence or railing on one side and a wooden plank or

> beam on the other. The dark gray and browns in the image suggest that it's a naturally

> occurring wood structure, with the dark wood being the type used for the deck or

> railing.

7.58 tokens/sec generation, about 46 s to the first token (the vision encoder and

the 729 image tokens are processed on the phone's CPU first).

<img src="assets/pixel6a-pocketpal-deck.png" alt="GRAFT-1B Vision describing a deck photo in PocketPal on a Pixel 6a" width="360">

Limitations

• Memory: the model plus vision file need about 2 GB of free RAM. On 6 GB phones, close other apps first.

• Zero-shot connector. The connector was trained for Qwen3-1.7B, not GRAFT-1B. It

works well for general scenes, objects, animals and colors, but small details are

often missed or guessed (in the examples above it missed a pair of TV remotes and a

TV).

• Not for reading text. One 384×384 view per image, so small text, documents and

screenshots are not read reliably.

• Counting and fine detail are weak.
• Everything in the GRAFT-1B limitations

still applies: it is a 1B model and can invent details confidently.

Credits and licenses

• GRAFT-1B language model: SmallAICreator / UltraLabs, Apache-2.0.
• Image encoder: google/siglip-so400m-patch14-384, Apache-2.0.
• Connector (and the encoder weights used with it): taken unchanged from

markendo/llava-extract-qwen3-1.7B,

the Extract model from *Downscaling Intelligence: Exploring Perception and Reasoning

Bottlenecks in Small Multimodal Models* (paper,

code, MIT). Thanks to the authors.

• Packaged with llama.cpp's LLaVA encoder converter.