---

license: apache-2.0

language:

• en

pipeline_tag: image-text-to-text

tags:

• vision-language-model
• multimodal
• edge
• on-device
• nanovlm
• vqa
• image-text-to-text
• gguf
• quantized
• llama.cpp
• imatrix

base_model:

• qvac/VisionPsy-Nano-460M
• lusxvr/nanoVLM-460M-8k

---

VisionPsy-Nano-460M-GGUFs

VisionPsy-Nano-460M-GGUFs provides GGUF weights of

VisionPsy-Nano-460M for fast, fully on-device multimodal inference with our patched llama.cpp fork (not stock upstream). Requires the patched llama.cpp. VisionPsy-Nano-460M is a compact (~460M parameter) vision-language model that compares favourably with other models in its ~0.5B weight class on 16 of 17 public benchmarks; these GGUF builds bring that quality to phones, tablets, and edge hardware.

We release some GGUF variants, ranging from unquantized FP32 and BF16 references to lossless 8-bit, high-quality 5-bit, and ultra-compact 3-bit formats. This range supports deployment across diverse hardware settings, from high-end workstations to resource-constrained on-device environments. All sub-8-bit variants (except the legacy Q4_0) use importance-matrix (imatrix) calibration, which consistently reduces quality degradation.

> Each GGUF is a vision-language model: alongside the text weights (*.gguf) you also need the

> multimodal projector (mmproj-*.gguf, the SigLIP2 vision encoder) to run image inputs. See

> Usage.

| | |

| --- | --- |

| Developed by | Tether AI Research^* |

| Model type | Vision-language model (image-text-to-text), GGUF quantized |

| Base model | VisionPsy-Nano-460M |

| Architecture | nanoVLM — SigLIP2 vision encoder + SmolLM2-360M language backbone |

| Language | English |

| License | Apache 2.0 |

| Quantization tool | llama.cpp |

| Calibration | importance-matrix (imatrix) for all -imat variants |

| All variants | VisionPsy-Nano-460M · VisionPsy-Nano-460M-Flash · VisionPsy-Nano-460M-GGUFs · VisionPsy-Nano-460M-Flash-GGUFs |

^* References to Tether AI Research are references to Tether Data, S.A. de C.V.

---

Available Files

All published GGUF files are generated with llama.cpp. Quantization labels refer exclusively to the language-model weights, while the vision encoder and multimodal projector are fixed to Q8_0 for all variants. FP32 and BF16 are unquantized language-model references evaluated within the same inference harness. Q8_0 does not use importance-matrix (imatrix) calibration, as it provides no measurable benefit at 8-bit precision. Q4_0 is the legacy 4-bit format without imatrix calibration and is retained mainly for cross-model on-device latency comparisons. All remaining sub-8-bit language-model variants use imatrix calibration.

Δ AVG below is measured against the FP32 GGUF baseline using our Normalized (0-100) score, the mean of 17 public VLM benchmarks (see Benchmarks). Δ AVG (pts) is the change in points; Δ AVG (rel %) normalizes by the baseline.

| File | Format | Imatrix | Size (MB) | Δ AVG (pts) | Δ AVG (rel %) | Recommended For |

| --- | --- | --- | --- | --- | --- | --- |

| visionpsy-nano-460m-fp32.gguf | FP32 | n/a | 1562 | ≈0.00 | ≈0.00% | Unquantized GGUF reference |

| visionpsy-nano-460m-bf16.gguf | BF16 | n/a | 782 | ≈0.09 | -0.15% | Unquantized GGUF |

| visionpsy-nano-460m-q8_0.gguf | Q8_0 | no (not needed) | 416 | ≈0.00 | ≈0.00% | Best quality, lossless |

| visionpsy-nano-460m-q5_k_m-imat.gguf | Q5_K_M | yes | 310 | −0.59 | −0.96% | 5-bit with imatrix (prefer the non -imat build) |

| visionpsy-nano-460m-q5_k_m.gguf | Q5_K_M | no | 310 | −0.12 | −0.19% | Recommended high-quality 5-bit |

| visionpsy-nano-460m-q4_k_m-imat.gguf | Q4_K_M | yes | 289 | −0.24 | −0.39% | Recommended for mobile/laptop (best size/quality) |

| visionpsy-nano-460m-iq4_xs-imat.gguf | IQ4_XS | yes | 242 | −0.92 | −1.49% | Smaller 4-bit alternative |

| visionpsy-nano-460m-iq4_nl-imat.gguf | IQ4_NL | yes | 244 | −0.96 | −1.55% | Alternative 4-bit |

| visionpsy-nano-460m-q4_0.gguf | Q4_0 | no | 244 | −2.85 | −4.62% | ⚠ Legacy 4-bit (no imatrix) — used for latency benchmarks; prefer Q4_K_M |

| visionpsy-nano-460m-iq3_m-imat.gguf | IQ3_M | yes | 240 | −1.36 | −2.20% | Strong compact 3-bit for tight memory |

| visionpsy-nano-460m-iq3_xxs-imat.gguf | IQ3_XXS | yes | 230 | −1.70 | −2.75% | Ultra-compact 3-bit (accept some quality loss) |

| mmproj-visionpsy-nano-460m-q8.gguf | Q_8 (vision) | n/a | 104 | n/a | n/a | Required multimodal projector for all files above |

---

Quick Recommendation

| Your constraint | Choose |

| --- | --- |

| You want a llama.cpp-native unquantized FP32 file | FP32 — no quantization applied on LLM |

| You want a llama.cpp-native unquantized BF16 file | BF16 — no quantization applied on LLM |

| You want the best quality at half the size | Q8_0 — lossless |

| You want extra quality headroom over 4-bit | Q5_K_M — only −0.12 pts (−0.19%) |

| You want the best size/quality trade-off (most users) | Q4_K_M (imatrix) — −0.24 pts (−0.39%), fits high-end mobile/laptop |

| You need a smaller 4-bit file | IQ4_XS (imatrix) — −0.92 pts (−1.49%) |

| You need the smallest recommended file | IQ3_M (imatrix) — −1.36 pts (−2.20%) |

| You are cross-benchmarking on-device latency | Q4_0 — the legacy format used in our device tests (lower quality; prefer Q4_K_M for deployment) |

---

Benchmarks

All scores are computed **in-house with a single VLMEvalKit

harness** so every configuration is scored identically, using each benchmark's official metric

(POPE = F1, MMVet = partial credit, MM-IFEval = instruction-following accuracy, **MME = Perception +

Reasoning points, OCRBench = Final Score /1000**; accuracy otherwise). LLM-as-judge scoring uses

Qwen3-27B-FP8. Each column is a different GGUF quantization / calibration configuration.

Legend: -i = importance-matrix (imatrix) calibration. MME = P+R points, OCRBench = /1000,

all others accuracy/F1. Normalized = mean across the 17 benchmarks scaled to 0-100 (MME /28,

OCRBench /1000). Weights = on-disk LM GGUF only (excludes shared mmproj Q8_0, ~104 MB).

| Benchmark | FP32 | BF16 | Q8_0 | Q5_K_M | Q5_K_M-i | Q4_0 | IQ4_XS-i | IQ4_NL-i | Q4_K_M-i | IQ3_XXS-i | IQ3_M-i |

| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |

| Weights | 1562 MB | 782 MB | 416 MB | 310 MB | 310 MB | 244 MB | 242 MB | 244 MB | 289 MB | 230 MB | 240 MB |

| MMStar | 46.2 | 45.9 | 46.1 | 46.7 | 45.6 | 41.7 | 46.3 | 46.8 | 45.9 | 46.5 | 45.9 |

| MMBench | 62.1 | 62.2 | 61.8 | 60.3 | 60.9 | 56.1 | 59.3 | 59.3 | 60.7 | 58.8 | 60.9 |

| RealWorldQA | 59.7 | 59.6 | 59.1 | 60.5 | 59.9 | 60.3 | 57.9 | 58.0 | 60.3 | 59.1 | 58.7 |

| MME | 1541.3 | 1542.9 | 1553.5 | 1569.9 | 1548.7 | 1553.3 | 1553.4 | 1548.0 | 1529.6 | 1571.2 | 1532.2 |

| SEEDBench | 68.9 | 68.9 | 68.9 | 68.2 | 68.6 | 67.7 | 67.6 | 67.8 | 68.6 | 67.0 | 67.3 |

| POPE | 87.9 | 87.8 | 87.7 | 87.4 | 87.7 | 87.4 | 87.4 | 87.8 | 87.7 | 87.0 | 87.1 |

| MMMU | 32.1 | 31.6 | 32.6 | 32.1 | 33.0 | 31.3 | 30.4 | 30.2 | 31.8 | 31.7 | 31.4 |

| MathVista \* | 47.5 | 47.5 | 47.8 | 47.4 | 45.9 | 41.8 | 46.6 | 46.3 | 47.6 | 44.8 | 48.3 |

| AI2D | 66.0 | 65.6 | 65.3 | 65.5 | 65.7 | 64.2 | 65.9 | 65.7 | 65.4 | 65.9 | 65.5 |

| ScienceQA | 84.7 | 84.7 | 84.7 | 84.2 | 83.9 | 80.8 | 82.9 | 82.8 | 84.5 | 81.1 | 82.5 |

| OCRBench \* | 763 | 765 | 764 | 756 | 747 | 725 | 740 | 747 | 765 | 723 | 736 |

| ChartQA \* | 77.2 | 77.4 | 77.5 | 77.1 | 77.0 | 75.4 | 75.8 | 75.9 | 77.0 | 75.0 | 75.9 |

| TextVQA \* | 79.5 | 79.6 | 79.1 | 78.7 | 79.1 | 76.7 | 78.4 | 78.2 | 79.3 | 76.4 | 77.5 |

| DocVQA \* | 83.5 | 83.5 | 83.7 | 83.3 | 83.5 | 80.8 | 82.7 | 82.5 | 83.1 | 82.1 | 82.4 |

| InfoVQA \* | 48.1 | 48.0 | 47.8 | 46.8 | 46.5 | 44.2 | 47.0 | 46.8 | 46.6 | 45.8 | 46.2 |

| MM-IFEval | 43.1 | 42.6 | 43.5 | 45.0 | 41.6 | 34.3 | 43.7 | 42.9 | 44.2 | 37.9 | 38.9 |

| MMVet \* | 32.0 | 31.9 | 32.3 | 32.7 | 31.0 | 30.6 | 32.8 | 32.3 | 31.8 | 33.4 | 29.9 |

| Normalized | 61.75 | 61.66 | 61.75 | 61.63 | 61.16 | 58.9 | 60.83 | 60.79 | 61.51 | 60.05 | 60.39 |

\* ChartQA, TextVQA, DocVQA, InfoVQA, OCRBench, MathVista, and MMVet use an LLM-as-judge Qwen3-27B-FP8, which evaluates free-form answers more reliably than strict string matching (e.g. "12%" vs "12 percent", paraphrases, units, formatting).

---

Key Findings

• Q8_0 is effectively lossless: no difference on the Normalized score, at roughly half the size of BF16, with no imatrix needed.
• Q5_K_M is the recommended high-quality option: only −0.12 pts (−0.19%), and it is also better than Q5_K_M with imatrix (−0.59 pts).
• Q4_K_M with imatrix is the sweet spot: −0.24 pts (−0.39%) for a substantial size reduction, the best size/quality choice for mobile/laptop.
• imatrix beats legacy Q4_0 decisively: the imatrix 3-bit IQ3_M (−1.36 pts) and even

IQ3_XXS (−1.70 pts) both outperform the legacy 4-bit Q4_0 (−2.85 pts). If you are not

latency-benchmarking, there is no reason to prefer Q4_0 over an imatrix build.

• Compact 3-bit stays strong: IQ3_M holds a Normalized score of 60.39, making it a viable option for the tightest memory budgets.

---

Usage

These are multimodal GGUF files: you need both the text model (*.gguf) and the vision

projector (mmproj-*.gguf).

llama.cpp

# Download the recommended text weights (Q4_K_M with imatrix) + the vision projector
huggingface-cli download qvac/VisionPsy-Nano-460M-GGUFs \
    visionpsy-nano-460m-q4_k_m-imat.gguf \
    mmproj-visionpsy-nano-460m-q8.gguf \
    --local-dir .

First build the patched llama.cpp — see the build instructions in

llama-cpp-inference, then run multimodal inference:

# Run multimodal inference (image + prompt)
./llama-mtmd-cli \
    -m visionpsy-nano-460m-q4_k_m-imat.gguf \
    --mmproj mmproj-visionpsy-nano-460m-q8.gguf \
    --image your_image.jpg \
    -p "What is in this image?" \
    -n 128

# Or start an OpenAI-compatible server with a web UI
./llama-server \
    -m visionpsy-nano-460m-q4_k_m-imat.gguf \
    --mmproj mmproj-visionpsy-nano-460m-q8.gguf

---

Intended use

VisionPsy-Nano-460M-GGUFs targets latency- and memory-constrained, on-device multimodal applications: visual question answering, document/chart/diagram understanding, scene-text reading, and lightweight visual instruction following on phones and other edge hardware. Because of its small size, we recommend fine-tuning on your specific domain to maximize quality.

Limitations

• Single-image by design: the model is trained and optimized for one image per query; multi-image

prompts are outside its intended use.

• As a compact model, it may occasionally hallucinate or miscount and is best suited to focused tasks

rather than very dense documents or long multi-step math, where larger models have an edge.

• Quantization artifacts: lower bit counts can subtly degrade outputs in ways aggregate

benchmarks do not fully capture. We recommend Q4_K_M or higher for production and prefer the

imatrix builds over the legacy Q4_0.

• Primarily English; other languages are not officially supported yet.
• Not intended for safety-critical or high-stakes automated decisions.
• Benchmark scores are produced with a fixed in-house harness and an LLM judge (Qwen3-27B-FP8); absolute numbers may differ from other reported setups.

Acknowledgements

Built on the excellent open-source work of nanoVLM,

SmolLM2,

SigLIP2, and

llama.cpp.

Citation

@misc{visionpsynano2026,
title = {VisionPsy-Nano-460M: A Compact Vision-Language Model for On-Device Inference},
author = {Tether AI Research},
year = {2026},
note = {Hugging Face model card}
}

Copyright

We will take appropriate actions in response to notices of copyright infringement. If you believe

your work has been used or copied in a manner that infringes upon your intellectual property rights,

please email data-apps@tether.io identifying and describing both the copyrighted work and alleged

infringing content.

Licensing

This model, which was finetuned as described in the blog post, is licensed by Tether Data,

S.A. de C.V. under the Apache 2.0 license. As described in the blog post, this model is a version of

the NanoVLM-460M-8k pre-trained model (https://huggingface.co/lusxvr/nanoVLM-460M-8k), which is made

available under the MIT license.

The FineVision dataset (https://huggingface.co/datasets/HuggingFaceM4/FineVision) is made available

under the CC-BY-4.0 (Creative Commons - Attribution 4.0) license. FineVision is an aggregation of a

number of public sources unified into a single corpus. Individual subsets within the collection may

inherit specific underlying terms from their original creators. As described in the blog post, a

subset of the FineVision dataset was used as a part of finetuning the model.

The NVIDIA Nemotron-Image-Training-v3 dataset

(https://huggingface.co/datasets/nvidia/Nemotron-Image-Training-v3) is made available under the

CC-BY-4.0 (Creative Commons - Attribution 4.0). The mPLUG TinyChartData dataset

(https://huggingface.co/datasets/mPLUG/TinyChartData) is made available under the Apache 2.0

license. The TabMWP dataset (https://promptpg.github.io/) is made available under the CC BY-NC-SA

4.0 (Creative-Commons-Attribution-NonCommercial-ShareAlike 4.0). The PopVQA dataset

(https://huggingface.co/datasets/idoco/PopVQA) is made available under the MIT license. The

InfoSeek dataset (https://github.com/open-vision-language/infoseek) is made available under the

Apache 2.0 license. The MMKU-Bench dataset (https://huggingface.co/datasets/baochenfu/MMKU-Bench)

is made available under the Apache 2.0 license. The VisionFoundry-10K dataset

(https://huggingface.co/datasets/zlab-princeton/VisionFoundry-10K) is made available under the

Apache 2.0 license. The PKU-SafeRLHF-V dataset

(https://huggingface.co/datasets/PKU-Alignment/PKU-SafeRLHF-V) is made available under the CC-BY-NC

4.0 (Attribution-NonCommercial 4.0 International). As described in the blog post, the NVIDIA

Nemotron-Image-Training-v3, mPLUG TinyChartData, TabMWP, PopVQA, InfoSeek, MMKU-Bench,

VisionFoundry-10K and PKU-SafeRLHF-V datasets were used as a part of finetuning the model.