MDfox/OvisOCR2-GGUF-SOTA-20260823 overview
license: apache 2.0 library name: llama.cpp pipeline tag: image text to text base model: ATH MaaS/OvisOCR2 tags: gguf llama.cpp multimodal vision ocr document …
Runs locally from ~1.1 MB disk (4 GB VRAM class GPUs with llama.cpp / guIDE).
Repository Files & Downloads
| File | Type | Quantization | Size | Link |
|---|---|---|---|---|
| OvisOCR2-BF16.gguf | GGUF | BF16 | 1.41 GB | Download |
| OvisOCR2-IQ3_M-imatrix.gguf | GGUF | IQ3_M | 432.9 MB | Download |
| OvisOCR2-IQ3_XXS-imatrix.gguf | GGUF | IQ3_XXS | 379.6 MB | Download |
| OvisOCR2-OmniDocBench-Q8BF16Predictions-imatrix.gguf | GGUF | Q8BF16PREDICTIONS | 1.1 MB | Download |
| OvisOCR2-Q2_K.gguf | GGUF | Q2_K | 402.8 MB | Download |
| OvisOCR2-Q3_K_M.gguf | GGUF | Q3_K_M | 444.6 MB | Download |
| OvisOCR2-Q3_K_S.gguf | GGUF | Q3_K_S | 415.2 MB | Download |
| OvisOCR2-Q4_K_M.gguf | GGUF | Q4_K_M | 504.8 MB | Download |
| OvisOCR2-Q5_K_M.gguf | GGUF | Q5_K_M | 551.2 MB | Download |
| OvisOCR2-Q6_K.gguf | GGUF | Q6_K | 600.6 MB | Download |
| OvisOCR2-Q8_0.gguf | GGUF | Q8_0 | 774.2 MB | Download |
| mmproj-OvisOCR2-BF16.gguf | GGUF | BF16 | 197.7 MB | Download |
| mmproj-OvisOCR2-Q8_0.gguf | GGUF | Q8_0 | 110.6 MB | Download |
Model Details
| Model ID | MDfox/OvisOCR2-GGUF-SOTA-20260823 |
|---|---|
| Author | MDfox |
| Pipeline | image-text-to-text |
| License | apache-2.0 |
| Base model | ATH-MaaS/OvisOCR2 |
| Last modified | 2026-09-12T05:42:44.000Z |
Model README
---
license: apache-2.0
library_name: llama.cpp
pipeline_tag: image-text-to-text
base_model: ATH-MaaS/OvisOCR2
tags:
- gguf
- llama.cpp
- multimodal
- vision
- ocr
- document-parsing
- omnidocbench
---
English | 简体中文
OvisOCR2 GGUF
Unofficial GGUF conversion of ATH-MaaS/OvisOCR2, built and evaluated with llama.cpp v0.4.0 (5266f24). Updated 2026-09-12.
- Highest measured quality: Q8_0 + BF16 mmproj — 96.4547.
- Compact choice: Q3_K_M + Q8_0 mmproj — 96.2012, 555.2 MiB combined.
- Download one language model and one
mmprojfile. BF16, Q6_K and Q5_K_M language models are also available, without full v0.4.0 evaluations.
OmniDocBench v1.6
Official quick-match evaluator, all 1,651 pages; independent reproduction.
| Model | mmproj | MiB¹ | Overall ↑ | Text Edit ↓ | CDM ↑ | TEDS ↑ |
| --- | --- | ---: | ---: | ---: | ---: | ---: |
| Q8_0 | BF16 | 972.0 | 96.4547 | 0.02468 | 97.4232 | 94.4092 |
| Q8_0 | Q8_0 | 884.8 | 96.2864 | 0.02458 | 97.1160 | 94.2011 |
| Q3_K_M | Q8_0 | 555.2 | 96.2012 | 0.02605 | 97.2573 | 93.9517 |
| Q4_K_M | Q8_0 | 615.3 | 95.9089 | 0.02495 | 96.8344 | 93.3870 |
| Q3_K_S | Q8_0 | 525.7 | 95.6908 | 0.02835 | 96.3066 | 93.6009 |
| Q2_K | Q8_0 | 513.3 | 87.1185 | 0.07639 | 82.8282 | 86.1669 |
| IQ3_M* | Q8_0 | 543.5 | 95.9456 | 0.02581 | 96.9425 | 93.4754 |
| IQ3_XXS* | Q8_0 | 490.1 | 95.1446 | 0.02940 | 95.8875 | 92.4862 |
¹ Combined file size, not VRAM usage. IQ3 uses an importance matrix calibrated on OmniDocBench predictions; these scores are not clean held-out results. Q2_K recorded 1,650 successful pages, 1,651 prediction files and 197 token-limit hits; see the raw summaries for retry/failure counters.
RTX 5060 Ti 16 GB · 3 parallel slots · 32K context per slot · BF16 KV · temperature 0 · 16,384 output-token limit · 196–8,100 image tokens. Full results and settings · CSV.
Run
Use llama.cpp v0.4.0 and replace page.png with your image:
hf download MDfox/OvisOCR2-GGUF-SOTA-20260823 OvisOCR2-Q3_K_M.gguf mmproj-OvisOCR2-Q8_0.gguf prompt.txt --local-dir .
./llama-mtmd-cli -m OvisOCR2-Q3_K_M.gguf \
--mmproj mmproj-OvisOCR2-Q8_0.gguf --image page.png \
-f prompt.txt -ngl all -c 32768 -n 16384 --temp 0 \
--flash-attn on --image-min-tokens 196 --image-max-tokens 8100Run MDfox/OvisOCR2-GGUF-SOTA-20260823 with guIDE
Download guIDE — the AI-native code editor with local LLM inference and 69 built-in tools.
Source: Hugging Face · Compare models