---

license: apache-2.0

language:

• en
• zh

tags:

• ocr
• multimodal
• document-parsing
• gguf
• llama.cpp
• qwen2.5-vl
• table-recognition
• formula-recognition

pipeline_tag: image-text-to-text

---

NaviDC-OCR GGUF Quantizations

Static GGUF quantizations of StarDoc-AI/NaviDC-OCR for high-speed local document parsing, formula recognition, table transcription, and structured Markdown OCR using llama.cpp.

NaviDC-OCR is a specialized document vision-language model trained to parse complex digital and camera-captured documents, extracting headings, body text, LaTeX math formulas, and Markdown tables with geometric precision.

---

📊 Benchmark & Evaluation Results

Benchmarked on NVIDIA RTX GPU (CUDA) using llama-server on a complex technical document page containing headings, Schrödinger quantum equations, and multi-column tables:

| Quantization | Model Size | BPW | Generation Speed | Prompt Speed | Latency | Accuracy / Elements | Recommended |

| :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- |

| NaviDC-OCR-Q4_K_M.gguf | 461.8 MB | 4.34 | 211.4 tok/s | 1,320.4 tok/s | 1.22s | 100% (8/8 elements) | 🏆 Best Overall |

| NaviDC-OCR-Q5_K_M.gguf | 525.8 MB | 5.81 | 205.7 tok/s | 1,262.8 tok/s | 1.26s | 100% (8/8 elements) | ⭐️ High Quality |

| NaviDC-OCR-Q8_0.gguf | 767.5 MB | 8.50 | 195.5 tok/s | 1,243.4 tok/s | 1.30s | 100% (8/8 elements) | Reference |

| NaviDC-OCR-Q3_K_M.gguf | 394.8 MB | 3.50 | 179.1 tok/s | 1,238.5 tok/s | 6.26s | Degraded | Lightweight |

Companion Multimodal Projector (mmproj):

• mmproj-NaviDC-OCR-F16.gguf (1.33 GB) — required for vision encoding alongside any language model quant.

---

🚀 How to Run with llama.cpp

1. Start llama-server

llama-server \
  -m NaviDC-OCR-Q4_K_M.gguf \
  --mmproj mmproj-NaviDC-OCR-F16.gguf \
  --port 8090 \
  -ngl 99 \
  -c 4096

2. Query via OpenAI-compatible API

import requests, base64

with open("document_page.png", "rb") as f:
    img_b64 = base64.b64encode(f.read()).decode("utf-8")

response = requests.post(
    "http://127.0.0.1:8090/v1/chat/completions",
    json={
        "messages": [
            {
                "role": "user",
                "content": [
                    {"type": "text", "text": "Extract text."},
                    {"type": "image_url", "image_url": {"url": f"data:image/png;base64,{img_b64}"}}
                ]
            }
        ],
        "max_tokens": 2048,
        "temperature": 0.0
    }
)

print(response.json()["choices"][0]["message"]["content"])

---

📦 Files in this Repository

| Filename | Description | Size |

| :--- | :--- | :--- |

| mmproj-NaviDC-OCR-F16.gguf | Vision Projector (required for all quants) | 1.33 GB |

| NaviDC-OCR-Q4_K_M.gguf | Optimal 4-bit Medium Quantization (Fastest & recommended) | 461.8 MB |

| NaviDC-OCR-Q5_K_M.gguf | High-fidelity 5-bit Medium Quantization | 525.8 MB |

| NaviDC-OCR-Q8_0.gguf | 8-bit Reference Quantization | 767.5 MB |

| NaviDC-OCR-Q3_K_M.gguf | 3-bit Turbo Quantization | 394.8 MB |

| NaviDC-OCR-F16.gguf | Unquantized FP16 Baseline | 1.50 GB |

---

💡 Acknowledgements & Citation

• Original Model: StarDoc-AI/NaviDC-OCR
• GGUF Conversion & Benchmarks: Pluto AI by @konradjr007