TraiNC/vlm_ocr_gguf overview
TraiNC/vlm ocr gguf GGUF Q8 0 quantized version of TraiNC/vlm ocr lora https://huggingface.co/TraiNC/vlm ocr lora . Files | File | Description | Size | | | | |…
Runs locally from ~195.5 MB disk (4 GB VRAM class GPUs with llama.cpp / guIDE).
Repository Files & Downloads
Model Details
Model README
---
license: apache-2.0
base_model: unsloth/Qwen3.5-0.8B
tags:
- gguf
- llama.cpp
- qwen3_5
---
TraiNC/vlm_ocr_gguf
GGUF Q8_0 quantized version of TraiNC/vlm_ocr_lora.
Files
| File | Description | Size |
|---|---|---|
| model.gguf | Text decoder (Q8_0) + vision encoder | ~795 MB |
| mmproj-f16.gguf | Vision projector, F16 | ~205 MB |
Run with images
llama-mtmd-cli -m model.gguf --mmproj mmproj-f16.gguf \\
--image your_image.jpg -p "Describe this image"| model-f16.gguf | Full precision reference (F16) | ~1.5 GB |
How to run
Text
llama-cli -m model.gguf -p "Hello" -n 256Server (OpenAI-compatible API)
llama-server -m model.gguf --host 0.0.0.0 --port 8080Notes
- Q8_0 applied to weight tensors; embeddings and norms stay at higher precision.
- Built with llama.cpp.
Run TraiNC/vlm_ocr_gguf with guIDE
Download guIDE — the AI-native code editor with local LLM inference and 69 built-in tools.
Source: Hugging Face · Compare models