TraiNC/vlm_ocr_gguf_5e overview
TraiNC/vlm ocr gguf 5e GGUF Q8 0 quantized version of TraiNC/vlm ocr lora 5e https://huggingface.co/TraiNC/vlm ocr lora 5e . Files | File | Description | Size …
Runs locally from ~195.5 MB disk (4 GB VRAM class GPUs with llama.cpp / guIDE).
Repository Files & Downloads
Model Details
Model README
---
license: apache-2.0
base_model: unsloth/Qwen3.5-0.8B
tags:
- gguf
- llama.cpp
- qwen3_5
---
TraiNC/vlm_ocr_gguf_5e
GGUF Q8_0 quantized version of TraiNC/vlm_ocr_lora_5e.
Files
| File | Description | Size |
|---|---|---|
| model.gguf | Text decoder (Q8_0) + vision encoder | ~795 MB |
| mmproj-f16.gguf | Vision projector, F16 | ~205 MB |
Run with images
llama-mtmd-cli -m model.gguf --mmproj mmproj-f16.gguf \\
--image your_image.jpg -p "Describe this image"| model-f16.gguf | Full precision reference (F16) | ~1.5 GB |
How to run
Text
llama-cli -m model.gguf -p "Hello" -n 256Server (OpenAI-compatible API)
llama-server -m model.gguf --host 0.0.0.0 --port 8080Notes
- Q8_0 applied to weight tensors; embeddings and norms stay at higher precision.
- Built with llama.cpp.
Run TraiNC/vlm_ocr_gguf_5e with guIDE
Download guIDE — the AI-native code editor with local LLM inference and 69 built-in tools.
Source: Hugging Face · Compare models