---

license: apache-2.0

base_model:

• sionic-ai/pepperocr-1-4b

language:

• en

pipeline_tag: image-text-to-text

library_name: transformers

tags:

• llama-cpp
• text-generation-inference
• document-ai
• document-parsing
• ocr
• multilingual
• safetensors
• vllm

---

pepperocr-1-4b-GGUF

> PepperOCR-1-4B is a 4.54-billion-parameter multilingual vision-language model for document parsing, fine-tuned from Qwen3.5-4B (architecture Qwen3_5ForConditionalGeneration) by sionic-ai, designed to convert document images directly into Markdown output and released as a public pre-release evaluation repository with weights distributed in BF16 safetensors (2 shards, ~9.1 GB). The model ships with a fully reproducible MDPBench evaluation runtime locked via uv with stable vLLM 0.24.0, PyTorch 2.11.0+cu130, CUDA 13.0, and Transformers 5.11.0, processing all 17 public MDPBench languages with a CPU-based document-orientation classifier (PP-LCNet_x1_0_doc_ori) preceding inference; on locally reproduced Korean and Thai public-set scores it achieves 92.2 and 83.2 overall accuracy respectively (87.7 KO/TH macro average), broken down further into digital versus photographed document subsets, using a greedy-first decoding strategy with a degeneration guard that retried roughly 10% of samples. As a pre-release evaluation preview, no final license has yet been applied to the fine-tuning contribution itself (though the underlying Qwen3.5-4B base remains Apache-2.0, and the final PepperOCR-1-4B release is planned under Apache-2.0 pending evaluation approval), and public downloadability does not currently grant redistribution or reuse rights.

Model Files

File Name | Quant Type | File Size | File Link |

|-----------|------------|-----------|-----------|

| pepperocr-1-4b.BF16.gguf | BF16 | 8.42 GB | Download |

| pepperocr-1-4b.F16.gguf | F16 | 8.42 GB | Download |

| pepperocr-1-4b.Q3_K_L.gguf | Q3_K_L | 2.42 GB | Download |

| pepperocr-1-4b.Q3_K_M.gguf | Q3_K_M | 2.26 GB | Download |

| pepperocr-1-4b.Q3_K_S.gguf | Q3_K_S | 2.07 GB | Download |

| pepperocr-1-4b.Q4_K_M.gguf | Q4_K_M | 2.71 GB | Download |

| pepperocr-1-4b.Q4_K_S.gguf | Q4_K_S | 2.56 GB | Download |

| pepperocr-1-4b.Q5_K_M.gguf | Q5_K_M | 3.07 GB | Download |

| pepperocr-1-4b.Q5_K_S.gguf | Q5_K_S | 2.99 GB | Download |

| pepperocr-1-4b.Q8_0.gguf | Q8_0 | 4.48 GB | Download |

| pepperocr-1-4b.mmproj-bf16.gguf | mmproj-bf16 | 676 MB | Download |

| pepperocr-1-4b.mmproj-f16.gguf | mmproj-f16 | 676 MB | Download |

| pepperocr-1-4b.mmproj-q8_0.gguf | mmproj-q8_0 | 367 MB | Download |

llama.cpp

LLM inference in C/C++ — https://github.com/ggml-org/llama.cpp