---

license: apache-2.0

base_model: ds4sd/SmolDocling-256M-preview

tags:

- ocr

- document-understanding

- doctags

- vision-language

- gguf

- crispembed

- ggml

language:

- en

---

SmolDocling-256M GGUF

GGUF conversions of ds4sd/SmolDocling-256M-preview for CrispEmbed inference.

Ultra-compact document conversion model (256M params). Generates DocTags structured markup from page images — OCR, layout, tables, formulas, code, charts.

Model variants

| File | Quant | Size | Notes |

|------|-------|------|-------|

| smoldocling-f16.gguf | F16 | 491 MB | Full precision |

| smoldocling-q8_0.gguf | Q8_0 | 261 MB | Recommended |

| smoldocling-q4_k.gguf | Q4_K | 153 MB | Max compression |

Architecture

• Vision: SigLIP ViT (12L, 768d, 12 heads, patch=16, 512px)
• Connector: Pixel shuffle (scale=4, 1024→64 tokens) + Linear(12288→576)
• LLM: SmolLM2-135M (30L, 576d, GQA 9/3, SwiGLU, RoPE)
• Parameters: 256M total (93M vision + 135M LLM + connector)
• Output: DocTags (structured XML-like document markup)

Parity vs HF reference: vision cos=0.9998, connector cos=0.9999.

Usage

# CLI
./crispembed -m smoldocling-q8_0.gguf --ocr document.png

# Server
./crispembed-server --ocr smoldocling-q8_0.gguf --port 8080
curl -X POST http://localhost:8080/math/ocr -F "image=@document.png"

from crispembed import CrispMathOcr

ocr = CrispMathOcr("smoldocling-q8_0.gguf")
doctags = ocr.recognize("document.png")
print(doctags)  # <doctag><text>...</text>...</doctag>

License

Apache-2.0 — same as the base model.

Credits

Original model by Docling Team, IBM Research. GGUF conversion and inference engine by CrispEmbed.