cstr/got-ocr2-crispembed-GGUF overview

---

license: apache-2.0

base_model: stepfun-ai/GOT-OCR2_0

tags:

• gguf
• ocr
• crispembed
• got-ocr2

library_name: crispembed

---

GGUF conversion of stepfun-ai/GOT-OCR2_0 for use with CrispEmbed.

Architecture

• Vision: SAM ViT-B (12 layers, 768d, 12 heads, 16×16 patches, 1024×1024 input)

- Windowed attention (ws=14) with global attention at layers [2, 5, 8, 11]

- Decomposed relative position encoding

- Neck: Conv(768→256) → LN2d → Conv(256→256) → LN2d

- Downsample: Conv(256→512→1024, stride 2) → 256 vision tokens

- Projector: Linear(1024, 1024)

• LLM: Qwen2-0.5B (24 layers, 1024d, MHA 16/16, SiLU SwiGLU, RoPE θ=1M)
• Tokenizer: tiktoken (151860 vocab)
• Total: ~0.7B parameters

Files

| File | Precision | Size | Notes |

|------|-----------|------|-------|

| got-ocr2-q4_k.gguf | Q4_K | 445 MB | Recommended / default. Correct OCR, fastest decode on Apple Silicon |

| got-ocr2-q8_0.gguf | Q8_0 | 599 MB | Correct OCR; on M1 the Q8_0 mul_mv path is slower per-token than Q4_K, so Q4_K is preferred |

| got-ocr2-f16.gguf | F16 | 1.44 GB | Full precision baseline |

Precision & parity

The Qwen2-0.5B decoder quantizes cleanly to Q4_K and Q8_0 — all three

builds above produce identical, correct OCR. Verified against the real HF model

(transformers GotOcr2) plus a Python f32 reference:

• Vision (ViT layers, neck, downsample, projector): cos ≥ 0.998 vs HF.
• LLM decoder (per-layer, Q8_0 weights vs f32 reference): cos ≥ 0.99996.

Per-token decode speed on an M1 (256 vision tokens spliced into the prompt):

| Build | Decode |

|------|--------|

| Q4_K | ~20 ms/tok |

| F16 | ~38 ms/tok |

| Q8_0 | ~42 ms/tok |

Q4_K is ~2× faster to decode than F16 and 3× smaller, so it is the default.

> Note on earlier builds. A prior version of this repo shipped an

> F16-decoder build and claimed the 0.5B decoder was "catastrophically

> sensitive to quantization" (llm_layer_0 cos ≈ 0.936 at Q8_0). That number

> was a measurement artifact of a per-row bug in the diff harness (it used

> the token count as the row length), not real quant sensitivity. With the

> corrected harness the Q8_0/Q4_K decoder matches f32 at cos ≥ 0.99996 and OCR

> output is identical to F16. See CrispEmbed issue #25.

Usage

crispembed --ocr got-ocr2 image.png

Reproducing the quants

crispembed-quantize got-ocr2-f16.gguf got-ocr2-q4_k.gguf q4_k
crispembed-quantize got-ocr2-f16.gguf got-ocr2-q8_0.gguf q8_0

(The quantizer also has an optional --decoder-f16 flag that keeps the decoder

weights at F16; it is not needed for correctness and is retained only for

diagnostic / comparison use.)

License

Apache-2.0 (same as upstream model)

Provenance and EU AI Act Art. 53 note

• Upstream model: stepfun-ai/GOT-OCR2_0 — published by stepfun-ai.
• Upstream licence: apache-2.0. This repository redistributes under the same terms; it grants no rights the upstream licence does not.
• What was done here: format conversion and/or quantisation only (GGUF). No training, no fine-tuning, no merging, no distillation, no change to architecture, vocabulary or capability. Only the numeric representation of the upstream weights differs.
• Training data: documented — where it is documented at all — by the upstream provider; see the upstream model card. No training data was used, added or selected by this repository. No training-content summary was found on the upstream model card at the time of writing; that documentation gap is upstream's and is not filled here.
• Provider status: under Regulation (EU) 2024/1689 the upstream authors remain the provider of this model. Converting the serialisation format does not make this repository the provider of a new general-purpose AI model, and no such claim is made. Questions about training content, copyright policy or model capability belong upstream.