ChristianTeroerde/ovgenai-gguf-tokenizer-oob-poc overview
PoC: heap OOB reads in OpenVINO GenAI's GGUF tokenizer builder Security proof of concept for two memory safety bugs one root cause when openvinotoolkit/openvin…
Runs locally from ~0.0 MB disk (4 GB VRAM class GPUs with llama.cpp / guIDE).
Repository Files & Downloads
Model Details
| Model ID | ChristianTeroerde/ovgenai-gguf-tokenizer-oob-poc |
|---|---|
| Author | ChristianTeroerde |
| Pipeline | — |
| License | mit |
| Base model | — |
| Last modified | 2026-06-18T07:56:47.000Z |
Model README
---
license: mit
tags:
- security-poc
- gguf
- openvino
---
PoC: heap OOB-reads in OpenVINO GenAI's GGUF tokenizer builder
Security proof-of-concept for two memory-safety bugs (one root cause) when
openvinotoolkit/openvino.genai
builds a tokenizer from a crafted .gguf (src/cpp/src/gguf_utils/gguf_tokenizer.cpp).
Reached by ov::genai::Tokenizer("<file>.gguf") or LLMPipeline("<file>.gguf","CPU").
The crafted files are benign PoCs — they contain no payload; they only make the
tokenizer index heap buffers out of bounds so the bug is observable under valgrind.
Affected: openvino-genai >= 2025.2.0.0 (GGUF loader's first release) through current
master, verified at 2025.2.0.0 / 2025.4.0.0 / 2026.2.1.0. Distinct from llama.cpp/ggml
(openvino.genai's own tokenizer code).
Two PoCs
unk-index/ — attacker-controlled index → OOB-read + crash (CWE-129) — the strong one
parse_bbpe_config() (:423-428) uses a file-controlled u32 unknown_token_id directly as
a std::vector<std::string> index with no bounds check:
uint32_t unknown_token_id = tensor.data<uint32_t>()[0]; // file-controlled, 0..4294967295
unk_token = vocab_from_config[unknown_token_id]; // operator[] — OOB
crafted.gguf: 256 tokens, unknown_token_id = 512 → vocab_from_config[512] reads a
std::string past the vector; copying it dereferences the garbage pointer → SIGSEGV (DoS).
cd unk-index && OVGENAI_VER=2026.2.1.0 ./run.sh # valgrind "Invalid read of size 8" in parse_bbpe_config, then SIGSEGV
cross-array/ — token/token_type length mismatch → silent OOB-read (CWE-125)
create_tokenizer_from_config() (:472-478) and parse_bbpe_config() (:408-414) loop to the
token count while indexing the independent token_type buffer, with no N == M check.
crafted.gguf: 4096 tokens, 1 token_type → reads ~16 KB past the 4-byte i32 tensor.
cd cross-array && OVGENAI_VER=2026.2.1.0 ./run.sh # valgrind "Invalid read of size 4" in create_tokenizer_from_config
Each folder has build_gguf.py (regenerates the file), crafted.gguf, load_crafted.py,
run.sh (docker + valgrind), and poc-evidence.txt / valgrind-full.log.
Suggested fix
Bounds-check unknown_token_id against vocab.size() before operator[]; require
tokens.size() == token_type.size() before the special-token loops.
Run ChristianTeroerde/ovgenai-gguf-tokenizer-oob-poc with guIDE
Download guIDE — the AI-native code editor with local LLM inference and 69 built-in tools.
Source: Hugging Face · Compare models