---

license: cc-by-nc-sa-4.0

base_model: TheREZOR/TinyTalk-2

language:

• en

pipeline_tag: text-generation

tags:

• gguf
• gpt2
• chat
• conversational
• tiny
• llama.cpp
• ollama

---

TinyTalk 2 — GGUF

GGUF builds of TheREZOR/TinyTalk-2,

an 8M-parameter chatbot originally built to run on an ESP32-S3 microcontroller

(cardputer-ai firmware).

Run with Ollama

ollama run hf.co/TheREZOR/TinyTalk-2-GGUF

Run with llama.cpp

llama-cli -m TinyTalk-2-f16.gguf -p "User: what sound does a dog make?\nBot:" \
  --temp 0.8 --top-p 0.9 -n 40 -r "<|endoftext|>"

Files

| File | Bytes/weight | Size | Note |

|---|---|---|---|

| TinyTalk-2-f16.gguf | f16 | 39 MB | recommended — tiny models lose quality fast below f16 |

| TinyTalk-2-Q8_0.gguf | q8_0 | 21 MB | fine in practice |

Architecture note (why this says "gpt2")

The original model is GPT-Neo, which llama.cpp does not support. These GGUFs

are a mathematically exact GPT-2 conversion (verified: max logit delta

2.6e-05, identical greedy decodes): GPT-Neo's missing 1/sqrt(head_dim)

attention scaling is baked into the q-projection weights, and at the model's

256-token context the alternating local attention (window 256) is identical

to global attention. Conversion script:

tools/export_gpt2.py.

Prompt format

User: <message>
Bot: <reply><|endoftext|>

Story mode: Summary: <what the story is about>\nStory:

The chat template is embedded in the GGUF metadata; <|endoftext|> is the

stop token. Context length: 256.

License

CC BY-NC-SA 4.0 (non-commercial) — see the

main model card for full

dataset attribution.