---

license: cc-by-nc-4.0

language:

• ro
• en

base_model: rostlabs/rost-1b-instruct-v2

tags:

• gguf
• llama.cpp
• romanian
• bilingual

pipeline_tag: text-generation

---

rost-1b-instruct-v2-GGUF

llama.cpp quantisations of

rostlabs/rost-1b-instruct-v2,

converted from revision 53597eef9d83dd2433d26fb737bf18dc648a06d4. The weights

in that revision are byte-identical to d833f89f, the revision every published

benchmark number was measured from.

| file | size | notes |

|---|---:|---|

| rost-1b-instruct-v2-BF16.gguf | 2.58 GiB | the precision the model was trained in |

| rost-1b-instruct-v2-Q8_0.gguf | 1.37 GiB | |

| rost-1b-instruct-v2-Q6_K.gguf | 1.06 GiB | |

| rost-1b-instruct-v2-Q4_K_M.gguf | 791 MiB | smallest |

You need a llama.cpp that knows this architecture

llama.cpp compiles architectures in rather than loading them dynamically, so

stock builds will not read these files. A fork with the one commit applied is at

rostlabs/llama.cpp, whose master is

upstream master plus model: add rost.

git clone -b master https://github.com/rostlabs/llama.cpp && cd llama.cpp
cmake -B build -DCMAKE_BUILD_TYPE=Release && cmake --build build -j
./build/bin/llama-server --model rost-1b-instruct-v2-BF16.gguf --ctx-size 4096

Ollama and LM Studio will not work. Both bundle their own llama.cpp, so

neither reads these files until the architecture is merged upstream.

How faithful is the conversion

The conversion is not a repacking. It absorbs the checkpoint's transposed rotary

convention into the sign of the q and k projections and turns a weightless QK

norm into ordinary norm weights, so the GGUF is an equivalent model built

differently — and equivalence is a claim worth checking rather than assuming.

Checked against the greedy generations recorded by the release's repetition

protocol, six prompts, temperature 0:

• Five of six reproduce character-for-character, including a 1,269-character

essay and a 493-character English answer.

• One diverges, after 768 identical characters, on the seasons prompt: the

recorded output reads Iarna (septembrie-noiembrie) and the GGUF reads

Iarna (septembrie-30 noiembrie).

That prompt is the model's known degenerate case — it is the one prompt v2 loops

on under every decoding setting tested, documented on the base model's card. Deep

inside a low-confidence repetitive region, an argmax can flip on floating-point

accumulation-order differences between two implementations. So the honest

statement is near-exact agreement, not bitwise equivalence: over roughly two

thousand characters of other output there is no difference at all, and the single

difference appears where the model is least certain.

If you need guaranteed parity with the published benchmark numbers, use the

safetensors model rather than a GGUF.

Decoding

The source repository deliberately prescribes no sampling parameters — no tested

configuration passed its preregistered gate of zero looping and zero token-cap

endings across six repetition-stress prompts. These files inherit that: they

carry the tokenizer, both stop tokens and the chat template, and no sampling

defaults. Choose your own, and see the base model's card for what was measured.

Licence

CC-BY-NC-4.0, inherited from the base model's training data.