---

license: apache-2.0

base_model: allenai/Olmo-3-7B-Think

base_model_relation: quantized

pipeline_tag: text-generation

library_name: gguf

tags:

- gguf

- llama.cpp

- olmo

- allenai

- reasoning

language:

- en

---

Olmo 3 7B Think - GGUF

Quantized from allenai/Olmo-3-7B-Think.

All credit to Ai2 - this is a format

conversion, with one change to the chat template so llama.cpp can tell the

thinking apart from the answer.

Chat template

The stock template puts the opening <think> into the prompt itself:

{{ '<|im_start|>assistant\n<think>' }}

So generation starts already inside a thought, and nothing downstream ever sees

one begin. llama.cpp cannot lift the reasoning into reasoning_content, and if

you apply a response_format schema the grammar takes the content stream from

the first token - which is the same stream the model wants to think in. It then

does not think at all.

Deleting the prefill lets the model emit its own opening tag:

-{{ '<|im_start|>assistant\n<think>' }}
+{{ '<|im_start|>assistant\n' }}

That is the whole change. Nothing else moves, and the weights are untouched -

the file is byte-identical to the source apart from the template field.

Files

| | Size | |

|---|---|---|

| olmo-3-7b-think-q4_k_m.gguf | 4.5 GB | the only one so far, ask if you want Q8_0 |

Running it

llama-server -m olmo-3-7b-think-q4_k_m.gguf --ctx-size 32768 \
    --reasoning on -ngl 99

It thinks at length - 20,000 to 50,000 characters on a tic tac toe move - so

give it room. --max-tokens 20000 is not excessive, and a small budget gets

you an empty answer rather than a short one.

Made with

gguf_new_metadata.py from llama.cpp b10223, rewriting only

tokenizer.chat_template on

lmstudio-community/Olmo-3-7B-Think-GGUF.

All 355 tensors are theirs, unchanged.