---

license: apache-2.0

base_model: allenai/Olmo-3-7B-Instruct

base_model_relation: quantized

pipeline_tag: text-generation

library_name: gguf

tags:

- gguf

- llama.cpp

- olmo

- allenai

language:

- en

---

Olmo 3 7B Instruct - GGUF

Quantized from allenai/Olmo-3-7B-Instruct.

All credit to Ai2 - this is a format

conversion, with the chat template tweaked to suit llama.cpp.

Chat template

llama.cpp renders the template with tools undefined rather than none, so we

guarded the two places that care:

-{%- if tools is none -%}
+{%- if tools is not defined or tools is none -%}
-{%- if tools is not none -%}
+{%- if tools is defined and tools is not none -%}

That is the whole change. Plain chat comes out exactly as Ai2 wrote it.

Files

| | Size | |

|---|---|---|

| olmo-3-7b-instruct-q4_k_m.gguf | 4.5 GB | the only one so far, ask if you want Q8_0 |

Running it

llama-server -m olmo-3-7b-instruct-q4_k_m.gguf --ctx-size 8192

Trained context is 65536. 8192 is just what we run.

Made with

llama.cpp b10223, --convert then --quantize out of the :full image,

template tweaked before converting.