SoAIHQ/gpt-oss-20b-VISION-GGUF overview

---

base_model: openai/gpt-oss-20b

base_model_relation: quantized

quantized_by: SoAI

license: apache-2.0

license_link: https://huggingface.co/openai/gpt-oss-20b/blob/main/LICENSE

pipeline_tag: text-generation

tags:

• gguf
• llama.cpp
• mxfp4
• soai
• multimodal
• image-text-to-text

---

<div align="center">

<a href="https://soai.to"><picture>

<source media="(prefers-color-scheme: dark)" srcset="https://huggingface.co/SoAIHQ/gpt-oss-20b-VISION-GGUF/resolve/main/soai-logo-dark.png">

<img alt="SoAI" src="https://huggingface.co/SoAIHQ/gpt-oss-20b-VISION-GGUF/resolve/main/soai-logo-light.png" width="480">

</picture></a>

</div>

<h1 align="center">gpt-oss-20b-VISION · GGUF</h1>

<div align="center">

OpenAI's open-weight reasoning model with its released weights unchanged, plus experimental image input.

    

</div>

OpenAI's original gpt-oss-20b weights, unchanged, packed as GGUF for llama.cpp and SoAI.

> [!TIP]

> A newer version is available: gpt-oss-20b-VISION-v2-GGUF. Its updated chat template improves reasoning and tool-call handling while preserving the same model weights. We recommend v2 for new downloads. If you already have these files, see the v2 model card for instructions on using its corrected chat template without downloading the weights again.

---

At a glance

| Text | Images | Audio | Reasoning | Tool calling | Context |

| :---: | :---: | :---: | :---: | :---: | :---: |

| Yes | Experimental | No | Yes | Yes | 128K |

| Specification | Details |

| :--- | :--- |

| Parameters | 21B total, 3.6B active |

| Architecture | Mixture of experts (32 experts, 4 active) |

| Context window | 131,072 tokens |

| Reasoning | Always on, depth set by reasoning_effort (low, medium, high) |

| Languages | Mainly English |

| Made by | OpenAI |

Choosing a file

| Download | Size | Quality | Best for |

| :--- | ---: | :--- | :--- |

| MXFP4 | 13.8 GB | Original | The released weights, unchanged |

| mmproj (F16) | 651.1 MB | Add-on | Needed only for image input |

> [!TIP]

> This repository has one model build because OpenAI post-trained the model with its expert

> weights already in MXFP4 and ran its published evaluations in that format. A Q4_K_M build would expand

> those 4-bit experts and round them again into another format, which adds error on top of the original

> rounding. A Q8_0 build would roughly double the size without recovering any precision.

The model needs memory for the file and for the context, and the context part grows with the context length you set. llama.cpp can run the model on the GPU, on the CPU, or with its layers split between them.

Images need the mmproj-gpt-oss-20b-f16.gguf add-on as well, loaded with --mmproj. llama-server -hf downloads it automatically.

Quick start

With llama.cpp, this command starts a local server with a built-in chat UI and an OpenAI-compatible API:

llama-server -hf SoAIHQ/gpt-oss-20b-VISION-GGUF:MXFP4

Open <http://localhost:8080> to chat, or send requests to the API:

curl http://localhost:8080/v1/chat/completions -H "Content-Type: application/json" -d '{
  "messages": [{"role": "user", "content": "Explain GGUF in one sentence."}],
  "chat_template_kwargs": {"reasoning_effort": "high"}
}'

<details>

<summary>Download the files to a folder first</summary>

hf download SoAIHQ/gpt-oss-20b-VISION-GGUF --include "gpt-oss-20b-MXFP4.gguf" "mmproj-*" --local-dir gpt-oss-20b-VISION-GGUF
llama-server -m gpt-oss-20b-VISION-GGUF/gpt-oss-20b-MXFP4.gguf \
  --mmproj gpt-oss-20b-VISION-GGUF/mmproj-gpt-oss-20b-f16.gguf

</details>

The model only works with its harmony prompt format. The original chat template that produces it is embedded in the GGUF, so runtimes apply it automatically, and llama.cpp parses tool calls back into OpenAI-compatible tool_calls.

Recommended settings

| Mode | temperature | top_p |

| :--- | :---: | :---: |

| All tasks | 1.0 | 1.0 |

OpenAI recommends these values in the original model card. Reasoning cannot be switched off, so use low effort for quick answers. Allow at least 16K output tokens to leave room for long answers after the reasoning.

In llama-server, you set reasoning per request with "chat_template_kwargs": {"reasoning_effort": "high"}.

How these files are made

The mixture-of-experts weights are copied in the MXFP4 format that the model was post-trained and

evaluated in, and every other tensor keeps its released BF16 precision. Nothing is requantized, so the

conversion changes the container and leaves the weights as released. Some other GGUF builds of this model

compress part of it further, which this build does not do.

Before any quantization, the build checks that the chat, thinking and tool-call markers are stored as

special tokens and that the embedded chat template matches the original. A conversion that imports those

markers as plain text breaks turns, thinking and tool calls without any error message, so the build stops

instead of publishing the file.

The original model reads text only. This repository adds experimental image input through the vision encoder and projector from OpenGVLab/InternVL3_5-GPT-OSS-20B-A4B-Preview (revision aaabe6aa48), which its authors trained for this model family. The mmproj-*.gguf file contains only those vision tensors and the language model is the original release, so text, reasoning and tool calling work the same with or without it. The image input has not been benchmarked, and the vision weights keep the donor's license, which the donor model card describes.

<details>

<summary>All files and SHA-256 checksums</summary>

| File | Type | Size | SHA-256 |

| :--- | :--- | ---: | :--- |

| gpt-oss-20b-MXFP4.gguf | MXFP4 | 13.8 GB | aab205256a9b6361… |

| mmproj-gpt-oss-20b-f16.gguf | mmproj (F16) | 651.1 MB | 29b69220b8fa8da9… |

The full checksums are in SHA256SUMS.

</details>

Provenance

| Item | Details |

| :--- | :--- |

| Original model | openai/gpt-oss-20b by OpenAI, revision 6cee5e81ee |

| Converted with | llama.cpp v0.5.0, commit 7fe450e19 |

| Precision | MoE experts in native MXFP4 and all other tensors in BF16, both as released |

| Weights | The original release, with no fine-tuning, merging or retraining. Only the storage format differs. |

| Built | 2026-09-26 |

License and credits

These files are distributed under the original model's license (apache-2.0), and its full

terms continue to apply. The license and the original model card

also describe the model's capabilities, training data, benchmarks and limitations.

OpenAI designed and trained the model, and SoAI only repackages the

published weights. The conversion uses llama.cpp by Georgi

Gerganov and the ggml contributors.

Run it with SoAI

Your models, your hardware, one software suite. Download SoAI for Windows,

macOS, Linux or Android.

<div align="center">

_{Packaged by <a href="https://soai.to">SoAI</a>.}

</div>