SoAIHQ/gpt-oss-120b-VISION-v2-GGUF overview

---

base_model: openai/gpt-oss-120b

base_model_relation: quantized

quantized_by: SoAI

license: apache-2.0

license_link: https://huggingface.co/openai/gpt-oss-120b/blob/main/LICENSE

pipeline_tag: text-generation

tags:

• gguf
• llama.cpp
• mxfp4
• soai
• multimodal
• image-text-to-text
• tool-calling
• function-calling
• agent
• reasoning
• harmony
• vision

---

<div align="center">

<a href="https://soai.to"><picture>

<source media="(prefers-color-scheme: dark)" srcset="https://huggingface.co/SoAIHQ/gpt-oss-120b-VISION-v2-GGUF/resolve/main/soai-logo-dark.png">

<img alt="SoAI" src="https://huggingface.co/SoAIHQ/gpt-oss-120b-VISION-v2-GGUF/resolve/main/soai-logo-light.png" width="480">

</picture></a>

</div>

<h1 align="center">gpt-oss-120b-VISION-v2 · GGUF</h1>

<div align="center">

OpenAI's larger open-weight reasoning model with its released weights unchanged, a corrected chat template that fixes malformed tool-call output, and image input.

    

</div>

OpenAI's original gpt-oss-120b weights, unchanged, packed as GGUF for llama.cpp and SoAI.

---

What's new in v2

Its weights are identical to the first release, byte for byte. This release replaces the embedded chat template, fixing malformed output and invalid JSON that gpt-oss could produce around tool calls.

With OpenAI's original template, gpt-oss occasionally wrote a damaged channel header around a tool call, such as <|channel|>comment.... llama.cpp could not parse such replies, so clients received raw text or an error instead of a tool call with valid JSON arguments. gpt-oss users report the same malformed JSON with Ollama. The corrected template opens each reply inside gpt-oss's analysis channel, where it always begins, so the model never has to write that header itself. In our testing, llama.cpp parsed every reply and every tool call arrived with valid JSON arguments.

It also corrects how earlier tool calls and results are written back into a conversation:

| Original template | Effect | This release |

| :--- | :--- | :--- |

| Writes only the first of several parallel tool calls into history | The prompt shows results for calls missing from history | Writes every call as its own message |

| Names every tool result after the most recent call | Results of parallel calls are attributed to the wrong tool | Matches each result to its call by tool_call_id |

| JSON-encodes string arguments and tool results a second time | The model reads escaped JSON inside a string and copies the escaping into new calls | Writes strings as they are |

| Rejects a tool-call message that has both reasoning and visible text | Requests from some clients fail with a template error | Sends reasoning to the analysis channel and text to the commentary channel |

If you already have the first release's files (gpt-oss-120b-VISION-GGUF), keep them and load the corrected template instead of downloading again: get chat_template.jinja from this repository and start llama-server with -m gpt-oss-120b-MXFP4-00001-of-00002.gguf --jinja --chat-template-file chat_template.jinja.

At a glance

| Text | Images | Audio | Reasoning | Tool calling | Context |

| :---: | :---: | :---: | :---: | :---: | :---: |

| Yes | Yes | No | Yes | Yes | 128K |

| Specification | Details |

| :--- | :--- |

| Parameters | 117B total, 5.1B active |

| Architecture | Mixture of experts (128 experts, 4 active) |

| Context window | 131,072 tokens |

| Reasoning | Always on, depth set by reasoning_effort (low, medium, high) |

| Languages | Mainly English |

| Made by | OpenAI |

Choosing a file

| Download | Size | Quality | Best for |

| :--- | ---: | :--- | :--- |

| MXFP4 (part 1 + part 2) | 65.4 GB | Original | The released weights, unchanged. Download both parts and load the first one |

| mmproj (F16) | 651.1 MB | Add-on | Needed only for image input |

> [!TIP]

> This repository has one model build because OpenAI post-trained this model with its expert

> weights already in MXFP4 and ran its published evaluations in that format. This file keeps that format,

> so it is the model as OpenAI released and evaluated it.

Running it takes memory for the file and for its context, which grows with the context length you set. llama.cpp can run this model on GPU, on CPU, or split its layers between both.

Images need the mmproj-gpt-oss-120b-VISION-v2-f16.gguf add-on as well, loaded with --mmproj. llama-server -hf downloads it automatically.

Quick start

With llama.cpp, this command starts a local server with a built-in chat UI and an OpenAI-compatible API:

llama-server -hf SoAIHQ/gpt-oss-120b-VISION-v2-GGUF:MXFP4

Open <http://localhost:8080> to chat, or send requests to the API:

curl http://localhost:8080/v1/chat/completions -H "Content-Type: application/json" -d '{
  "messages": [{"role": "user", "content": "Explain GGUF in one sentence."}],
  "chat_template_kwargs": {"reasoning_effort": "high"}
}'

<details>

<summary>Download the files to a folder first</summary>

hf download SoAIHQ/gpt-oss-120b-VISION-v2-GGUF --include "gpt-oss-120b-VISION-v2-MXFP4-*.gguf" "mmproj-*" --local-dir gpt-oss-120b-VISION-v2-GGUF
llama-server -m gpt-oss-120b-VISION-v2-GGUF/gpt-oss-120b-VISION-v2-MXFP4-00001-of-00002.gguf \
  --mmproj gpt-oss-120b-VISION-v2-GGUF/mmproj-gpt-oss-120b-VISION-v2-f16.gguf

</details>

gpt-oss only works with its harmony prompt format. Its corrected chat template, described above, is embedded in the GGUF, so runtimes apply it automatically and llama.cpp parses tool calls back into OpenAI-compatible tool_calls.

Recommended settings

| Mode | temperature | top_p |

| :--- | :---: | :---: |

| All tasks | 1.0 | 1.0 |

OpenAI recommends these values in the original model card. Reasoning is always on, and low effort gives quick answers. Allow at least 16K output tokens to leave room for long answers after reasoning.

In llama-server, you set reasoning per request with "chat_template_kwargs": {"reasoning_effort": "high"}.

Agents and tool calling

gpt-oss plans in its reasoning. During a sequence of tool calls, send each step's reasoning back with its assistant message, as reasoning_content in llama-server's OpenAI-compatible API. Once the model gives its final answer, the template drops that turn's reasoning from later prompts on its own, as OpenAI's harmony guide describes.

{"role": "assistant", "content": "", "reasoning_content": "Need the file list first.",
 "tool_calls": [{"id": "call_1", "type": "function", "function": {"name": "list_directory", "arguments": "{\"path\": \".\"}"}}]}

Sending it back keeps the model's plan intact from one step to the next, and SoAI does this during tool sequences.

Tested on SoAI

We tested this chat template on SoAI with gpt-oss as a full agent, in Execute mode with all of SoAI's default tools and recommended settings. In every run, llama.cpp parsed every reply, and the model based its final report on tools it had actually called.

How these files are made

Its mixture-of-experts weights are copied in MXFP4, the format OpenAI post-trained and

evaluated them in, and every other tensor keeps its released BF16 precision. Nothing is requantized: conversion

changes only the container, and weights stay as released.

Before packaging, the build checks that chat, thinking and tool-call markers are stored as special

tokens. A conversion that imports those markers

as plain text silently breaks turns, thinking and tool calls, so in that case the build stops instead of

publishing anything.

After those checks, the build swaps in the corrected chat template described above and changes nothing

else. This template is also published here as chat_template.jinja.

The original model reads text only. This repository adds image input through a vision encoder and projector from OpenGVLab/InternVL3_5-GPT-OSS-20B-A4B-Preview (revision aaabe6aa48), which its authors trained for this model family. The mmproj-*.gguf file holds only those vision tensors, and its language model is the original release, so text, reasoning and tool calling work identically with or without it. Its vision weights keep the donor's license, described on its model card.

For best image answers, put your image before your question in one message.

<details>

<summary>All files and SHA-256 checksums</summary>

| File | Type | Size | SHA-256 |

| :--- | :--- | ---: | :--- |

| gpt-oss-120b-VISION-v2-MXFP4-00001-of-00002.gguf | MXFP4 | 44.6 GB | 6e29b189eaaa4323… |

| gpt-oss-120b-VISION-v2-MXFP4-00002-of-00002.gguf | MXFP4 | 20.8 GB | b160e767d5f4111b… |

| mmproj-gpt-oss-120b-VISION-v2-f16.gguf | mmproj (F16) | 651.1 MB | 60cf88b4cea6d1e0… |

The full checksums are in SHA256SUMS.

</details>

Provenance

| Item | Details |

| :--- | :--- |

| Original model | openai/gpt-oss-120b by OpenAI, revision b5c939de8f |

| Converted with | llama.cpp |

| Precision | MoE experts in native MXFP4 and all other tensors in BF16, both as released |

| Weights | The original release, with no fine-tuning, merging or retraining. Only the storage format differs. |

| Chat template | Corrected version of the original, published as chat_template.jinja |

| Built | 2026-10-07 |

License and credits

These files are distributed under the original model's license (apache-2.0), and all of its

terms continue to apply. The license and original model card also

cover this model's capabilities, training data, benchmarks and limitations.

OpenAI designed and trained this model, and SoAI only repackages its

published weights. Conversion uses llama.cpp by Georgi Gerganov

and ggml contributors.

Run it with SoAI

Your models, your hardware, one software suite. Download SoAI for Windows,

macOS, Linux or Android.

<div align="center">

_{Packaged by <a href="https://soai.to">SoAI</a>.}

</div>