---

language:

• en

license: apache-2.0

base_model: openai/gpt-oss-20b

library_name: llama.cpp

pipeline_tag: text-generation

tags:

• openai
• gpt-oss
• gpt-oss-20b
• gguf
• q6_k
• quantized
• reasoning
• local-ai
• llama.cpp

---

OpenAI GPT-OSS 20B - Q6_K GGUF

A Q6_K quantized GGUF release of OpenAI's GPT-OSS 20B,

prepared for local inference and self-hosted AI applications.

Published by oscarwest1.

Model Information

| Property | Details |

|---|---|

| Developer | OpenAI |

| Original Model | openai/gpt-oss-20b |

| Source Weights | unsloth/gpt-oss-20b-BF16 |

| Total Parameters | Approximately 21 Billion |

| Active Parameters | Approximately 3.6 Billion |

| Architecture | Mixture of Experts (MoE) |

| Quantization | Q6_K |

| Model Format | GGUF |

| Runtime | llama.cpp |

| Publisher | oscarwest1 |

| License | Apache 2.0 |

About This Release

This repository provides a Q6_K GGUF conversion of

OpenAI's GPT-OSS 20B model.

The conversion is intended to provide a local inference

option for supported GGUF runtimes.

The source BF16 weights were distributed by Unsloth.

This release does not claim to recover precision lost

in any earlier quantization or conversion steps.

Q6_K is a lossy quantization format. Differences in

quality, memory usage and performance depend on

the model, hardware and inference configuration.

No additional fine-tuning or behavioral modification

was performed as part of this GGUF quantization.

Capabilities

The upstream GPT-OSS model is designed for:

• Reasoning and analytical tasks
• Programming assistance
• Technical troubleshooting
• Research and experimentation
• Instruction following
• Local AI assistants
• Self-hosted inference
• Structured problem solving

Actual results depend on the inference runtime,

prompt formatting and model configuration.

Local Inference

llama.cpp

Download the GGUF file and run:

llama-cli -m gpt-oss-20b-Q6_K.gguf -c 32768 -cnv

Local API Server

llama-server -m gpt-oss-20b-Q6_K.gguf --host 127.0.0.1 --port 8080 -c 32768

Use a compatible llama.cpp version with GPT-OSS support.

Quantization Details

• Format: GGUF
• Quantization: Q6_K
• Source: Unsloth GPT-OSS 20B BF16
• Quantization tools: llama.cpp
• Fine-tuning: None performed for this release
• Independent quality benchmarks: Not provided

This release is not guaranteed to outperform the

original MXFP4 distribution.

Original Model and Attribution

Original model by OpenAI:

https://huggingface.co/openai/gpt-oss-20b

Source BF16 distribution:

https://huggingface.co/unsloth/gpt-oss-20b-BF16

GGUF conversion and quantization:

https://github.com/ggml-org/llama.cpp

GPT-OSS is an OpenAI open-weight model family.

This is an independent community quantization,

not an official OpenAI or ChatGPT release.

License

Apache License 2.0.

Please review the original model repository for

applicable license terms and attribution requirements.

---

A Personal Note

In loving memory of NYZ.

I miss you, NYZ.