---

license: apache-2.0

pipeline_tag: text-generation

library_name: gguf

base_model: unsloth/gpt-oss-20b-GGUF

tags:

- gguf

- gpt-oss

- openai

- conversational

- layer-shards

---

Model Card for aquaduck/gpt-oss-20b-GGUF

Layer-sharded Q4_K_M GGUF chunks of OpenAI gpt-oss-20b, derived from unsloth/gpt-oss-20b-GGUF.

These files are not a new quantization. They are contiguous 4-layer packages cut from the full Unsloth Q4_K_M GGUF for staged / multi-node loading (Aquaduck Arc layer-package-v1).

Model lineage

openai/gpt-oss-20b

└── quantized → unsloth/gpt-oss-20b-GGUF (Q4_K_M)

└── layer-split → aquaduck/gpt-oss-20b-GGUF (this repo)

• Base weights: https://huggingface.co/openai/gpt-oss-20b (Apache 2.0)
• Quantization source: https://huggingface.co/unsloth/gpt-oss-20b-GGUF (tag Q4_K_M)
• This repo: 4-layer GGUF shards for distributed inference

Model Details

Model Description

• Developed by: Aquaduck (layer packaging only; base model by OpenAI; GGUF quant by Unsloth / llama.cpp ecosystem)
• Shared by: Aquaduck AI
• Model type: Causal language model (gpt-oss MoE), GGUF Q4_K_M, layer-sharded
• Language(s): Multilingual (same as base)
• License: Apache-2.0 (inherits from OpenAI gpt-oss)
• Finetuned from model: N/A — not a fine-tune
• Derived from: unsloth/gpt-oss-20b-GGUF ← openai/gpt-oss-20b

Model Sources

• Base model card / paper: https://huggingface.co/openai/gpt-oss-20b , https://arxiv.org/abs/2508.10925
• Quantized GGUF source: https://huggingface.co/unsloth/gpt-oss-20b-GGUF

Files

| File | Layers (inclusive start … end exclusive in filename) | Approx. size |

|------|------------------------------------------------------|--------------|

| gpt-oss-20b-Q4_K_M-layers-0-4.gguf | 0–3 | ~2.18 GB |

| gpt-oss-20b-Q4_K_M-layers-4-8.gguf | 4–7 | ~2.18 GB |

| gpt-oss-20b-Q4_K_M-layers-8-12.gguf | 8–11 | ~2.18 GB |

| gpt-oss-20b-Q4_K_M-layers-12-16.gguf | 12–15 | ~2.18 GB |

| gpt-oss-20b-Q4_K_M-layers-16-20.gguf | 16–19 | ~2.18 GB |

| gpt-oss-20b-Q4_K_M-layers-20-24.gguf | 20–23 | ~2.79 GB |

• Total layers: 24
• Valid split boundaries: 4, 8, 12, 16, 20
• Full-model equivalent: ~11.6 GB (same as Unsloth Q4_K_M)

Filenames use exclusive end indices (layers-{start}-{endExclusive}).

Uses

Direct Use

Intended for Aquaduck / Arc (and compatible) runtimes that download and merge layer packages, not as a single drop-in file for stock llama.cpp.

For a normal single-file local run, use unsloth/gpt-oss-20b-GGUF or openai/gpt-oss-20b instead.

Use the base model’s harmony chat format; other formats will not work correctly.

Out-of-Scope Use

• Expecting any one chunk to run as a complete model
• Treating this repo as a new training run or re-quant
• Uses prohibited by the Apache-2.0 license or OpenAI’s model card guidance

Bias, Risks, and Limitations

Same capabilities, biases, and risks as openai/gpt-oss-20b. Q4_K_M quantization (from Unsloth) can degrade quality vs. the original MXFP4 / higher-precision releases. Layer sharding does not change weights beyond packaging.

Recommendations

Follow OpenAI’s gpt-oss docs for reasoning levels, tools, and harmony. Prefer the full Unsloth GGUF when you do not need staged loading.

How to Get Started

These shards are meant to be loaded automatically by an Aquaduck worker.

1. Install / open the Aquaduck desktop app and sign in as a worker.
2. Connect so the app can fetch your device’s model assignment from the model catalog (openai/gpt-oss-20b).
3. If model shards do not automatically download: Use Get latest assigned model (under Settings). The worker will:

- download only the contiguous 4-layer chunk GGUFs for your assigned stage from this repo

- merge multi-chunk stages into one stage GGUF

- keep that stage ready for split serving

You do not need to pick individual -layers-.gguf files. Assignment and download are driven by model catalog metadata.

This repo is not a single-file Q4_K_M drop-in for stock llama.cpp. For that, use https://huggingface.co/unsloth/gpt-oss-20b-GGUF instead.

Training Details

No training. Weights come from OpenAI; Q4_K_M GGUF from Unsloth; this repo only splits that GGUF into 4-layer packages.

Evaluation

No separate evals for the shards. See openai/gpt-oss-20b and https://arxiv.org/abs/2508.10925.

Technical Specifications

• Architecture: gpt-oss (21B params, ~3.6B active)
• Quantization: Q4_K_M (Unsloth)
• Packaging: Arc gguf-shard / catalog catalog:split → *-layers-{start}-{endExclusive}.gguf
• Package format: layer-package-v1

Citation

@misc{openai2025gptoss120bgptoss20bmodel,
  title={gpt-oss-120b & gpt-oss-20b Model Card},
  author={OpenAI},
  year={2025},
  eprint={2508.10925},
  archivePrefix={arXiv},
  primaryClass={cs.CL},
  url={https://arxiv.org/abs/2508.10925},
}

Credit:

• Unsloth (https://huggingface.co/unsloth/gpt-oss-20b-GGUF) for the GGUF quantization
• llama.cpp (https://github.com/ggml-org/llama.cpp) for GGUF support

Model Card Contact

Aquaduck AI — https://huggingface.co/aquaduck