---

base_model: openai/gpt-oss-120b

base_model_relation: quantized

library_name: gguf

pipeline_tag: text-generation

tags:

• quantization
• gguf
• mixture-of-experts
• text-generation

---

gpt-oss-120b — PS-IQ4_NL GGUF

GGUF quantization of gpt-oss-120b by PeasantSmith. Derived release, not the original: check the upstream card for license, intended use, and citation.

Specs

Format: GGUF

Size: 71.75 GB / 66.8 GiB, 4.91 bits/param

Source: F16 GGUF (233.78 GB), converted from unsloth/gpt-oss-120b-BF16, not the native MXFP4 release

Context: 131,072 tokens

Architecture: MoE, 36 layers x 128 experts, 5.1B active params

Quantization split

| Tensor class | Type | Count |

|---|---|---|

| Expert gate/up (ffn_gate_exps, ffn_up_exps) | IQ4_NL | 72 |

| Expert down (ffn_down_exps) | Q5_0 | 36 |

| Attention, router, output (attn_*, ffn_gate_inp, output) | Q8_0 | 146 |

| Norms / biases | F32 | 433 |

Why not IQ2_XXS/Q2_K (the smaller PS-IQ2_XXS family): gpt-oss-120b's tensors are all 2880-column (2880 % 256 = 64). IQ2_XXS and Q2_K use 256-element blocks and are geometrically impossible on this model. IQ4_NL uses 32-element blocks, which fit exactly. This is a hard constraint of the model geometry, independent of calibration data.

The imatrix was generated (160 chunks, 108/108 expert tensors covered, zero NaN/Inf) but is not consumed by 32-element-block types in llama.cpp.

Testing

Raw perplexity doesn't work here: gpt-oss needs the harmony response format, and even the F16 source returns NaN on unformatted text past chunk 8. Validated with task-based tests instead:

Quantization quality was validated with task-based tests instead:

| Test | Result |

|---|---|

| Greedy generation (factual question) | PASS, correct answer + complete reasoning block |

| LRU cache implementation task (8192 ctx) | PASS, doubly-linked-list, O(1) get/put, coherent |

| Needle recall (magic value 9137@8k ctx)| PASS, exact answer: 9137 |

| Runtime errors | 0 |

Performance (measured)

• Prompt processing: 65.6–66.1 t/s
• Generation: 14.5–14.7 t/s

Hardware: 3× NVIDIA RTX 3060 12GB, 36 GB VRAM total, 123 GiB RAM DDR4. Expert tensors are CPU-resident via llama.cpp --n-cpu-moe; attention is GPU-resident. This is the hardware-realistic configuration for a 117B-class model on this box.

Recommended parameters

• Temperature: 1.0
• Top-k: 0 (i.e. all)
• Top-p: 1.0

Example llama.cpp invocation:

llama-cli -m gpt-oss-120b-PS-IQ4_NL.gguf \
  -c 8192 -fa -ngl 99 -ncmoe 36 \
  --split-mode layer --tensor-split 1,1,1 \
  --temp 1.0 --top-k 0 --top-p 1.0

Flag names vary across llama.cpp builds.

Reproduce/Verify

llama-quantize \
  --tensor-type "ffn_gate_exps=iq4_nl" \
  --tensor-type "ffn_up_exps=iq4_nl" \
  --tensor-type "ffn_down_exps=q5_0" \
  --token-embedding-type q8_0 \
  gpt-oss-120b-F16.gguf gpt-oss-120b-PS-IQ4_NL.gguf q8_0

License

Quantized derivative of gpt-oss-120b: license and usage terms follow upstream, verify before redistribution. Base model by OpenAI, quantization by PeasantSmith.

Integrity

SHA-256: 4421fab9f392d8d1005abb0df47f06422e79edbd26e8c2ba823da8eae897f428 File size: 71,748,100,896 bytes. Verify after download with sha256sum.

Additional information

The model name PS-IQ4_NL is the PeasantSmith release signature. It identifies this quantized release and should not be confused with the upstream model's original naming.