davidfred/gpt-oss-20b-f16-gguf overview

{
  "metadata": {},
  "card_data": {
    "frontmatter": {},
    "hero_image_url": "",
    "summary": "This is a high-quality F16 GGUF conversion of the GPT-OSS 20B model, optimized for llama.cpp inference.",
    "quick_links": [],
    "benchmark_table_html": "",
    "readme_markdown": "\nlicense: apache-2.0\nlanguage:\n- en\ntags:\n- gguf\n- quantized\n- gpt-oss\n- llama-cpp\n- mixture-of-experts\n- f16\nmodel_type: gpt-oss\n---\n\n# GPT-OSS 20B F16 GGUF\n\nThis is a high-quality F16 GGUF conversion of the GPT-OSS 20B model, optimized for llama.cpp inference.\n\n## Model Details\n\n- **Model Type**: GPT-OSS (Mixture of Experts)\n- **Parameters**: 20.91B total, 1.8B active\n- **Precision**: F16 (16-bit floating point)\n- **File Size**: 12.83 GiB\n- **Context Length**: 131,072 tokens\n- **Experts**: 32 total, 4 active per token\n- **Architecture**: Sliding window attention with expert routing\n\n## Key Features\n\n- ✅ **Fully Functional**: All 459 tensors intact, including mxfp4 expert weights\n- ✅ **High Quality**: F16 precision maintains model performance\n- ✅ **Complete MoE Support**: Expert routing and gating fully preserved\n- ✅ **Extended Context**: 131K token context window with YARN scaling\n\n## Usage with llama.cpp\n\n```bash\n# Download the model\nhuggingface-cli download davidfred/gpt-oss-20b-f16-gguf gpt-oss-20B-F16.gguf\n\n\n## Hardware Requirements\n\n- **Minimum RAM**: 16 GB (recommended: 24 GB+)\n- **CPU**: Multi-core recommended (tested on 8 vCPU)\n- **Storage**: ~13 GB free space\n- **OS**: Compatible with llama.cpp (Linux, Windows, macOS)\n\n## Performance Notes\n\n- Efficiently runs on CPU-only setups\n- Utilizes mixture of experts for optimal parameter efficiency\n- Supports both interactive and batch inference modes\n- Compatible with llama.cpp server mode for API access\n\n## Model Origin\n\nConverted from the original GPT-OSS 20B model using llama.cpp conversion tools. This F16 GGUF preserves all model capabilities while providing efficient inference performance.\n\n## License\n\nApache 2.0 - Same as the original GPT-OSS model.\nEOF\n\n# Upload the README\nhuggingface-cli upload davidfred/gpt-oss-20b-f16-gguf /tmp/README.md README.md\n\n./llama-cli -m gpt-oss-20B-F16.gguf --prompt \"Your prompt here\" -n 128 --threads 8\n\n\n\n## Hardware Requirements\n\n- **Minimum RAM**: 16 GB (recommended: 24 GB+)\n- **CPU**: Multi-core recommended (tested on 8 vCPU)\n- **Storage**: ~13 GB free space\n- **OS**: Compatible with llama.cpp (Linux, Windows, macOS)\n\n## Performance Notes\n\n- Efficiently runs on CPU-only setups\n- Utilizes mixture of experts for optimal parameter efficiency\n- Supports both interactive and batch inference modes\n- Compatible with llama.cpp server mode for API access\n\n## Model Origin\n\nConverted from the original GPT-OSS 20B model using llama.cpp conversion tools. This F16 GGUF preserves all model capabilities while providing efficient inference performance.\n\n## License\n\nApache 2.0 - Same as the original GPT-OSS model.\nEOF\n\n# Upload the README\nhuggingface-cli upload davidfred/gpt-oss-20b-f16-gguf /tmp/README.md README.md\nQuick Upload Script (Alternative)\nIf you prefer a single script approach:\n\ncat > /tmp/upload_model.py << 'EOF'\nfrom huggingface_hub import HfApi, create_repo\nimport os\n\n# Configuration\nrepo_id = \"davidfred/gpt-oss-20b-f16-gguf\"\nmodel_path = os.path.expanduser(\"~/openai/gpt-oss-20b/gpt-oss-20B-F16.gguf\")\n\n# Create repository\ncreate_repo(repo_id, exist_ok=True)\n\n# Initialize API\napi = HfApi()\n\n# Upload main model file\nprint(\"Uploading main model file...\")\napi.upload_file(\n    path_or_fileobj=model_path,\n    path_in_repo=\"gpt-oss-20B-F16.gguf\",\n    repo_id=repo_id,\n    commit_message=\"Add GPT-OSS 20B F16 GGUF model\"\n)\n\nprint(f\"✅ Model uploaded successfully!\")\nprint(f\"🔗 Repository: https://huggingface.co/{repo_id}\")\nprint(f\"📁 Direct download: https://huggingface.co/{repo_id}/resolve/main/gpt-oss-20B-F16.gguf\")\nEOF\n\npython /tmp/upload_model.py\n\n",
    "related_quantizations": []
  },
  "tags": [
    "gguf",
    "gpt_oss",
    "endpoints_compatible",
    "mxfp4",
    "region:us",
    "conversational"
  ],
  "likes": 0,
  "downloads": 99,
  "gated": false,
  "private": false,
  "last_modified": "2025-08-07T17:19:21.000Z",
  "created_at": "2025-08-07T02:58:17.000Z",
  "pipeline_tag": "",
  "library_name": ""
}

{
  "_id": "68941649ee204d3dd5d206bf",
  "id": "davidfred/gpt-oss-20b-f16-gguf",
  "modelId": "davidfred/gpt-oss-20b-f16-gguf",
  "sha": "31d01c5da3f939110fb113618724900c3ac66d5a",
  "createdAt": "2025-08-07T02:58:17.000Z",
  "lastModified": "2025-08-07T17:19:21.000Z",
  "author": "davidfred",
  "downloads": 99,
  "likes": 0,
  "gated": false,
  "private": false,
  "pipeline_tag": "",
  "library_name": "",
  "siblings_count": 9
}