mudler/qwen3-coder-30b-apex-gguf overview

{
  "metadata": {},
  "card_data": {
    "license": "apache-2.0",
    "base_model": "Qwen/Qwen3-Coder-30B-A3B-Instruct",
    "tags": [
      "gguf",
      "quantized",
      "apex",
      "moe",
      "mixture-of-experts",
      "qwen3",
      "coding"
    ],
    "frontmatter": {
      "license": "apache-2.0",
      "base_model": "Qwen/Qwen3-Coder-30B-A3B-Instruct",
      "tags": [
        "gguf",
        "quantized",
        "apex",
        "moe",
        "mixture-of-experts",
        "qwen3",
        "coding"
      ]
    },
    "hero_image_url": "",
    "summary": "**APEX (Adaptive Precision for EXpert Models)** quantizations of Qwen3-Coder-30B-A3B-Instruct. **Brought to you by the LocalAI team** | APEX Project | Technical Report",
    "quick_links": [],
    "benchmark_table_html": "",
    "readme_markdown": "---\nlicense: apache-2.0\nbase_model: Qwen/Qwen3-Coder-30B-A3B-Instruct\ntags:\n  - gguf\n  - quantized\n  - apex\n  - moe\n  - mixture-of-experts\n  - qwen3\n  - coding\n---\n\n# Qwen3-Coder-30B-A3B APEX GGUF\n\n**APEX (Adaptive Precision for EXpert Models)** quantizations of [Qwen3-Coder-30B-A3B-Instruct](https://huggingface.co/Qwen/Qwen3-Coder-30B-A3B-Instruct).\n\n**Brought to you by the [LocalAI](https://github.com/mudler/LocalAI) team** | [APEX Project](https://github.com/mudler/apex-quant) | [Technical Report](https://github.com/mudler/apex-quant/blob/main/paper/APEX_Technical_Report.pdf)\n\n## Benchmark Results\n\nAll measurements on NVIDIA DGX Spark (GB10, 128 GB VRAM). Perplexity on wikitext-2-raw, context 2048. Accuracy benchmarks via llama.cpp (400 tasks).\n\n| Configuration | Size (GB) | Perplexity | KL mean | HellaSwag | Winogrande | MMLU | ARC | TruthfulQA | tg128 (t/s) |\n|---------------|-----------|-----------|---------|-----------|------------|------|-----|------------|-------------|\n| Q8_0 | 30.3 | 9.537 | 0.0031 | 75.8% | 68.0% | 39.6% | 45.8% | 30.0% | 57.1 |\n| **APEX I-Balanced** | **20.8** | **9.516** | **0.0074** | **76.5%** | **68.3%** | **40.2%** | **46.2%** | **30.4%** | **68.5** |\n| **APEX I-Quality** | **18.1** | **9.535** | **0.0108** | **75.3%** | **68.5%** | **39.8%** | **44.8%** | **30.5%** | **74.1** |\n| **APEX Quality** | **18.1** | **9.560** | **0.0117** | **75.5%** | **68.0%** | **40.1%** | **44.5%** | **31.8%** | **73.7** |\n| **APEX Balanced** | **20.5** | **9.563** | **0.0083** | **75.5%** | **68.5%** | **39.6%** | **45.2%** | **30.5%** | **68.1** |\n| Unsloth Q5_K_S | 19.6 | 9.513 | 0.0119 | 75.3% | 68.5% | 39.8% | 45.2% | 30.2% | 72.2 |\n| Unsloth UD-Q4_K_XL | 16.5 | 9.676 | 0.0246 | 76.3% | 67.0% | 39.7% | 47.5% | 30.5% | 82.3 |\n| **APEX I-Compact** | **13.8** | **9.667** | **0.0418** | **76.3%** | **68.8%** | **39.0%** | **44.1%** | **29.0%** | **84.5** |\n| **APEX Compact** | **13.8** | **9.765** | **0.0492** | **75.0%** | **67.0%** | **39.1%** | **45.8%** | **30.4%** | **83.8** |\n| **APEX Mini** | **11.3** | **9.838** | **0.0862** | **73.5%** | **68.8%** | **39.0%** | **44.1%** | **31.0%** | **91.4** |\n\n### Highlights\n\n- **APEX I-Balanced beats Q8_0** in PPL (9.516 vs 9.537), HellaSwag (76.5% vs 75.8%), MMLU (40.2% vs 39.6%), and ARC (46.2% vs 45.8%) while being **31% smaller** and **20% faster**.\n- **APEX I-Compact matches UD-Q4_K_XL** quality at 16% less size (13.8 vs 16.5 GB) with higher Winogrande (68.8% vs 67.0%).\n- **APEX Mini (11.3 GB)** delivers 91.4 t/s -- fastest of any configuration -- while maintaining viable quality for coding tasks.\n\n## Available Files\n\n| File | Profile | Size | Best For |\n|------|---------|------|----------|\n| Qwen3-Coder-30B-APEX-I-Balanced.gguf | I-Balanced | 20.8 GB | Best overall -- beats Q8_0 quality |\n| Qwen3-Coder-30B-APEX-I-Quality.gguf | I-Quality | 18.1 GB | Best accuracy with imatrix |\n| Qwen3-Coder-30B-APEX-Quality.gguf | Quality | 18.1 GB | Lowest perplexity at this size |\n| Qwen3-Coder-30B-APEX-Balanced.gguf | Balanced | 20.5 GB | General purpose, low KL |\n| Qwen3-Coder-30B-APEX-I-Compact.gguf | I-Compact | 13.8 GB | Consumer GPUs, best quality at size |\n| Qwen3-Coder-30B-APEX-Compact.gguf | Compact | 13.8 GB | Consumer 24 GB GPUs |\n| Qwen3-Coder-30B-APEX-Mini.gguf | Mini | 11.3 GB | 16 GB VRAM, fastest inference |\n\n## What is APEX?\n\nAPEX is a quantization strategy for Mixture-of-Experts (MoE) models. It classifies tensors by role (routed expert, shared expert, attention) and applies a layer-wise precision gradient -- edge layers get higher precision, middle layers get more aggressive compression. I-variants use diverse imatrix calibration (chat, code, reasoning, tool-calling -- no Wikipedia).\n\nSee the [APEX project](https://github.com/mudler/apex-quant) for full details, technical report, and scripts.\n\n## Run with LocalAI\n\n```bash\nlocal-ai run mudler/Qwen3-Coder-30B-APEX-GGUF@Qwen3-Coder-30B-APEX-I-Balanced.gguf\n```\n\n## Credits\n\nAPEX is brought to you by the [LocalAI](https://github.com/mudler/LocalAI) team. Developed through human-driven, AI-assisted research. Built on [llama.cpp](https://github.com/ggerganov/llama.cpp).\n",
    "related_quantizations": []
  },
  "tags": [
    "gguf",
    "quantized",
    "apex",
    "moe",
    "mixture-of-experts",
    "qwen3",
    "coding",
    "base_model:Qwen/Qwen3-Coder-30B-A3B-Instruct",
    "base_model:quantized:Qwen/Qwen3-Coder-30B-A3B-Instruct",
    "license:apache-2.0",
    "endpoints_compatible",
    "region:us",
    "conversational"
  ],
  "likes": 11,
  "downloads": 5094,
  "gated": false,
  "private": false,
  "last_modified": "2026-04-02T11:46:25.000Z",
  "created_at": "2026-04-02T09:19:36.000Z",
  "pipeline_tag": "",
  "library_name": ""
}

{
  "_id": "69ce34a8682255c56854fa17",
  "id": "mudler/Qwen3-Coder-30B-APEX-GGUF",
  "modelId": "mudler/Qwen3-Coder-30B-APEX-GGUF",
  "sha": "18161dc56721be14577fb9f4b0b2f65c665e1262",
  "createdAt": "2026-04-02T09:19:36.000Z",
  "lastModified": "2026-04-02T11:46:25.000Z",
  "author": "mudler",
  "downloads": 5094,
  "likes": 11,
  "gated": false,
  "private": false,
  "pipeline_tag": "",
  "library_name": "",
  "siblings_count": 9
}