jc-builds/qwen2.5-0.5b-instruct-q4_k_m-gguf overview

{
  "metadata": {},
  "card_data": {
    "license": "apache-2.0",
    "base_model": "Qwen/Qwen2.5-0.5B-Instruct",
    "tags": [
      "gguf",
      "quantized",
      "llama.cpp",
      "ios",
      "mobile",
      "edge"
    ],
    "model_type": "qwen2",
    "quantized_by": "jc-builds",
    "frontmatter": {
      "license": "apache-2.0",
      "base_model": "Qwen/Qwen2.5-0.5B-Instruct",
      "tags": [
        "gguf",
        "quantized",
        "llama.cpp",
        "ios",
        "mobile",
        "edge"
      ],
      "model_type": "qwen2",
      "quantized_by": "jc-builds"
    },
    "hero_image_url": "",
    "summary": "This is a **Q4_K_M quantized GGUF** conversion of Qwen/Qwen2.5-0.5B-Instruct optimized for on-device inference with llama.cpp.",
    "quick_links": [],
    "benchmark_table_html": "",
    "readme_markdown": "---\nlicense: apache-2.0\nbase_model: Qwen/Qwen2.5-0.5B-Instruct\ntags:\n  - gguf\n  - quantized\n  - llama.cpp\n  - ios\n  - mobile\n  - edge\nmodel_type: qwen2\nquantized_by: jc-builds\n---\n\n# Qwen2.5-0.5B-Instruct Q4_K_M GGUF\n\nThis is a **Q4_K_M quantized GGUF** conversion of [Qwen/Qwen2.5-0.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct) optimized for on-device inference with llama.cpp.\n\n## Model Details\n\n| Property | Value |\n|----------|-------|\n| **Original Model** | Qwen2.5-0.5B-Instruct |\n| **Parameters** | 490 million (360M non-embedding) |\n| **Quantization** | Q4_K_M (4-bit, medium quality) |\n| **File Size** | ~379 MB |\n| **Context Window** | 32,768 tokens |\n| **Architecture** | Qwen2 (RoPE, SwiGLU, RMSNorm) |\n\n## Intended Use\n\nThis model is optimized for:\n- **Mobile/Edge Deployment**: Runs efficiently on all iOS devices including older models\n- **llama.cpp Integration**: Compatible with llama.cpp and its bindings\n- **On-Device AI**: Private, offline inference without cloud dependencies\n\n## Capabilities\n\n- **Best-in-class for sub-1B**: Excellent performance for its size\n- **Multilingual**: Supports 29+ languages\n- **Long Context**: 32K token context window\n- **Structured Output**: Good at JSON and formatted responses\n- **Fast Inference**: Quick responses with minimal resources\n\n## Usage with llama.cpp\n\n```bash\n./llama-cli -m Qwen2.5-0.5B-Instruct-Q4_K_M.gguf -p \"Your prompt here\" -n 512\n```\n\n## License\n\nThis model inherits the **Apache 2.0** license from the original Qwen2.5 model.\n\n## Attribution\n\n- **Original Model**: [Qwen2.5-0.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct) by Qwen Team, Alibaba Cloud\n- **Quantization**: jc-builds\n",
    "related_quantizations": []
  },
  "tags": [
    "gguf",
    "quantized",
    "llama.cpp",
    "ios",
    "mobile",
    "edge",
    "base_model:Qwen/Qwen2.5-0.5B-Instruct",
    "base_model:quantized:Qwen/Qwen2.5-0.5B-Instruct",
    "license:apache-2.0",
    "endpoints_compatible",
    "region:us",
    "conversational"
  ],
  "likes": 0,
  "downloads": 107,
  "gated": false,
  "private": false,
  "last_modified": "2026-01-15T05:15:14.000Z",
  "created_at": "2026-01-15T01:51:04.000Z",
  "pipeline_tag": "",
  "library_name": ""
}

{
  "_id": "696848087c8edbccd1d0f931",
  "id": "jc-builds/Qwen2.5-0.5B-Instruct-Q4_K_M-GGUF",
  "modelId": "jc-builds/Qwen2.5-0.5B-Instruct-Q4_K_M-GGUF",
  "sha": "a98b4786c1ba2e59e9ef39afda78e93438b84a91",
  "createdAt": "2026-01-15T01:51:04.000Z",
  "lastModified": "2026-01-15T05:15:14.000Z",
  "author": "jc-builds",
  "downloads": 107,
  "likes": 0,
  "gated": false,
  "private": false,
  "pipeline_tag": "",
  "library_name": "",
  "siblings_count": 3
}