worthdoing/deepseek-r1-distill-qwen-7b-gguf overview

{
  "metadata": {},
  "card_data": {
    "language": [
      "en",
      "fr",
      "multilingual"
    ],
    "license": "apache-2.0",
    "tags": [
      "gguf",
      "quantized",
      "mac",
      "apple-silicon",
      "local-inference",
      "worthdoing"
    ],
    "base_model": "deepseek-ai/DeepSeek-R1-Distill-Qwen-7B",
    "quantized_by": "worthdoing",
    "pipeline_tag": "text-generation",
    "frontmatter": {
      "language": [
        "en",
        "fr",
        "multilingual"
      ],
      "license": "apache-2.0",
      "tags": [
        "gguf",
        "quantized",
        "mac",
        "apple-silicon",
        "local-inference",
        "worthdoing"
      ],
      "base_model": "deepseek-ai/DeepSeek-R1-Distill-Qwen-7B",
      "quantized_by": "worthdoing",
      "pipeline_tag": "text-generation"
    },
    "hero_image_url": "",
    "summary": "> Quantized for local Mac inference (Apple Silicon / Metal) by **worthdoing**",
    "quick_links": [],
    "benchmark_table_html": "",
    "readme_markdown": "---\nlanguage:\n- en\n- fr\n- multilingual\nlicense: apache-2.0\ntags:\n- gguf\n- quantized\n- mac\n- apple-silicon\n- local-inference\n- worthdoing\nbase_model: deepseek-ai/DeepSeek-R1-Distill-Qwen-7B\nquantized_by: worthdoing\npipeline_tag: text-generation\n---\n\n# DeepSeek-R1-Distill-Qwen-7B - GGUF Quantized by worthdoing\n\n> Quantized for local Mac inference (Apple Silicon / Metal) by **worthdoing**\n\n## About\n\nThis is a GGUF quantized version of [DeepSeek-R1-Distill-Qwen-7B](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-7B), optimized for running locally on Apple Silicon Macs with `llama.cpp`, `Ollama`, or `LM Studio`.\n\n- **Original model:** [deepseek-ai/DeepSeek-R1-Distill-Qwen-7B](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-7B)\n- **Parameters:** 7B\n- **Quantized by:** worthdoing\n- **Pipeline:** corelm-model v1.0\n\n## Description\n\nDeepSeek R1 reasoning distilled into Qwen 7B. Chain-of-thought powerhouse.\n\n## Available Quantizations\n\n| File | Quant | BPW | Size | Use Case |\n|------|-------|-----|------|----------|\n| `deepseek-r1-distill-qwen-7b-Q4_K_M-worthdoing.gguf` | Q4_K_M | 4.58 | ~3.7 GB | **Recommended** - Best quality/size ratio |\n| `deepseek-r1-distill-qwen-7b-Q5_K_M-worthdoing.gguf` | Q5_K_M | 5.33 | ~4.3 GB | Higher quality, still fast |\n| `deepseek-r1-distill-qwen-7b-Q8_0-worthdoing.gguf` | Q8_0 | 7.96 | ~6.5 GB | Near-original quality |\n\n## How to Use\n\n### With Ollama\n```bash\n# Create a Modelfile\ncat > Modelfile <<'MODELEOF'\nFROM ./deepseek-r1-distill-qwen-7b-Q4_K_M-worthdoing.gguf\nMODELEOF\n\nollama create deepseek-r1-distill-qwen-7b -f Modelfile\nollama run deepseek-r1-distill-qwen-7b\n```\n\n### With llama.cpp\n```bash\nllama-cli -m deepseek-r1-distill-qwen-7b-Q4_K_M-worthdoing.gguf -p \"Your prompt here\" -ngl 99\n```\n\n### With LM Studio\n1. Download the GGUF file\n2. Open LM Studio -> My Models -> Import\n3. Select the GGUF file and start chatting\n\n## Recommended Hardware\n\n| Quant | Min RAM | Recommended |\n|-------|---------|-------------|\n| Q4_K_M | 4 GB | Mac with 8 GB+ RAM |\n| Q5_K_M | 5 GB | Mac with 8 GB+ RAM |\n| Q8_0 | 8 GB | Mac with 12 GB+ RAM |\n\n## Tags\n\n`reasoning`, `math`, `coding`, `chain-of-thought`\n\n---\n\n*Quantized with corelm-model pipeline by **worthdoing** on 2026-04-16*\n",
    "related_quantizations": []
  },
  "tags": [
    "gguf",
    "quantized",
    "mac",
    "apple-silicon",
    "local-inference",
    "worthdoing",
    "text-generation",
    "en",
    "fr",
    "multilingual",
    "base_model:deepseek-ai/DeepSeek-R1-Distill-Qwen-7B",
    "base_model:quantized:deepseek-ai/DeepSeek-R1-Distill-Qwen-7B",
    "license:apache-2.0",
    "endpoints_compatible",
    "region:us",
    "conversational"
  ],
  "likes": 0,
  "downloads": 342,
  "gated": false,
  "private": false,
  "last_modified": "2026-04-16T02:02:52.000Z",
  "created_at": "2026-04-16T01:58:24.000Z",
  "pipeline_tag": "text-generation",
  "library_name": ""
}

{
  "_id": "69e04240531ff090c30c8d3c",
  "id": "worthdoing/DeepSeek-R1-Distill-Qwen-7B-GGUF",
  "modelId": "worthdoing/DeepSeek-R1-Distill-Qwen-7B-GGUF",
  "sha": "b92bca17357729027418b39696d3f40061fe396a",
  "createdAt": "2026-04-16T01:58:24.000Z",
  "lastModified": "2026-04-16T02:02:52.000Z",
  "author": "worthdoing",
  "downloads": 342,
  "likes": 0,
  "gated": false,
  "private": false,
  "pipeline_tag": "text-generation",
  "library_name": "",
  "siblings_count": 5
}