glogwa68/granite-4.0-h-350m-distill-gemini-think-gguf overview

{
  "metadata": {},
  "card_data": {
    "base_model": "glogwa68/granite-4.0-h-350m-DISTILL-gemini-think",
    "library_name": "gguf",
    "license": "apache-2.0",
    "language": [
      "en",
      "fr"
    ],
    "tags": [
      "granite",
      "gguf",
      "quantized",
      "llama.cpp",
      "ollama"
    ],
    "quantized_by": "llama.cpp",
    "pipeline_tag": "text-generation",
    "frontmatter": {
      "base_model": "glogwa68/granite-4.0-h-350m-DISTILL-gemini-think",
      "library_name": "gguf",
      "license": "apache-2.0",
      "language": [
        "en",
        "fr"
      ],
      "tags": [
        "granite",
        "gguf",
        "quantized",
        "llama.cpp",
        "ollama"
      ],
      "quantized_by": "llama.cpp",
      "pipeline_tag": "text-generation"
    },
    "hero_image_url": "",
    "summary": "GGUF quantized versions of granite-4.0-h-350m-DISTILL-gemini-think",
    "quick_links": [],
    "benchmark_table_html": "",
    "readme_markdown": "---\nbase_model: glogwa68/granite-4.0-h-350m-DISTILL-gemini-think\nlibrary_name: gguf\nlicense: apache-2.0\nlanguage:\n- en\n- fr\ntags:\n- granite\n- gguf\n- quantized\n- llama.cpp\n- ollama\nquantized_by: llama.cpp\npipeline_tag: text-generation\n---\n\n# granite-4.0-h-350m-DISTILL-gemini-think-GGUF\n\nGGUF quantized versions of [granite-4.0-h-350m-DISTILL-gemini-think](https://huggingface.co/glogwa68/granite-4.0-h-350m-DISTILL-gemini-think)\n\n## Available Formats\n\n| Filename | Size | Quant Type | Description |\n|----------|------|------------|-------------|\n| [granite-4.0-h-350m-DISTILL-gemini-3-pro-think-f16.gguf](https://huggingface.co/glogwa68/granite-4.0-h-350m-DISTILL-gemini-think-GGUF/blob/main/granite-4.0-h-350m-DISTILL-gemini-3-pro-think-f16.gguf) | 0.64 GB | GRANITE-4.0-H-350M-DISTILL-GEMINI-3-PRO-THINK-F16 |  |\n| [granite-4.0-h-350m-DISTILL-gemini-3-pro-think-q2_k.gguf](https://huggingface.co/glogwa68/granite-4.0-h-350m-DISTILL-gemini-think-GGUF/blob/main/granite-4.0-h-350m-DISTILL-gemini-3-pro-think-q2_k.gguf) | 0.15 GB | GRANITE-4.0-H-350M-DISTILL-GEMINI-3-PRO-THINK-Q2_K |  |\n| [granite-4.0-h-350m-DISTILL-gemini-3-pro-think-q3_k_l.gguf](https://huggingface.co/glogwa68/granite-4.0-h-350m-DISTILL-gemini-think-GGUF/blob/main/granite-4.0-h-350m-DISTILL-gemini-3-pro-think-q3_k_l.gguf) | 0.18 GB | GRANITE-4.0-H-350M-DISTILL-GEMINI-3-PRO-THINK-Q3_K_L |  |\n| [granite-4.0-h-350m-DISTILL-gemini-3-pro-think-q3_k_m.gguf](https://huggingface.co/glogwa68/granite-4.0-h-350m-DISTILL-gemini-think-GGUF/blob/main/granite-4.0-h-350m-DISTILL-gemini-3-pro-think-q3_k_m.gguf) | 0.18 GB | GRANITE-4.0-H-350M-DISTILL-GEMINI-3-PRO-THINK-Q3_K_M |  |\n| [granite-4.0-h-350m-DISTILL-gemini-3-pro-think-q3_k_s.gguf](https://huggingface.co/glogwa68/granite-4.0-h-350m-DISTILL-gemini-think-GGUF/blob/main/granite-4.0-h-350m-DISTILL-gemini-3-pro-think-q3_k_s.gguf) | 0.17 GB | GRANITE-4.0-H-350M-DISTILL-GEMINI-3-PRO-THINK-Q3_K_S |  |\n| [granite-4.0-h-350m-DISTILL-gemini-3-pro-think-q4_0.gguf](https://huggingface.co/glogwa68/granite-4.0-h-350m-DISTILL-gemini-think-GGUF/blob/main/granite-4.0-h-350m-DISTILL-gemini-3-pro-think-q4_0.gguf) | 0.20 GB | GRANITE-4.0-H-350M-DISTILL-GEMINI-3-PRO-THINK-Q4_0 |  |\n| [granite-4.0-h-350m-DISTILL-gemini-3-pro-think-q4_1.gguf](https://huggingface.co/glogwa68/granite-4.0-h-350m-DISTILL-gemini-think-GGUF/blob/main/granite-4.0-h-350m-DISTILL-gemini-3-pro-think-q4_1.gguf) | 0.22 GB | GRANITE-4.0-H-350M-DISTILL-GEMINI-3-PRO-THINK-Q4_1 |  |\n| [granite-4.0-h-350m-DISTILL-gemini-3-pro-think-q4_k_m.gguf](https://huggingface.co/glogwa68/granite-4.0-h-350m-DISTILL-gemini-think-GGUF/blob/main/granite-4.0-h-350m-DISTILL-gemini-3-pro-think-q4_k_m.gguf) | 0.21 GB | GRANITE-4.0-H-350M-DISTILL-GEMINI-3-PRO-THINK-Q4_K_M |  |\n| [granite-4.0-h-350m-DISTILL-gemini-3-pro-think-q4_k_s.gguf](https://huggingface.co/glogwa68/granite-4.0-h-350m-DISTILL-gemini-think-GGUF/blob/main/granite-4.0-h-350m-DISTILL-gemini-3-pro-think-q4_k_s.gguf) | 0.20 GB | GRANITE-4.0-H-350M-DISTILL-GEMINI-3-PRO-THINK-Q4_K_S |  |\n| [granite-4.0-h-350m-DISTILL-gemini-3-pro-think-q5_0.gguf](https://huggingface.co/glogwa68/granite-4.0-h-350m-DISTILL-gemini-think-GGUF/blob/main/granite-4.0-h-350m-DISTILL-gemini-3-pro-think-q5_0.gguf) | 0.23 GB | GRANITE-4.0-H-350M-DISTILL-GEMINI-3-PRO-THINK-Q5_0 |  |\n| [granite-4.0-h-350m-DISTILL-gemini-3-pro-think-q5_1.gguf](https://huggingface.co/glogwa68/granite-4.0-h-350m-DISTILL-gemini-think-GGUF/blob/main/granite-4.0-h-350m-DISTILL-gemini-3-pro-think-q5_1.gguf) | 0.25 GB | GRANITE-4.0-H-350M-DISTILL-GEMINI-3-PRO-THINK-Q5_1 |  |\n| [granite-4.0-h-350m-DISTILL-gemini-3-pro-think-q5_k_m.gguf](https://huggingface.co/glogwa68/granite-4.0-h-350m-DISTILL-gemini-think-GGUF/blob/main/granite-4.0-h-350m-DISTILL-gemini-3-pro-think-q5_k_m.gguf) | 0.24 GB | GRANITE-4.0-H-350M-DISTILL-GEMINI-3-PRO-THINK-Q5_K_M |  |\n| [granite-4.0-h-350m-DISTILL-gemini-3-pro-think-q5_k_s.gguf](https://huggingface.co/glogwa68/granite-4.0-h-350m-DISTILL-gemini-think-GGUF/blob/main/granite-4.0-h-350m-DISTILL-gemini-3-pro-think-q5_k_s.gguf) | 0.23 GB | GRANITE-4.0-H-350M-DISTILL-GEMINI-3-PRO-THINK-Q5_K_S |  |\n| [granite-4.0-h-350m-DISTILL-gemini-3-pro-think-q6_k.gguf](https://huggingface.co/glogwa68/granite-4.0-h-350m-DISTILL-gemini-think-GGUF/blob/main/granite-4.0-h-350m-DISTILL-gemini-3-pro-think-q6_k.gguf) | 0.26 GB | GRANITE-4.0-H-350M-DISTILL-GEMINI-3-PRO-THINK-Q6_K |  |\n| [granite-4.0-h-350m-DISTILL-gemini-3-pro-think-q8_0.gguf](https://huggingface.co/glogwa68/granite-4.0-h-350m-DISTILL-gemini-think-GGUF/blob/main/granite-4.0-h-350m-DISTILL-gemini-3-pro-think-q8_0.gguf) | 0.34 GB | GRANITE-4.0-H-350M-DISTILL-GEMINI-3-PRO-THINK-Q8_0 |  |\n\n\n## Quick Start\n\n### Ollama\n\n```bash\n# Use Q4_K_M (recommended)\nollama run hf.co/glogwa68/granite-4.0-h-350m-DISTILL-gemini-think-GGUF:Q4_K_M\n\n# Or other quantizations\nollama run hf.co/glogwa68/granite-4.0-h-350m-DISTILL-gemini-think-GGUF:Q8_0\nollama run hf.co/glogwa68/granite-4.0-h-350m-DISTILL-gemini-think-GGUF:Q2_K\n```\n\n### llama.cpp\n\n```bash\n# Download and run\nllama-cli --hf-repo glogwa68/granite-4.0-h-350m-DISTILL-gemini-think-GGUF --hf-file granite-4.0-h-350m-distill-gemini-think-q4_k_m.gguf -p \"Hello, how are you?\"\n\n# With server\nllama-server --hf-repo glogwa68/granite-4.0-h-350m-DISTILL-gemini-think-GGUF --hf-file granite-4.0-h-350m-distill-gemini-think-q4_k_m.gguf -c 2048\n```\n\n### LM Studio / GPT4All\n\nDownload the `.gguf` file of your choice and load it in your application.\n\n## Quantization Details\n\n| Type | Bits | Use Case |\n|------|------|----------|\n| Q2_K | 2 | Extreme compression, low quality |\n| Q3_K_M | 3 | Very compressed |\n| Q4_K_M | 4 | **Recommended** - Best size/quality |\n| Q5_K_M | 5 | High quality |\n| Q6_K | 6 | Very high quality |\n| Q8_0 | 8 | Near lossless |\n| F16 | 16 | Original precision |\n\n## Original Model\n\nThis is the quantized version of [granite-4.0-h-350m-DISTILL-gemini-think](https://huggingface.co/glogwa68/granite-4.0-h-350m-DISTILL-gemini-think)\n\n- **Base Model:** ibm-granite/granite-4.0-h-350m\n- **Fine-tuning Dataset:** TeichAI/gemini-3-pro-preview-high-reasoning-1000x\n- **Special Feature:** Thinking/Reasoning with `<think>` tags\n",
    "related_quantizations": []
  },
  "tags": [
    "gguf",
    "granite",
    "quantized",
    "llama.cpp",
    "ollama",
    "text-generation",
    "en",
    "fr",
    "base_model:glogwa68/granite-4.0-h-350m-DISTILL-gemini-think",
    "base_model:quantized:glogwa68/granite-4.0-h-350m-DISTILL-gemini-think",
    "license:apache-2.0",
    "endpoints_compatible",
    "region:us",
    "conversational"
  ],
  "likes": 0,
  "downloads": 221,
  "gated": false,
  "private": false,
  "last_modified": "2025-12-23T14:13:50.000Z",
  "created_at": "2025-12-23T14:10:45.000Z",
  "pipeline_tag": "text-generation",
  "library_name": "gguf"
}

{
  "_id": "694aa2e5d52ec8785676c575",
  "id": "glogwa68/granite-4.0-h-350m-DISTILL-gemini-think-GGUF",
  "modelId": "glogwa68/granite-4.0-h-350m-DISTILL-gemini-think-GGUF",
  "sha": "c5df054df589fd53d8668207661561bd250df5ea",
  "createdAt": "2025-12-23T14:10:45.000Z",
  "lastModified": "2025-12-23T14:13:50.000Z",
  "author": "glogwa68",
  "downloads": 221,
  "likes": 0,
  "gated": false,
  "private": false,
  "pipeline_tag": "text-generation",
  "library_name": "gguf",
  "siblings_count": 17
}