unsloth/falcon-h1-1.5b-deep-instruct-gguf overview

{
  "metadata": {},
  "card_data": {
    "library_name": "transformers",
    "tags": [
      "falcon-h1",
      "unsloth"
    ],
    "license": "other",
    "license_name": "falcon-llm-license",
    "license_link": "https://falconllm.tii.ae/falcon-terms-and-conditions.html",
    "base_model": [
      "tiiuae/Falcon-H1-1.5B-Deep-Instruct"
    ],
    "inference": true,
    "frontmatter": {
      "library_name": "transformers",
      "tags": [
        "falcon-h1",
        "unsloth"
      ],
      "license": "other",
      "license_name": "falcon-llm-license",
      "license_link": "https://falconllm.tii.ae/falcon-terms-and-conditions.html",
      "base_model": [
        "tiiuae/Falcon-H1-1.5B-Deep-Instruct"
      ],
      "inference": "true"
    },
    "hero_image_url": "https://github.com/unslothai/unsloth/raw/main/images/unsloth%20new%20logo.png",
    "summary": "0. TL;DR 1. Model Details 2. Training Details 3. Usage 4. Evaluation 5. Citation # TL;DR # Model Details",
    "quick_links": [],
    "benchmark_table_html": "",
    "readme_markdown": "---\nlibrary_name: transformers\ntags:\n- falcon-h1\n- unsloth\nlicense: other\nlicense_name: falcon-llm-license\nlicense_link: https://falconllm.tii.ae/falcon-terms-and-conditions.html\nbase_model:\n- tiiuae/Falcon-H1-1.5B-Deep-Instruct\ninference: true\n---\n> [!NOTE]\n>  Includes our **chat template fixes**! <br> For `llama.cpp`, use `--jinja`\n>\n\n<div>\n<p style=\"margin-top: 0;margin-bottom: 0;\">\n    <em><a href=\"https://docs.unsloth.ai/basics/unsloth-dynamic-v2.0-gguf\">Unsloth Dynamic 2.0</a> achieves superior accuracy & outperforms other leading quants.</em>\n  </p>\n  <div style=\"display: flex; gap: 5px; align-items: center; \">\n    <a href=\"https://github.com/unslothai/unsloth/\">\n      <img src=\"https://github.com/unslothai/unsloth/raw/main/images/unsloth%20new%20logo.png\" width=\"133\">\n    </a>\n    <a href=\"https://discord.gg/unsloth\">\n      <img src=\"https://github.com/unslothai/unsloth/raw/main/images/Discord%20button.png\" width=\"173\">\n    </a>\n    <a href=\"https://docs.unsloth.ai/\">\n      <img src=\"https://raw.githubusercontent.com/unslothai/unsloth/refs/heads/main/images/documentation%20green%20button.png\" width=\"143\">\n    </a>\n  </div>\n</div>\n\n\n<img src=\"https://huggingface.co/datasets/tiiuae/documentation-images/resolve/main/falcon_mamba/falcon-h1-logo.png\" alt=\"drawing\" width=\"800\"/>\n\n#  Table of Contents\n\n0. [TL;DR](#TL;DR)\n1. [Model Details](#model-details)\n2. [Training Details](#training-details)\n3. [Usage](#usage)\n4. [Evaluation](#evaluation)\n5. [Citation](#citation)\n\n# TL;DR\n\n# Model Details\n\n## Model Description\n\n- **Developed by:** [https://www.tii.ae](https://www.tii.ae)\n- **Model type:** Causal decoder-only\n- **Architecture:** Hybrid Transformers + Mamba architecture\n- **Language(s) (NLP):** English, Multilingual\n- **License:** Falcon-LLM License\n\n# Training details\n\nFor more details about the training protocol of this model, please refer to the [Falcon-H1 technical blogpost](https://falcon-lm.github.io/blog/falcon-h1/).\n\n# Usage\n\nCurrently to use this model you can either rely on Hugging Face `transformers`, `vLLM` or `llama.cpp` library.\n\n## Inference\n\nMake sure to install the latest version of `transformers` or `vllm`, eventually install these packages from source:\n\n```bash\npip install git+https://github.com/huggingface/transformers.git\n```\n\nFor vLLM, make sure to install `vllm>=0.9.0`:\n\n```bash\npip install \"vllm>=0.9.0\"\n```\n### 🤗 transformers\n\nRefer to the snippet below to run H1 models using 🤗 transformers:\n\n```python\nimport torch\nfrom transformers import AutoModelForCausalLM, AutoTokenizer\n\nmodel_id = \"tiiuae/Falcon-H1-1B-Base\"\n\nmodel = AutoModelForCausalLM.from_pretrained(\n  model_id,\n  torch_dtype=torch.bfloat16,\n  device_map=\"auto\"\n)\n\n# Perform text generation\n```\n\n### vLLM\n\nFor vLLM, simply start a server by executing the command below:\n\n```\n# pip install vllm>=0.9.0\nvllm serve tiiuae/Falcon-H1-1B-Instruct --tensor-parallel-size 2 --data-parallel-size 1\n```\n\n### `llama.cpp`\n\nYou can find all GGUF files compatible with `llama.cpp` under [our official collection](https://huggingface.co/collections/tiiuae/falcon-h1-6819f2795bc406da60fab8df)\n\n# Evaluation\n\nFalcon-H1 series perform very well on a variety of tasks, including reasoning tasks. \n\n| Tasks | Falcon-H1-1.5B-deep | Qwen3-1.7B | Qwen2.5-1.5B | Gemma3-1B | Llama3.2-1B | Falcon3-1B |\n| --- | --- | --- | --- | --- | --- | --- |\n| **General**  | | | | | |\n| BBH | **54.43** | 35.18 | 42.41 | 35.86 | 33.21 | 34.47 |\n| ARC-C | **43.86** | 34.81 | 40.53 | 34.13 | 34.64 | 43.09 |\n| TruthfulQA | **50.48** | 49.39 | 47.05 | 42.17 | 42.08 | 42.31 |\n| HellaSwag | **65.54** | 49.27 | 62.23 | 42.24 | 55.3 | 58.53 |\n| MMLU | **66.11** | 57.04 | 59.76 | 40.87 | 45.93 | 46.1 |\n| **Math**  | | | | | |\n| GSM8k | **82.34** | 69.83 | 57.47 | 42.38 | 44.28 | 44.05 |\n| MATH-500 | **77.8** | 73.0 | 48.4 | 45.4 | 13.2 | 19.8 |\n| AMC-23 | **56.56** | 46.09 | 24.06 | 19.22 | 7.19 | 6.87 |\n| AIME-24 | **14.37** | 12.5 | 2.29 | 0.42 | 1.46 | 0.41 |\n| AIME-25 | **11.04** | 8.12 | 1.25 | 1.25 | 0.0 | 0.21 |\n| **Science**  | | | | | |\n| GPQA | **33.22** | 27.68 | 26.26 | 28.19 | 26.59 | 26.76 |\n| GPQA_Diamond | **40.57** | 33.33 | 25.59 | 21.55 | 25.08 | 31.31 |\n| MMLU-Pro | **41.89** | 23.54 | 28.35 | 14.46 | 16.2 | 18.49 |\n| MMLU-stem | **67.3** | 54.3 | 54.04 | 35.39 | 39.16 | 39.64 |\n| **Code**  | | | | | |\n| HumanEval | **73.78** | 67.68 | 56.1 | 40.85 | 34.15 | 22.56 |\n| HumanEval+ | **68.9** | 60.96 | 50.61 | 37.2 | 29.88 | 20.73 |\n| MBPP | **68.25** | 58.73 | 64.81 | 57.67 | 33.6 | 20.63 |\n| MBPP+ | **56.61** | 49.74 | 56.08 | 50.0 | 29.37 | 17.2 |\n| LiveCodeBench | **23.87** | 14.87 | 12.52 | 5.09 | 2.35 | 0.78 |\n| CRUXEval | **52.32** | 18.88 | 34.76 | 12.7 | 0.06 | 15.58 |\n| **Instruction Following**  | | | | | |\n| IFEval | **83.5** | 70.77 | 45.33 | 61.48 | 55.34 | 54.26 |\n| Alpaca-Eval | **27.12** | 21.89 | 9.54 | 17.87 | 9.38 | 6.98 |\n| MTBench | **8.53** | 7.61 | 7.1 | 7.03 | 6.37 | 6.03 |\n| LiveBench | 36.83 | **40.73** | 21.65 | 18.79 | 14.97 | 14.1 |\n\nYou can check more in detail on our [our release blogpost](https://falcon-lm.github.io/blog/falcon-h1/), detailed benchmarks.\n\n# Useful links\n\n- View [our release blogpost](https://falcon-lm.github.io/blog/falcon-h1/).\n- Feel free to join [our discord server](https://discord.gg/trwMYP9PYm) if you have any questions or to interact with our researchers and developers.\n\n# Citation\n\nIf the Falcon-H1 family of models were helpful to your work, feel free to give us a cite.\n\n```\n@misc{tiifalconh1,\n    title = {Falcon-H1: A Family of Hybrid-Head Language Models Redefining Efficiency and Performance},\n    url = {https://falcon-lm.github.io/blog/falcon-h1},\n    author = {Falcon-LLM Team},\n    month = {May},\n    year = {2025}\n}\n```",
    "related_quantizations": []
  },
  "tags": [
    "transformers",
    "gguf",
    "falcon-h1",
    "unsloth",
    "base_model:tiiuae/Falcon-H1-1.5B-Deep-Instruct",
    "base_model:quantized:tiiuae/Falcon-H1-1.5B-Deep-Instruct",
    "license:other",
    "endpoints_compatible",
    "region:us",
    "conversational"
  ],
  "likes": 3,
  "downloads": 2960,
  "gated": false,
  "private": false,
  "last_modified": "2025-07-10T08:45:36.000Z",
  "created_at": "2025-07-10T07:57:52.000Z",
  "pipeline_tag": "",
  "library_name": "transformers"
}

{
  "_id": "686f728082831ad8a5ed5dc5",
  "id": "unsloth/Falcon-H1-1.5B-Deep-Instruct-GGUF",
  "modelId": "unsloth/Falcon-H1-1.5B-Deep-Instruct-GGUF",
  "sha": "fe88d4e94f0fa3c00c6eaaf807fa2f6243dd1d7c",
  "createdAt": "2025-07-10T07:57:52.000Z",
  "lastModified": "2025-07-10T08:45:36.000Z",
  "author": "unsloth",
  "downloads": 2960,
  "likes": 3,
  "gated": false,
  "private": false,
  "pipeline_tag": "",
  "library_name": "transformers",
  "siblings_count": 29
}