unsloth/lfm2-2.6b-exp-gguf overview

{
  "metadata": {},
  "card_data": {
    "base_model": [
      "LiquidAI/LFM2-2.6B-Exp"
    ],
    "library_name": "transformers",
    "license": "other",
    "license_name": "lfm1.0",
    "license_link": "LICENSE",
    "language": [
      "en",
      "ar",
      "zh",
      "fr",
      "de",
      "ja",
      "ko",
      "es"
    ],
    "pipeline_tag": "text-generation",
    "tags": [
      "liquid",
      "unsloth",
      "lfm2",
      "edge"
    ],
    "frontmatter": {
      "base_model": [
        "LiquidAI/LFM2-2.6B-Exp"
      ],
      "library_name": "transformers",
      "license": "other",
      "license_name": "lfm1.0",
      "license_link": "LICENSE",
      "language": [
        "en",
        "ar",
        "zh",
        "fr",
        "de",
        "ja",
        "ko",
        "es"
      ],
      "pipeline_tag": "text-generation",
      "tags": [
        "liquid",
        "unsloth",
        "lfm2",
        "edge"
      ]
    },
    "hero_image_url": "https://github.com/unslothai/unsloth/raw/main/images/unsloth%20new%20logo.png",
    "summary": "LFM2-2.6B-Exp is an experimental checkpoint built on LFM2-2.6B using pure reinforcement learning. Specifically trained on instruction following, knowledge, and math, it delivers particularly strong performance compared to other 3B models. In particular, its IFBench score surpasses DeepSeek R1-0528, a model 263 times larger. !LFM2.6B-Exp-White_v1 Find more information about LFM2 in our blog post.",
    "quick_links": [],
    "benchmark_table_html": "",
    "readme_markdown": "---\nbase_model:\n- LiquidAI/LFM2-2.6B-Exp\nlibrary_name: transformers\nlicense: other\nlicense_name: lfm1.0\nlicense_link: LICENSE\nlanguage:\n- en\n- ar\n- zh\n- fr\n- de\n- ja\n- ko\n- es\npipeline_tag: text-generation\ntags:\n- liquid\n- unsloth\n- lfm2\n- edge\n---\n> [!NOTE]\n>  Includes Unsloth **chat template fixes**! <br> For `llama.cpp`, use `--jinja`\n>\n\n<div>\n<p style=\"margin-top: 0;margin-bottom: 0;\">\n    <em><a href=\"https://docs.unsloth.ai/basics/unsloth-dynamic-v2.0-gguf\">Unsloth Dynamic 2.0</a> achieves superior accuracy & outperforms other leading quants.</em>\n  </p>\n  <div style=\"display: flex; gap: 5px; align-items: center; \">\n    <a href=\"https://github.com/unslothai/unsloth/\">\n      <img src=\"https://github.com/unslothai/unsloth/raw/main/images/unsloth%20new%20logo.png\" width=\"133\">\n    </a>\n    <a href=\"https://discord.gg/unsloth\">\n      <img src=\"https://github.com/unslothai/unsloth/raw/main/images/Discord%20button.png\" width=\"173\">\n    </a>\n    <a href=\"https://docs.unsloth.ai/\">\n      <img src=\"https://raw.githubusercontent.com/unslothai/unsloth/refs/heads/main/images/documentation%20green%20button.png\" width=\"143\">\n    </a>\n  </div>\n</div>\n\n\n<center>\n<div style=\"text-align: center;\">\n  <img \n    src=\"https://cdn-uploads.huggingface.co/production/uploads/61b8e2ba285851687028d395/2b08LKpev0DNEk6DlnWkY.png\" \n    alt=\"Liquid AI\"\n    style=\"width: 100%; max-width: 100%; height: auto; display: inline-block; margin-bottom: 0.5em; margin-top: 0.5em;\"\n  />\n</div>\n<div style=\"display: flex; justify-content: center; gap: 0.5em;\">\n  <a href=\"https://playground.liquid.ai/chat\">\n<a href=\"https://playground.liquid.ai/\"><strong>Try LFM</strong></a> • <a href=\"https://docs.liquid.ai/lfm\"><strong>Documentation</strong></a> • <a href=\"https://leap.liquid.ai/\"><strong>LEAP</strong></a></a>\n</div>\n</center>\n\n# LFM2-2.6B-Exp\n\nLFM2-2.6B-Exp is an experimental checkpoint built on [LFM2-2.6B](https://huggingface.co/LiquidAI/LFM2-2.6B) using pure reinforcement learning.\n\nSpecifically trained on instruction following, knowledge, and math, it delivers particularly strong performance compared to other 3B models. \nIn particular, its IFBench score surpasses DeepSeek R1-0528, a model 263 times larger.\n\n![LFM2.6B-Exp-White_v1](https://cdn-uploads.huggingface.co/production/uploads/61b8e2ba285851687028d395/5q5J1GZ-gAfguC1qafr8S.png)\n\nFind more information about LFM2 in our [blog post](https://www.liquid.ai/blog/liquid-foundation-models-v2-our-second-series-of-generative-ai-models).\n\n## 📄 Model details\n\nDue to their small size, **we recommend fine-tuning LFM2 models on narrow use cases** to maximize performance. \nThey are particularly suited for agentic tasks, data extraction, RAG, creative writing, and multi-turn conversations. \nHowever, we do not recommend using them for tasks that are knowledge-intensive or require programming skills.\n\n| Property            | [**LFM2-350M**](https://huggingface.co/LiquidAI/LFM2-350M) | [**LFM2-700M**](https://huggingface.co/LiquidAI/LFM2-700M) | [**LFM2-1.2B**](https://huggingface.co/LiquidAI/LFM2-1.2B) | [**LFM2-2.6B**](https://huggingface.co/LiquidAI/LFM2-2.6B) |\n| ------------------- | ----------------------------- | ----------------------------- | ----------------------------- | ----------------------------- |\n| **Parameters**      | 354,483,968                   | 742,489,344                   | 1,170,340,608                 | 2,569,272,320                 |\n| **Layers**          | 16 (10 conv + 6 attn)         | 16 (10 conv + 6 attn)         | 16 (10 conv + 6 attn)         | 30 (22 conv + 8 attn)         |\n| **Context length**  | 32,768 tokens                 | 32,768 tokens                 | 32,768 tokens                 | 32,768 tokens                 |\n| **Vocabulary size** | 65,536                        | 65,536                        | 65,536                        | 65,536                        |\n| **Precision**       | bfloat16                      | bfloat16                      | bfloat16                      | bfloat16                      |\n| **Training budget** | 10 trillion tokens            | 10 trillion tokens            | 10 trillion tokens            | 10 trillion tokens            |\n| **License**         | LFM Open License v1.0         | LFM Open License v1.0         | LFM Open License v1.0         | LFM Open License v1.0         \n\n**Supported languages**: English, Arabic, Chinese, French, German, Japanese, Korean, and Spanish.\n\n**Generation parameters**: We recommend the following parameters:\n* `temperature=0.3`\n* `min_p=0.15`\n* `repetition_penalty=1.05`\n\n**Chat template**: LFM2 uses a ChatML-like chat template as follows:\n\n```\n<|startoftext|><|im_start|>system\nYou are a helpful assistant trained by Liquid AI.<|im_end|>\n<|im_start|>user\nWhat is C. elegans?<|im_end|>\n<|im_start|>assistant\nIt's a tiny nematode that lives in temperate soil environments.<|im_end|>\n```\n\nYou can automatically apply it using the dedicated [`.apply_chat_template()`](https://huggingface.co/docs/transformers/en/chat_templating#applychattemplate) function from Hugging Face transformers.\n\n**Tool use**: It consists of four main steps:\n1. **Function definition**: LFM2 takes JSON function definitions as input (JSON objects between `<|tool_list_start|>` and `<|tool_list_end|>` special tokens), usually in the system prompt\n2. **Function call**: LFM2 writes Pythonic function calls (a Python list between `<|tool_call_start|>` and `<|tool_call_end|>` special tokens), as the assistant answer.\n3. **Function execution**: The function call is executed and the result is returned (string between `<|tool_response_start|>` and `<|tool_response_end|>` special tokens), as a \"tool\" role.\n4. **Final answer**: LFM2 interprets the outcome of the function call to address the original user prompt in plain text.\n\nHere is a simple example of a conversation using tool use:\n\n```\n<|startoftext|><|im_start|>system\nList of tools: <|tool_list_start|>[{\"name\": \"get_candidate_status\", \"description\": \"Retrieves the current status of a candidate in the recruitment process\", \"parameters\": {\"type\": \"object\", \"properties\": {\"candidate_id\": {\"type\": \"string\", \"description\": \"Unique identifier for the candidate\"}}, \"required\": [\"candidate_id\"]}}]<|tool_list_end|><|im_end|>\n<|im_start|>user\nWhat is the current status of candidate ID 12345?<|im_end|>\n<|im_start|>assistant\n<|tool_call_start|>[get_candidate_status(candidate_id=\"12345\")]<|tool_call_end|>Checking the current status of candidate ID 12345.<|im_end|>\n<|im_start|>tool\n<|tool_response_start|>[{\"candidate_id\": \"12345\", \"status\": \"Interview Scheduled\", \"position\": \"Clinical Research Associate\", \"date\": \"2023-11-20\"}]<|tool_response_end|><|im_end|>\n<|im_start|>assistant\nThe candidate with ID 12345 is currently in the \"Interview Scheduled\" stage for the position of Clinical Research Associate, with an interview date set for 2023-11-20.<|im_end|>\n```\n\nYou can directly pass tools as JSON schema or Python functions with `.apply_chat_template()` as shown in [this page](https://huggingface.co/docs/transformers/en/chat_extras) to automatically format the system prompt.\n\n**Architecture**: Hybrid model with multiplicative gates and short convolutions: 10 double-gated short-range LIV convolution blocks and 6 grouped query attention (GQA) blocks.\n\n**Pre-training mixture**: Approximately 75% English, 20% multilingual, and 5% code data sourced from the web and licensed materials.\n\n**Training approach**:\n* Very large-scale SFT on 50% downstream tasks, 50% general domains\n* Custom DPO with length normalization and semi-online datasets\n* Iterative model merging\n* Reinforcement learning with verifiable rewards\n\n## 🏃 How to run LFM2\n\n### 1. Transformers\n\nTo run LFM2, you need to install Hugging Face [`transformers`](https://github.com/huggingface/transformers) v4.55 or a more recent version as follows:\n\n```bash\npip install -U transformers\n```\n\nHere is an example of how to generate an answer with transformers in Python:\n\n```python\nfrom transformers import AutoModelForCausalLM, AutoTokenizer\n\n# Load model and tokenizer\nmodel_id = \"LiquidAI/LFM2-2.6B-Exp\"\nmodel = AutoModelForCausalLM.from_pretrained(\n    model_id,\n    device_map=\"auto\",\n    torch_dtype=\"bfloat16\",\n#    attn_implementation=\"flash_attention_2\" <- uncomment on compatible GPU\n)\ntokenizer = AutoTokenizer.from_pretrained(model_id)\n\n# Generate answer\nprompt = \"What is C. elegans?\"\ninput_ids = tokenizer.apply_chat_template(\n    [{\"role\": \"user\", \"content\": prompt}],\n    add_generation_prompt=True,\n    return_tensors=\"pt\",\n    tokenize=True,\n).to(model.device)\n\noutput = model.generate(\n    input_ids,\n    do_sample=True,\n    temperature=0.3,\n    min_p=0.15,\n    repetition_penalty=1.05,\n    max_new_tokens=512,\n)\n\nprint(tokenizer.decode(output[0], skip_special_tokens=False))\n\n# <|startoftext|><|im_start|>user\n# What is C. elegans?<|im_end|>\n# <|im_start|>assistant\n# C. elegans, also known as Caenorhabditis elegans, is a small, free-living\n# nematode worm (roundworm) that belongs to the phylum Nematoda.\n```\n\nYou can directly run and test the model with this [Colab notebook](https://colab.research.google.com/drive/1_q3jQ6LtyiuPzFZv7Vw8xSfPU5FwkKZY?usp=sharing).\n\n### 2. vLLM\n\nYou need to install [`vLLM`](https://github.com/vllm-project/vllm) v0.10.2 or a more recent version as follows:\n\n```bash\nuv pip install vllm==0.10.2 --extra-index-url https://wheels.vllm.ai/0.10.2/ --torch-backend=auto\n```\n\nHere is an example of how to use it for inference:\n\n```python\nfrom vllm import LLM, SamplingParams\n\nprompts = [\n    \"What is C. elegans?\",\n    \"Say hi in JSON format\",\n    \"Define AI in Spanish\"\n]\nsampling_params = SamplingParams(temperature=0.3, min_p=0.15, repetition_penalty=1.05)\n\nllm = LLM(model=\"LiquidAI/LFM2-2.6B-Exp\")\n\noutputs = llm.generate(prompts, sampling_params)\n\nfor output in outputs:\n    prompt = output.prompt\n    generated_text = output.outputs[0].text\n    print(f\"Prompt: {prompt!r}, Generated text: {generated_text!r}\")\n```\n\n### 3. llama.cpp\n\nYou can run LFM2 with llama.cpp using its [GGUF checkpoint](https://huggingface.co/LiquidAI/LFM2-2.6B-Exp-GGUF). Find more information in the model card.\n\n## 🔧 How to fine-tune LFM2\n\nWe recommend fine-tuning LFM2 models on your use cases to maximize performance.\n\n| Notebook | Description | Link |\n|-------|------|------|\n| SFT (Unsloth) | Supervised Fine-Tuning (SFT) notebook with a LoRA adapter using Unsloth. | <a href=\"https://colab.research.google.com/drive/1HROdGaPFt1tATniBcos11-doVaH7kOI3?usp=sharing\"><img src=\"https://cdn-uploads.huggingface.co/production/uploads/61b8e2ba285851687028d395/vlOyMEjwHa_b_LXysEu2E.png\" width=\"110\" alt=\"Colab link\"></a> |\n| SFT (TRL) | Supervised Fine-Tuning (SFT) notebook with a LoRA adapter using TRL. | <a href=\"https://colab.research.google.com/drive/1j5Hk_SyBb2soUsuhU0eIEA9GwLNRnElF?usp=sharing\"><img src=\"https://cdn-uploads.huggingface.co/production/uploads/61b8e2ba285851687028d395/vlOyMEjwHa_b_LXysEu2E.png\" width=\"110\" alt=\"Colab link\"></a> |\n| DPO (TRL) | Preference alignment with Direct Preference Optimization (DPO) using TRL. | <a href=\"https://colab.research.google.com/drive/1MQdsPxFHeZweGsNx4RH7Ia8lG8PiGE1t?usp=sharing\"><img src=\"https://cdn-uploads.huggingface.co/production/uploads/61b8e2ba285851687028d395/vlOyMEjwHa_b_LXysEu2E.png\" width=\"110\" alt=\"Colab link\"></a> |\n\n## 📬 Contact\n\nIf you are interested in custom solutions with edge deployment, please contact [our sales team](https://www.liquid.ai/contact).\n\n## Citation\n\n```\n@article{liquidai2025lfm2,\n title={LFM2 Technical Report},\n author={Liquid AI},\n journal={arXiv preprint arXiv:2511.23404},\n year={2025}\n}\n```",
    "related_quantizations": []
  },
  "tags": [
    "transformers",
    "gguf",
    "liquid",
    "unsloth",
    "lfm2",
    "edge",
    "text-generation",
    "en",
    "ar",
    "zh",
    "fr",
    "de",
    "ja",
    "ko",
    "es",
    "arxiv:2511.23404",
    "base_model:LiquidAI/LFM2-2.6B-Exp",
    "base_model:quantized:LiquidAI/LFM2-2.6B-Exp",
    "license:other",
    "endpoints_compatible",
    "region:us",
    "conversational"
  ],
  "likes": 15,
  "downloads": 328,
  "gated": false,
  "private": false,
  "last_modified": "2025-12-26T12:44:02.000Z",
  "created_at": "2025-12-26T05:35:31.000Z",
  "pipeline_tag": "text-generation",
  "library_name": "transformers"
}

{
  "_id": "694e1ea3ff338b0ee67d56bf",
  "id": "unsloth/LFM2-2.6B-Exp-GGUF",
  "modelId": "unsloth/LFM2-2.6B-Exp-GGUF",
  "sha": "1b9e7ac14eb17bdf1146af206d22e9a151f63346",
  "createdAt": "2025-12-26T05:35:31.000Z",
  "lastModified": "2025-12-26T12:44:02.000Z",
  "author": "unsloth",
  "downloads": 328,
  "likes": 15,
  "gated": false,
  "private": false,
  "pipeline_tag": "text-generation",
  "library_name": "transformers",
  "siblings_count": 21
}