richarderkhov/qwen_-_qwen2.5-math-1.5b-instruct-gguf overview

{
  "metadata": {},
  "card_data": {
    "frontmatter": {},
    "hero_image_url": "http://qianwen-res.oss-accelerate-overseas.aliyuncs.com/Qwen2.5/qwen2.5-math-pipeline.jpeg",
    "summary": "> [!Warning] >  >  > 🚨 Qwen2.5-Math mainly supports solving English and Chinese math problems through CoT and TIR. We do not recommend using this series of models for other tasks. >  >",
    "quick_links": [],
    "benchmark_table_html": "",
    "readme_markdown": "Quantization made by Richard Erkhov.\n\n[Github](https://github.com/RichardErkhov)\n\n[Discord](https://discord.gg/pvy7H8DZMG)\n\n[Request more models](https://github.com/RichardErkhov/quant_request)\n\n\nQwen2.5-Math-1.5B-Instruct - GGUF\n- Model creator: https://huggingface.co/Qwen/\n- Original model: https://huggingface.co/Qwen/Qwen2.5-Math-1.5B-Instruct/\n\n\n| Name | Quant method | Size |\n| ---- | ---- | ---- |\n| [Qwen2.5-Math-1.5B-Instruct.Q2_K.gguf](https://huggingface.co/RichardErkhov/Qwen_-_Qwen2.5-Math-1.5B-Instruct-gguf/blob/main/Qwen2.5-Math-1.5B-Instruct.Q2_K.gguf) | Q2_K | 0.63GB |\n| [Qwen2.5-Math-1.5B-Instruct.IQ3_XS.gguf](https://huggingface.co/RichardErkhov/Qwen_-_Qwen2.5-Math-1.5B-Instruct-gguf/blob/main/Qwen2.5-Math-1.5B-Instruct.IQ3_XS.gguf) | IQ3_XS | 0.68GB |\n| [Qwen2.5-Math-1.5B-Instruct.IQ3_S.gguf](https://huggingface.co/RichardErkhov/Qwen_-_Qwen2.5-Math-1.5B-Instruct-gguf/blob/main/Qwen2.5-Math-1.5B-Instruct.IQ3_S.gguf) | IQ3_S | 0.71GB |\n| [Qwen2.5-Math-1.5B-Instruct.Q3_K_S.gguf](https://huggingface.co/RichardErkhov/Qwen_-_Qwen2.5-Math-1.5B-Instruct-gguf/blob/main/Qwen2.5-Math-1.5B-Instruct.Q3_K_S.gguf) | Q3_K_S | 0.71GB |\n| [Qwen2.5-Math-1.5B-Instruct.IQ3_M.gguf](https://huggingface.co/RichardErkhov/Qwen_-_Qwen2.5-Math-1.5B-Instruct-gguf/blob/main/Qwen2.5-Math-1.5B-Instruct.IQ3_M.gguf) | IQ3_M | 0.72GB |\n| [Qwen2.5-Math-1.5B-Instruct.Q3_K.gguf](https://huggingface.co/RichardErkhov/Qwen_-_Qwen2.5-Math-1.5B-Instruct-gguf/blob/main/Qwen2.5-Math-1.5B-Instruct.Q3_K.gguf) | Q3_K | 0.77GB |\n| [Qwen2.5-Math-1.5B-Instruct.Q3_K_M.gguf](https://huggingface.co/RichardErkhov/Qwen_-_Qwen2.5-Math-1.5B-Instruct-gguf/blob/main/Qwen2.5-Math-1.5B-Instruct.Q3_K_M.gguf) | Q3_K_M | 0.77GB |\n| [Qwen2.5-Math-1.5B-Instruct.Q3_K_L.gguf](https://huggingface.co/RichardErkhov/Qwen_-_Qwen2.5-Math-1.5B-Instruct-gguf/blob/main/Qwen2.5-Math-1.5B-Instruct.Q3_K_L.gguf) | Q3_K_L | 0.82GB |\n| [Qwen2.5-Math-1.5B-Instruct.IQ4_XS.gguf](https://huggingface.co/RichardErkhov/Qwen_-_Qwen2.5-Math-1.5B-Instruct-gguf/blob/main/Qwen2.5-Math-1.5B-Instruct.IQ4_XS.gguf) | IQ4_XS | 0.84GB |\n| [Qwen2.5-Math-1.5B-Instruct.Q4_0.gguf](https://huggingface.co/RichardErkhov/Qwen_-_Qwen2.5-Math-1.5B-Instruct-gguf/blob/main/Qwen2.5-Math-1.5B-Instruct.Q4_0.gguf) | Q4_0 | 0.87GB |\n| [Qwen2.5-Math-1.5B-Instruct.IQ4_NL.gguf](https://huggingface.co/RichardErkhov/Qwen_-_Qwen2.5-Math-1.5B-Instruct-gguf/blob/main/Qwen2.5-Math-1.5B-Instruct.IQ4_NL.gguf) | IQ4_NL | 0.88GB |\n| [Qwen2.5-Math-1.5B-Instruct.Q4_K_S.gguf](https://huggingface.co/RichardErkhov/Qwen_-_Qwen2.5-Math-1.5B-Instruct-gguf/blob/main/Qwen2.5-Math-1.5B-Instruct.Q4_K_S.gguf) | Q4_K_S | 0.88GB |\n| [Qwen2.5-Math-1.5B-Instruct.Q4_K.gguf](https://huggingface.co/RichardErkhov/Qwen_-_Qwen2.5-Math-1.5B-Instruct-gguf/blob/main/Qwen2.5-Math-1.5B-Instruct.Q4_K.gguf) | Q4_K | 0.92GB |\n| [Qwen2.5-Math-1.5B-Instruct.Q4_K_M.gguf](https://huggingface.co/RichardErkhov/Qwen_-_Qwen2.5-Math-1.5B-Instruct-gguf/blob/main/Qwen2.5-Math-1.5B-Instruct.Q4_K_M.gguf) | Q4_K_M | 0.92GB |\n| [Qwen2.5-Math-1.5B-Instruct.Q4_1.gguf](https://huggingface.co/RichardErkhov/Qwen_-_Qwen2.5-Math-1.5B-Instruct-gguf/blob/main/Qwen2.5-Math-1.5B-Instruct.Q4_1.gguf) | Q4_1 | 0.95GB |\n| [Qwen2.5-Math-1.5B-Instruct.Q5_0.gguf](https://huggingface.co/RichardErkhov/Qwen_-_Qwen2.5-Math-1.5B-Instruct-gguf/blob/main/Qwen2.5-Math-1.5B-Instruct.Q5_0.gguf) | Q5_0 | 1.02GB |\n| [Qwen2.5-Math-1.5B-Instruct.Q5_K_S.gguf](https://huggingface.co/RichardErkhov/Qwen_-_Qwen2.5-Math-1.5B-Instruct-gguf/blob/main/Qwen2.5-Math-1.5B-Instruct.Q5_K_S.gguf) | Q5_K_S | 1.02GB |\n| [Qwen2.5-Math-1.5B-Instruct.Q5_K.gguf](https://huggingface.co/RichardErkhov/Qwen_-_Qwen2.5-Math-1.5B-Instruct-gguf/blob/main/Qwen2.5-Math-1.5B-Instruct.Q5_K.gguf) | Q5_K | 1.05GB |\n| [Qwen2.5-Math-1.5B-Instruct.Q5_K_M.gguf](https://huggingface.co/RichardErkhov/Qwen_-_Qwen2.5-Math-1.5B-Instruct-gguf/blob/main/Qwen2.5-Math-1.5B-Instruct.Q5_K_M.gguf) | Q5_K_M | 1.05GB |\n| [Qwen2.5-Math-1.5B-Instruct.Q5_1.gguf](https://huggingface.co/RichardErkhov/Qwen_-_Qwen2.5-Math-1.5B-Instruct-gguf/blob/main/Qwen2.5-Math-1.5B-Instruct.Q5_1.gguf) | Q5_1 | 1.1GB |\n| [Qwen2.5-Math-1.5B-Instruct.Q6_K.gguf](https://huggingface.co/RichardErkhov/Qwen_-_Qwen2.5-Math-1.5B-Instruct-gguf/blob/main/Qwen2.5-Math-1.5B-Instruct.Q6_K.gguf) | Q6_K | 1.19GB |\n| [Qwen2.5-Math-1.5B-Instruct.Q8_0.gguf](https://huggingface.co/RichardErkhov/Qwen_-_Qwen2.5-Math-1.5B-Instruct-gguf/blob/main/Qwen2.5-Math-1.5B-Instruct.Q8_0.gguf) | Q8_0 | 1.53GB |\n\n\n\n\nOriginal model description:\n---\nbase_model: Qwen/Qwen2.5-Math-1.5B\nlanguage:\n- en\npipeline_tag: text-generation\ntags:\n- chat\nlibrary_name: transformers\nlicense: apache-2.0\nlicense_link: https://huggingface.co/Qwen/Qwen2.5-Math-1.5B-Instruct/blob/main/LICENSE\n---\n\n\n# Qwen2.5-Math-1.5B-Instruct\n\n> [!Warning]\n> <div align=\"center\">\n> <b>\n> 🚨 Qwen2.5-Math mainly supports solving English and Chinese math problems through CoT and TIR. We do not recommend using this series of models for other tasks.\n> </b>\n> </div>\n\n## Introduction\n\nIn August 2024, we released the first series of mathematical LLMs - [Qwen2-Math](https://qwenlm.github.io/blog/qwen2-math/) - of our Qwen family. A month later, we have upgraded it and open-sourced **Qwen2.5-Math** series, including base models **Qwen2.5-Math-1.5B/7B/72B**, instruction-tuned models **Qwen2.5-Math-1.5B/7B/72B-Instruct**, and mathematical reward model **Qwen2.5-Math-RM-72B**. \n                                                                              \nUnlike Qwen2-Math series which only supports using Chain-of-Thught (CoT) to solve English math problems, Qwen2.5-Math series is expanded to support using both CoT and Tool-integrated Reasoning (TIR) to solve math problems in both Chinese and English. The Qwen2.5-Math series models have achieved significant performance improvements compared to the Qwen2-Math series models on the Chinese and English mathematics benchmarks with CoT. \n\n![](http://qianwen-res.oss-accelerate-overseas.aliyuncs.com/Qwen2.5/qwen2.5-math-pipeline.jpeg)\n\nWhile CoT plays a vital role in enhancing the reasoning capabilities of LLMs, it faces challenges in achieving computational accuracy and handling complex mathematical or algorithmic reasoning tasks, such as finding the roots of a quadratic equation or computing the eigenvalues of a matrix. TIR can further improve the model's proficiency in precise computation, symbolic manipulation, and algorithmic manipulation. Qwen2.5-Math-1.5B/7B/72B-Instruct achieve 79.7, 85.3, and 87.8 respectively on the MATH benchmark using TIR. \n\n## Model Details\n\n\nFor more details, please refer to our [blog post](https://qwenlm.github.io/blog/qwen2.5-math/) and [GitHub repo](https://github.com/QwenLM/Qwen2.5-Math).\n\n\n## Requirements\n* `transformers>=4.37.0` for Qwen2.5-Math models. The latest version is recommended.\n\n> [!Warning]\n> <div align=\"center\">\n> <b>\n> 🚨 This is a must because <code>transformers</code> integrated Qwen2 codes since <code>4.37.0</code>.\n> </b>\n> </div>\n\nFor requirements on GPU memory and the respective throughput, see similar results of Qwen2 [here](https://qwen.readthedocs.io/en/latest/benchmark/speed_benchmark.html).\n\n## Quick Start\n\n> [!Important]\n>\n> **Qwen2.5-Math-1.5B-Instruct** is an instruction model for chatting;\n>\n> **Qwen2.5-Math-1.5B** is a base model typically used for completion and few-shot inference, serving as a better starting point for fine-tuning.\n> \n\n### 🤗 Hugging Face Transformers\n\nQwen2.5-Math can be deployed and infered in the same way as [Qwen2.5](https://github.com/QwenLM/Qwen2.5). Here we show a code snippet to show you how to use the chat model with `transformers`:\n\n```python\nfrom transformers import AutoModelForCausalLM, AutoTokenizer\n\nmodel_name = \"Qwen/Qwen2.5-Math-1.5B-Instruct\"\ndevice = \"cuda\" # the device to load the model onto\n\nmodel = AutoModelForCausalLM.from_pretrained(\n    model_name,\n    torch_dtype=\"auto\",\n    device_map=\"auto\"\n)\ntokenizer = AutoTokenizer.from_pretrained(model_name)\n\nprompt = \"Find the value of $x$ that satisfies the equation $4x+5 = 6x+7$.\"\n\n# CoT\nmessages = [\n    {\"role\": \"system\", \"content\": \"Please reason step by step, and put your final answer within \\\\boxed{}.\"},\n    {\"role\": \"user\", \"content\": prompt}\n]\n\n# TIR\nmessages = [\n    {\"role\": \"system\", \"content\": \"Please integrate natural language reasoning with programs to solve the problem above, and put your final answer within \\\\boxed{}.\"},\n    {\"role\": \"user\", \"content\": prompt}\n]\n\ntext = tokenizer.apply_chat_template(\n    messages,\n    tokenize=False,\n    add_generation_prompt=True\n)\nmodel_inputs = tokenizer([text], return_tensors=\"pt\").to(device)\n\ngenerated_ids = model.generate(\n    **model_inputs,\n    max_new_tokens=512\n)\ngenerated_ids = [\n    output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)\n]\n\nresponse = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]\n```\n\n## Citation\n\nIf you find our work helpful, feel free to give us a citation.\n\n```\n@article{yang2024qwen25mathtechnicalreportmathematical,\n  title={Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvement}, \n  author={An Yang and Beichen Zhang and Binyuan Hui and Bofei Gao and Bowen Yu and Chengpeng Li and Dayiheng Liu and Jianhong Tu and Jingren Zhou and Junyang Lin and Keming Lu and Mingfeng Xue and Runji Lin and Tianyu Liu and Xingzhang Ren and Zhenru Zhang},\n  journal={arXiv preprint arXiv:2409.12122},\n  year={2024}\n}\n```\n\n",
    "related_quantizations": []
  },
  "tags": [
    "gguf",
    "arxiv:2409.12122",
    "endpoints_compatible",
    "region:us",
    "conversational"
  ],
  "likes": 0,
  "downloads": 165,
  "gated": false,
  "private": false,
  "last_modified": "2024-09-28T01:57:10.000Z",
  "created_at": "2024-09-27T22:38:27.000Z",
  "pipeline_tag": "",
  "library_name": ""
}

{
  "_id": "66f733e3754d5facc9a1ebfe",
  "id": "RichardErkhov/Qwen_-_Qwen2.5-Math-1.5B-Instruct-gguf",
  "modelId": "RichardErkhov/Qwen_-_Qwen2.5-Math-1.5B-Instruct-gguf",
  "sha": "44b7196f84e5494430a9e94017c2bceb73f54be1",
  "createdAt": "2024-09-27T22:38:27.000Z",
  "lastModified": "2024-09-28T01:57:10.000Z",
  "author": "RichardErkhov",
  "downloads": 165,
  "likes": 0,
  "gated": false,
  "private": false,
  "pipeline_tag": "",
  "library_name": "",
  "siblings_count": 24
}