richarderkhov/deepseek-ai_-_deepseek-math-7b-rl-gguf overview

{
  "metadata": {},
  "card_data": {
    "frontmatter": {},
    "hero_image_url": "https://github.com/deepseek-ai/DeepSeek-LLM/blob/main/images/logo.png?raw=true",
    "summary": "Quantization made by Richard Erkhov. Github Discord Request more models deepseek-math-7b-rl - GGUF | Name | Quant method | Size | | ---- | ---- | ---- | | deepseek-math-7b-rl.Q2_K.gguf | Q2_K | 2.53GB | | deepseek-math-7b-rl.Q3_K_S.gguf | Q3_K_S | 2.92GB | | deepseek-math-7b-rl.Q3_K.gguf | Q3_K | 3.22GB | | deepseek-math-7b-rl.Q3_K_M.gguf | Q3_K_M | 3.22GB | | deepseek-math-7b-rl.Q3_K_L.gguf | Q3_K_L | 3.49GB | | deepseek-math-7b-rl.IQ4_XS.gguf | IQ4_XS | 3.56GB | | deepseek-math-7b-rl.Q4_0.gguf | Q4_0 | 3.73GB | | deepseek-math-7b-rl.IQ4_NL.gguf | IQ4_NL | 3.74GB | | deepseek-math-7b-rl.Q4_K_S.gguf | Q4_K_S | 3.75GB | | deepseek-math-7b-rl.Q4_K.gguf | Q4_K | 3.93GB | | deepseek-math-7b-rl.Q4_K_M.gguf | Q4_K_M | 3.93GB | | deepseek-math-7b-rl.Q4_1.gguf | Q4_1 | 4.1GB | | deepseek-math-7b-rl.Q5_0.gguf | Q5_0 | 4.48GB | | deepseek-math-7b-rl.Q5_K_S.gguf | Q5_K_S | 4.48GB | | deepseek-math-7b-rl.Q5_K.gguf | Q5_K | 4.59GB | | deepseek-math-7b-rl.Q5_K_M.gguf | Q5_K_M | 4.59GB | | deepseek-math-7b-rl.Q5_1.gguf | Q5_1 | 4.86GB | | deepseek-math-7b-rl.Q6_K.gguf | Q6_K | 5.28GB | | deepseek-math-7b-rl.Q8_0.gguf | Q8_0 | 6.84GB | Original model description: --- license: other license_name: deepseek license_link: https://github.com/deepseek-ai/DeepSeek-Math/blob/main/LICENSE-MODEL ---    [🏠Homepage]  |  [🤖 Chat with DeepSeek LLM]  |  [Discord]  |  [Wechat(微信)]   Paper Link👁️   ### 1. Introduction to DeepSeekMath See the Introduction for more details. ### 2. How to Use Here give some examples of how to use our model. **Chat Completion** ❗❗❗ **Please use chain-of-thought prompt to test DeepSeekMath-Instruct and DeepSeekMath-RL:** ``python import torch from transformers import AutoTokenizer, AutoModelForCausalLM, GenerationConfig model_name = \"deepseek-ai/deepseek-math-7b-instruct\" tokenizer = AutoTokenizer.from_pretrained(model_name) model = AutoModelForCausalLM.from_pretrained(model_name, torch_dtype=torch.bfloat16, device_map=\"auto\") model.generation_config = GenerationConfig.from_pretrained(model_name) model.generation_config.pad_token_id = model.generation_config.eos_token_id messages = [ {\"role\": \"user\", \"content\": \"what is the integral of x^2 from 0 to 2?\\nPlease reason step by step, and put your final answer within \\\\boxed{}.\"} ] input_tensor = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors=\"pt\") outputs = model.generate(input_tensor.to(model.device), max_new_tokens=100) result = tokenizer.decode(outputs[0][input_tensor.shape[1]:], skip_special_tokens=True) print(result) ` Avoiding the use of the provided function apply_chat_template, you can also interact with our model following the sample template. Note that messages should be replaced by your input. ` User: {messages[0]['content']} Assistant: {messages[1]['content']}User: {messages[2]['content']} Assistant: ` **Note:** By default (add_special_tokens=True), our tokenizer automatically adds a bos_token (`) before the input text. Additionally, since the system prompt is not compatible with this version of our models, we DO NOT RECOMMEND including the system prompt in your input. ### 3. License This code repository is licensed under the MIT License. The use of DeepSeekMath models is subject to the Model License. DeepSeekMath supports commercial use. See the LICENSE-MODEL for more details. ### 4. Contact If you have any questions, please raise an issue or contact us at service@deepseek.com.",
    "quick_links": [],
    "benchmark_table_html": "",
    "readme_markdown": "Quantization made by Richard Erkhov.\n\n[Github](https://github.com/RichardErkhov)\n\n[Discord](https://discord.gg/pvy7H8DZMG)\n\n[Request more models](https://github.com/RichardErkhov/quant_request)\n\n\ndeepseek-math-7b-rl - GGUF\n- Model creator: https://huggingface.co/deepseek-ai/\n- Original model: https://huggingface.co/deepseek-ai/deepseek-math-7b-rl/\n\n\n| Name | Quant method | Size |\n| ---- | ---- | ---- |\n| [deepseek-math-7b-rl.Q2_K.gguf](https://huggingface.co/RichardErkhov/deepseek-ai_-_deepseek-math-7b-rl-gguf/blob/main/deepseek-math-7b-rl.Q2_K.gguf) | Q2_K | 2.53GB |\n| [deepseek-math-7b-rl.Q3_K_S.gguf](https://huggingface.co/RichardErkhov/deepseek-ai_-_deepseek-math-7b-rl-gguf/blob/main/deepseek-math-7b-rl.Q3_K_S.gguf) | Q3_K_S | 2.92GB |\n| [deepseek-math-7b-rl.Q3_K.gguf](https://huggingface.co/RichardErkhov/deepseek-ai_-_deepseek-math-7b-rl-gguf/blob/main/deepseek-math-7b-rl.Q3_K.gguf) | Q3_K | 3.22GB |\n| [deepseek-math-7b-rl.Q3_K_M.gguf](https://huggingface.co/RichardErkhov/deepseek-ai_-_deepseek-math-7b-rl-gguf/blob/main/deepseek-math-7b-rl.Q3_K_M.gguf) | Q3_K_M | 3.22GB |\n| [deepseek-math-7b-rl.Q3_K_L.gguf](https://huggingface.co/RichardErkhov/deepseek-ai_-_deepseek-math-7b-rl-gguf/blob/main/deepseek-math-7b-rl.Q3_K_L.gguf) | Q3_K_L | 3.49GB |\n| [deepseek-math-7b-rl.IQ4_XS.gguf](https://huggingface.co/RichardErkhov/deepseek-ai_-_deepseek-math-7b-rl-gguf/blob/main/deepseek-math-7b-rl.IQ4_XS.gguf) | IQ4_XS | 3.56GB |\n| [deepseek-math-7b-rl.Q4_0.gguf](https://huggingface.co/RichardErkhov/deepseek-ai_-_deepseek-math-7b-rl-gguf/blob/main/deepseek-math-7b-rl.Q4_0.gguf) | Q4_0 | 3.73GB |\n| [deepseek-math-7b-rl.IQ4_NL.gguf](https://huggingface.co/RichardErkhov/deepseek-ai_-_deepseek-math-7b-rl-gguf/blob/main/deepseek-math-7b-rl.IQ4_NL.gguf) | IQ4_NL | 3.74GB |\n| [deepseek-math-7b-rl.Q4_K_S.gguf](https://huggingface.co/RichardErkhov/deepseek-ai_-_deepseek-math-7b-rl-gguf/blob/main/deepseek-math-7b-rl.Q4_K_S.gguf) | Q4_K_S | 3.75GB |\n| [deepseek-math-7b-rl.Q4_K.gguf](https://huggingface.co/RichardErkhov/deepseek-ai_-_deepseek-math-7b-rl-gguf/blob/main/deepseek-math-7b-rl.Q4_K.gguf) | Q4_K | 3.93GB |\n| [deepseek-math-7b-rl.Q4_K_M.gguf](https://huggingface.co/RichardErkhov/deepseek-ai_-_deepseek-math-7b-rl-gguf/blob/main/deepseek-math-7b-rl.Q4_K_M.gguf) | Q4_K_M | 3.93GB |\n| [deepseek-math-7b-rl.Q4_1.gguf](https://huggingface.co/RichardErkhov/deepseek-ai_-_deepseek-math-7b-rl-gguf/blob/main/deepseek-math-7b-rl.Q4_1.gguf) | Q4_1 | 4.1GB |\n| [deepseek-math-7b-rl.Q5_0.gguf](https://huggingface.co/RichardErkhov/deepseek-ai_-_deepseek-math-7b-rl-gguf/blob/main/deepseek-math-7b-rl.Q5_0.gguf) | Q5_0 | 4.48GB |\n| [deepseek-math-7b-rl.Q5_K_S.gguf](https://huggingface.co/RichardErkhov/deepseek-ai_-_deepseek-math-7b-rl-gguf/blob/main/deepseek-math-7b-rl.Q5_K_S.gguf) | Q5_K_S | 4.48GB |\n| [deepseek-math-7b-rl.Q5_K.gguf](https://huggingface.co/RichardErkhov/deepseek-ai_-_deepseek-math-7b-rl-gguf/blob/main/deepseek-math-7b-rl.Q5_K.gguf) | Q5_K | 4.59GB |\n| [deepseek-math-7b-rl.Q5_K_M.gguf](https://huggingface.co/RichardErkhov/deepseek-ai_-_deepseek-math-7b-rl-gguf/blob/main/deepseek-math-7b-rl.Q5_K_M.gguf) | Q5_K_M | 4.59GB |\n| [deepseek-math-7b-rl.Q5_1.gguf](https://huggingface.co/RichardErkhov/deepseek-ai_-_deepseek-math-7b-rl-gguf/blob/main/deepseek-math-7b-rl.Q5_1.gguf) | Q5_1 | 4.86GB |\n| [deepseek-math-7b-rl.Q6_K.gguf](https://huggingface.co/RichardErkhov/deepseek-ai_-_deepseek-math-7b-rl-gguf/blob/main/deepseek-math-7b-rl.Q6_K.gguf) | Q6_K | 5.28GB |\n| [deepseek-math-7b-rl.Q8_0.gguf](https://huggingface.co/RichardErkhov/deepseek-ai_-_deepseek-math-7b-rl-gguf/blob/main/deepseek-math-7b-rl.Q8_0.gguf) | Q8_0 | 6.84GB |\n\n\n\n\nOriginal model description:\n---\nlicense: other\nlicense_name: deepseek\nlicense_link: https://github.com/deepseek-ai/DeepSeek-Math/blob/main/LICENSE-MODEL\n---\n\n\n<p align=\"center\">\n<img width=\"500px\" alt=\"DeepSeek Chat\" src=\"https://github.com/deepseek-ai/DeepSeek-LLM/blob/main/images/logo.png?raw=true\">\n</p>\n<p align=\"center\"><a href=\"https://www.deepseek.com/\">[🏠Homepage]</a>  |  <a href=\"https://chat.deepseek.com/\">[🤖 Chat with DeepSeek LLM]</a>  |  <a href=\"https://discord.gg/Tc7c45Zzu5\">[Discord]</a>  |  <a href=\"https://github.com/deepseek-ai/DeepSeek-LLM/blob/main/images/qr.jpeg\">[Wechat(微信)]</a> </p>\n\n<p align=\"center\">\n  <a href=\"https://arxiv.org/pdf/2402.03300.pdf\"><b>Paper Link</b>👁️</a>\n</p>\n\n<hr>\n\n\n\n\n\n### 1. Introduction to DeepSeekMath\nSee the [Introduction](https://github.com/deepseek-ai/DeepSeek-Math) for more details.\n\n### 2. How to Use\nHere give some examples of how to use our model.\n\n**Chat Completion**\n\n❗❗❗ **Please use chain-of-thought prompt to test DeepSeekMath-Instruct and DeepSeekMath-RL:**\n\n- English questions: **{question}\\nPlease reason step by step, and put your final answer within \\\\boxed{}.**\n\n- Chinese questions: **{question}\\n请通过逐步推理来解答问题，并把最终答案放置于\\\\boxed{}中。**\n\n```python\nimport torch\nfrom transformers import AutoTokenizer, AutoModelForCausalLM, GenerationConfig\n\nmodel_name = \"deepseek-ai/deepseek-math-7b-instruct\"\ntokenizer = AutoTokenizer.from_pretrained(model_name)\nmodel = AutoModelForCausalLM.from_pretrained(model_name, torch_dtype=torch.bfloat16, device_map=\"auto\")\nmodel.generation_config = GenerationConfig.from_pretrained(model_name)\nmodel.generation_config.pad_token_id = model.generation_config.eos_token_id\n\nmessages = [\n    {\"role\": \"user\", \"content\": \"what is the integral of x^2 from 0 to 2?\\nPlease reason step by step, and put your final answer within \\\\boxed{}.\"}\n]\ninput_tensor = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors=\"pt\")\noutputs = model.generate(input_tensor.to(model.device), max_new_tokens=100)\n\nresult = tokenizer.decode(outputs[0][input_tensor.shape[1]:], skip_special_tokens=True)\nprint(result)\n```\n\nAvoiding the use of the provided function `apply_chat_template`, you can also interact with our model following the sample template. Note that `messages` should be replaced by your input.\n\n```\nUser: {messages[0]['content']}\n\nAssistant: {messages[1]['content']}<｜end▁of▁sentence｜>User: {messages[2]['content']}\n\nAssistant:\n```\n\n**Note:** By default (`add_special_tokens=True`), our tokenizer automatically adds a `bos_token` (`<｜begin▁of▁sentence｜>`) before the input text. Additionally, since the system prompt is not compatible with this version of our models, we DO NOT RECOMMEND including the system prompt in your input.\n\n### 3. License\nThis code repository is licensed under the MIT License. The use of DeepSeekMath models is subject to the Model License. DeepSeekMath supports commercial use.\n\nSee the [LICENSE-MODEL](https://github.com/deepseek-ai/DeepSeek-Math/blob/main/LICENSE-MODEL) for more details.\n\n### 4. Contact\n\nIf you have any questions, please raise an issue or contact us at [service@deepseek.com](mailto:service@deepseek.com).\n\n\n\n",
    "related_quantizations": []
  },
  "tags": [
    "gguf",
    "arxiv:2402.03300",
    "endpoints_compatible",
    "region:us",
    "conversational"
  ],
  "likes": 1,
  "downloads": 1307,
  "gated": false,
  "private": false,
  "last_modified": "2024-11-04T06:43:59.000Z",
  "created_at": "2024-11-04T04:41:49.000Z",
  "pipeline_tag": "",
  "library_name": ""
}

{
  "_id": "6728508deb65b7b2980c7af1",
  "id": "RichardErkhov/deepseek-ai_-_deepseek-math-7b-rl-gguf",
  "modelId": "RichardErkhov/deepseek-ai_-_deepseek-math-7b-rl-gguf",
  "sha": "61a715ccbf3978e19fb377ebe5144abadb989cf6",
  "createdAt": "2024-11-04T04:41:49.000Z",
  "lastModified": "2024-11-04T06:43:59.000Z",
  "author": "RichardErkhov",
  "downloads": 1307,
  "likes": 1,
  "gated": false,
  "private": false,
  "pipeline_tag": "",
  "library_name": "",
  "siblings_count": 21
}