richarderkhov/itpossible_-_chinese-mistral-7b-instruct-v0.1-gguf overview

{
  "metadata": {},
  "card_data": {
    "frontmatter": {},
    "hero_image_url": "",
    "summary": "Quantization made by Richard Erkhov. Github Discord Request more models Chinese-Mistral-7B-Instruct-v0.1 - GGUF | Name | Quant method | Size | | ---- | ---- | ---- | | Chinese-Mistral-7B-Instruct-v0.1.Q2_K.gguf | Q2_K | 2.67GB | | Chinese-Mistral-7B-Instruct-v0.1.IQ3_XS.gguf | IQ3_XS | 2.96GB | | Chinese-Mistral-7B-Instruct-v0.1.IQ3_S.gguf | IQ3_S | 3.12GB | | Chinese-Mistral-7B-Instruct-v0.1.Q3_K_S.gguf | Q3_K_S | 3.1GB | | Chinese-Mistral-7B-Instruct-v0.1.IQ3_M.gguf | IQ3_M | 3.21GB | | Chinese-Mistral-7B-Instruct-v0.1.Q3_K.gguf | Q3_K | 3.43GB | | Chinese-Mistral-7B-Instruct-v0.1.Q3_K_M.gguf | Q3_K_M | 3.43GB | | Chinese-Mistral-7B-Instruct-v0.1.Q3_K_L.gguf | Q3_K_L | 3.71GB | | Chinese-Mistral-7B-Instruct-v0.1.IQ4_XS.gguf | IQ4_XS | 3.84GB | | Chinese-Mistral-7B-Instruct-v0.1.Q4_0.gguf | Q4_0 | 4.0GB | | Chinese-Mistral-7B-Instruct-v0.1.IQ4_NL.gguf | IQ4_NL | 4.04GB | | Chinese-Mistral-7B-Instruct-v0.1.Q4_K_S.gguf | Q4_K_S | 4.02GB | | Chinese-Mistral-7B-Instruct-v0.1.Q4_K.gguf | Q4_K | 4.24GB | | Chinese-Mistral-7B-Instruct-v0.1.Q4_K_M.gguf | Q4_K_M | 4.24GB | | Chinese-Mistral-7B-Instruct-v0.1.Q4_1.gguf | Q4_1 | 4.42GB | | Chinese-Mistral-7B-Instruct-v0.1.Q5_0.gguf | Q5_0 | 4.84GB | | Chinese-Mistral-7B-Instruct-v0.1.Q5_K_S.gguf | Q5_K_S | 4.84GB | | Chinese-Mistral-7B-Instruct-v0.1.Q5_K.gguf | Q5_K | 4.96GB | | Chinese-Mistral-7B-Instruct-v0.1.Q5_K_M.gguf | Q5_K_M | 4.96GB | | Chinese-Mistral-7B-Instruct-v0.1.Q5_1.gguf | Q5_1 | 5.26GB | | Chinese-Mistral-7B-Instruct-v0.1.Q6_K.gguf | Q6_K | 5.73GB | | Chinese-Mistral-7B-Instruct-v0.1.Q8_0.gguf | Q8_0 | 7.43GB | Original model description:   Chinese-Mistral",
    "quick_links": [],
    "benchmark_table_html": "",
    "readme_markdown": "Quantization made by Richard Erkhov.\n\n[Github](https://github.com/RichardErkhov)\n\n[Discord](https://discord.gg/pvy7H8DZMG)\n\n[Request more models](https://github.com/RichardErkhov/quant_request)\n\n\nChinese-Mistral-7B-Instruct-v0.1 - GGUF\n- Model creator: https://huggingface.co/itpossible/\n- Original model: https://huggingface.co/itpossible/Chinese-Mistral-7B-Instruct-v0.1/\n\n\n| Name | Quant method | Size |\n| ---- | ---- | ---- |\n| [Chinese-Mistral-7B-Instruct-v0.1.Q2_K.gguf](https://huggingface.co/RichardErkhov/itpossible_-_Chinese-Mistral-7B-Instruct-v0.1-gguf/blob/main/Chinese-Mistral-7B-Instruct-v0.1.Q2_K.gguf) | Q2_K | 2.67GB |\n| [Chinese-Mistral-7B-Instruct-v0.1.IQ3_XS.gguf](https://huggingface.co/RichardErkhov/itpossible_-_Chinese-Mistral-7B-Instruct-v0.1-gguf/blob/main/Chinese-Mistral-7B-Instruct-v0.1.IQ3_XS.gguf) | IQ3_XS | 2.96GB |\n| [Chinese-Mistral-7B-Instruct-v0.1.IQ3_S.gguf](https://huggingface.co/RichardErkhov/itpossible_-_Chinese-Mistral-7B-Instruct-v0.1-gguf/blob/main/Chinese-Mistral-7B-Instruct-v0.1.IQ3_S.gguf) | IQ3_S | 3.12GB |\n| [Chinese-Mistral-7B-Instruct-v0.1.Q3_K_S.gguf](https://huggingface.co/RichardErkhov/itpossible_-_Chinese-Mistral-7B-Instruct-v0.1-gguf/blob/main/Chinese-Mistral-7B-Instruct-v0.1.Q3_K_S.gguf) | Q3_K_S | 3.1GB |\n| [Chinese-Mistral-7B-Instruct-v0.1.IQ3_M.gguf](https://huggingface.co/RichardErkhov/itpossible_-_Chinese-Mistral-7B-Instruct-v0.1-gguf/blob/main/Chinese-Mistral-7B-Instruct-v0.1.IQ3_M.gguf) | IQ3_M | 3.21GB |\n| [Chinese-Mistral-7B-Instruct-v0.1.Q3_K.gguf](https://huggingface.co/RichardErkhov/itpossible_-_Chinese-Mistral-7B-Instruct-v0.1-gguf/blob/main/Chinese-Mistral-7B-Instruct-v0.1.Q3_K.gguf) | Q3_K | 3.43GB |\n| [Chinese-Mistral-7B-Instruct-v0.1.Q3_K_M.gguf](https://huggingface.co/RichardErkhov/itpossible_-_Chinese-Mistral-7B-Instruct-v0.1-gguf/blob/main/Chinese-Mistral-7B-Instruct-v0.1.Q3_K_M.gguf) | Q3_K_M | 3.43GB |\n| [Chinese-Mistral-7B-Instruct-v0.1.Q3_K_L.gguf](https://huggingface.co/RichardErkhov/itpossible_-_Chinese-Mistral-7B-Instruct-v0.1-gguf/blob/main/Chinese-Mistral-7B-Instruct-v0.1.Q3_K_L.gguf) | Q3_K_L | 3.71GB |\n| [Chinese-Mistral-7B-Instruct-v0.1.IQ4_XS.gguf](https://huggingface.co/RichardErkhov/itpossible_-_Chinese-Mistral-7B-Instruct-v0.1-gguf/blob/main/Chinese-Mistral-7B-Instruct-v0.1.IQ4_XS.gguf) | IQ4_XS | 3.84GB |\n| [Chinese-Mistral-7B-Instruct-v0.1.Q4_0.gguf](https://huggingface.co/RichardErkhov/itpossible_-_Chinese-Mistral-7B-Instruct-v0.1-gguf/blob/main/Chinese-Mistral-7B-Instruct-v0.1.Q4_0.gguf) | Q4_0 | 4.0GB |\n| [Chinese-Mistral-7B-Instruct-v0.1.IQ4_NL.gguf](https://huggingface.co/RichardErkhov/itpossible_-_Chinese-Mistral-7B-Instruct-v0.1-gguf/blob/main/Chinese-Mistral-7B-Instruct-v0.1.IQ4_NL.gguf) | IQ4_NL | 4.04GB |\n| [Chinese-Mistral-7B-Instruct-v0.1.Q4_K_S.gguf](https://huggingface.co/RichardErkhov/itpossible_-_Chinese-Mistral-7B-Instruct-v0.1-gguf/blob/main/Chinese-Mistral-7B-Instruct-v0.1.Q4_K_S.gguf) | Q4_K_S | 4.02GB |\n| [Chinese-Mistral-7B-Instruct-v0.1.Q4_K.gguf](https://huggingface.co/RichardErkhov/itpossible_-_Chinese-Mistral-7B-Instruct-v0.1-gguf/blob/main/Chinese-Mistral-7B-Instruct-v0.1.Q4_K.gguf) | Q4_K | 4.24GB |\n| [Chinese-Mistral-7B-Instruct-v0.1.Q4_K_M.gguf](https://huggingface.co/RichardErkhov/itpossible_-_Chinese-Mistral-7B-Instruct-v0.1-gguf/blob/main/Chinese-Mistral-7B-Instruct-v0.1.Q4_K_M.gguf) | Q4_K_M | 4.24GB |\n| [Chinese-Mistral-7B-Instruct-v0.1.Q4_1.gguf](https://huggingface.co/RichardErkhov/itpossible_-_Chinese-Mistral-7B-Instruct-v0.1-gguf/blob/main/Chinese-Mistral-7B-Instruct-v0.1.Q4_1.gguf) | Q4_1 | 4.42GB |\n| [Chinese-Mistral-7B-Instruct-v0.1.Q5_0.gguf](https://huggingface.co/RichardErkhov/itpossible_-_Chinese-Mistral-7B-Instruct-v0.1-gguf/blob/main/Chinese-Mistral-7B-Instruct-v0.1.Q5_0.gguf) | Q5_0 | 4.84GB |\n| [Chinese-Mistral-7B-Instruct-v0.1.Q5_K_S.gguf](https://huggingface.co/RichardErkhov/itpossible_-_Chinese-Mistral-7B-Instruct-v0.1-gguf/blob/main/Chinese-Mistral-7B-Instruct-v0.1.Q5_K_S.gguf) | Q5_K_S | 4.84GB |\n| [Chinese-Mistral-7B-Instruct-v0.1.Q5_K.gguf](https://huggingface.co/RichardErkhov/itpossible_-_Chinese-Mistral-7B-Instruct-v0.1-gguf/blob/main/Chinese-Mistral-7B-Instruct-v0.1.Q5_K.gguf) | Q5_K | 4.96GB |\n| [Chinese-Mistral-7B-Instruct-v0.1.Q5_K_M.gguf](https://huggingface.co/RichardErkhov/itpossible_-_Chinese-Mistral-7B-Instruct-v0.1-gguf/blob/main/Chinese-Mistral-7B-Instruct-v0.1.Q5_K_M.gguf) | Q5_K_M | 4.96GB |\n| [Chinese-Mistral-7B-Instruct-v0.1.Q5_1.gguf](https://huggingface.co/RichardErkhov/itpossible_-_Chinese-Mistral-7B-Instruct-v0.1-gguf/blob/main/Chinese-Mistral-7B-Instruct-v0.1.Q5_1.gguf) | Q5_1 | 5.26GB |\n| [Chinese-Mistral-7B-Instruct-v0.1.Q6_K.gguf](https://huggingface.co/RichardErkhov/itpossible_-_Chinese-Mistral-7B-Instruct-v0.1-gguf/blob/main/Chinese-Mistral-7B-Instruct-v0.1.Q6_K.gguf) | Q6_K | 5.73GB |\n| [Chinese-Mistral-7B-Instruct-v0.1.Q8_0.gguf](https://huggingface.co/RichardErkhov/itpossible_-_Chinese-Mistral-7B-Instruct-v0.1-gguf/blob/main/Chinese-Mistral-7B-Instruct-v0.1.Q8_0.gguf) | Q8_0 | 7.43GB |\n\n\n\n\nOriginal model description:\n<div align=\"center\">\n    <h1>\n        Chinese-Mistral\n    </h1>\n</div>\n\n## 🎉 新闻\n- [2024-08-31] 发布[Chinese-Mistral-7B-Instruct-v0.2](https://huggingface.co/itpossible/Chinese-Mistral-7B-Instruct-v0.2)。\n- [2024-04-04] 发布[Chinese-Mistral-7B-Instruct-v0.1](https://huggingface.co/itpossible/Chinese-Mistral-7B-Instruct-v0.1)。\n- [2024-03-31] 发布[Chinese-Mistral-7B-v0.1](https://huggingface.co/itpossible/Chinese-Mistral-7B)基座模型。\n\n## 🚀 介绍\n\n随着Mistral AI公司开源其七十亿参数模型[Mistral-7B](https://huggingface.co/meta-llama/Llama-2-7b-hf)，该模型超越[Llama](https://huggingface.co/meta-llama)，成为当前最强大的开源模型之一。Mistral-7B在各类基准测试中，不仅超过了Llama2-13B，而且在推理、数学、代码生成任务中超过Llama2-34B。\n然而，Mistral-7B的训练语料主要为英文文本，其中文能力较为欠缺。其次，Mistral-7B的词表不支持中文，导致其对中文的编码和解码效率较低，限制了在中文场景的应用。<br>\n为了克服这一局限，清华大学地球系统科学系地球和空间信息科学实验室基于Mistral-7B进行了中文词表扩充和增量预训练，增强了Mistral-7B在中文任务上的表现，并提高了其对中文文本的编解码效率。<br>\n项目地址：https://github.com/THU-ESIS/Chinese-Mistral\n\n## 📥 模型下载\n\n本项目开源了Chinese-Mistral-7B与Chinese-Mistral-7B-instruct：\n\n|             模型             |                                     下载地址                                      |                                                         说明                                                          |\n|:-----------------------------:|:------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|\n|     Chinese-Mistral-7B     |     [HuggingFace](https://huggingface.co/itpossible/Chinese-Mistral-7B-v0.1)<br>[wisemodel](https://wisemodel.cn/models/itpossible/Chinese-Mistral-7B-v0.1)<br>[ModelScope](https://www.modelscope.cn/models/itpossible/Chinese-Mistral-7B-v0.1)     |                                                  完整基座模型                                                  |\n| Chinese-Mistral-7B-Instruct-v0.1 | [HuggingFace](https://huggingface.co/itpossible/Chinese-Mistral-7B-Instruct-v0.1)<br>[wisemodel](https://wisemodel.cn/models/itpossible/Chinese-Mistral-7B-Instruct-v0.1)<br>[ModelScope](https://www.modelscope.cn/models/itpossible/Chinese-Mistral-7B-Instruct-v0.1) | 完整指令精调模型<br>中英文alpaca_gpt4进行lora微调|\n| Chinese-Mistral-7B-Instruct-v0.2 | [HuggingFace](https://huggingface.co/itpossible/Chinese-Mistral-7B-Instruct-v0.2)<br>[wisemodel](https://wisemodel.cn/models/itpossible/Chinese-Mistral-7B-Instruct-v0.2)<br> | 完整指令精调模型<br>百万条高质量数据进行lora微调|\n\n\n## 📈 模型性能\n\n### 模型综合能力\n\n我们采用C-Eval、CMMLU和MMLU三个评测数据集全面评估Chinese-Mistral-7B：\n\n- C-Eval：它是一个全面的中文基础模型评估套件。包含13948个多项选择题，涵盖52个学科和四个难度级别。它旨在评估模型在人文、社科、理工等多个学科大类上的知识和推理能力。\n- CMMLU：它是一个综合性的中文评估基准。涵盖了从基础学科到高级专业水平的67个主题。它专门用于评估语言模型在中文语境下的知识和推理能力。\n- MMLU：它是一个包含了57个子任务的英文评测数据集。涵盖了从初等数学、美国历史、计算机科学到法律等多个领域，难度覆盖高中水平到专家水平，有效地衡量了模型在人文、社科和理工等多个学科大类中的综合知识能力。\n\n下表展示了开源社区较流行的中文Llama2、中文Mistral与我们发布的Chinese-Mistral-7B的评测结果。评测方式采用5-shot，采用opencompass在相同的实验条件下进行评测。\n\n|                                              模型名称                                              |    C-Eval     |      CMMLU    |    MMLU      |    平均得分        |\n|:-----------------------------------------------------------------------------------------------:|:-------------:|:-------------:|:------------:|:-----------------:|\n|    [Linly-Al/Chinese-LLaMA-2-7B-hf](https://huggingface.co/Linly-Al/Chinese-LLaMA-2-7B-hf)      |     31.2      |     30.14     |    35.09     |       32.14       |\n|             [hfl/chinese-llama-2-7b](https://huggingface.co/hfl/chinese-llama-2-7b)             |     27.4      |     33.38     |    37.25     |       32.68       |\n|    [Linly-Al/Chinese-LLaMA-2-13B-hf](https://huggingface.co/Linly-Al/Chinese-LLaMA-2-13B-hf)    |     39.9      |     42.48     |    52.54     |       44.97       |\n|            [hfl/chinese-llama-2-13b](https://huggingface.co/hfl/chinese-llama-2-13b)            |     41.0      |     43.25     |    52.94     |       45.73       |\n|      [gywy/Mistral-7B-v0.1-chinese](https://huggingface.co/gywy/Mistral-7B-v0.1-chinese)        |     37.4      |     36.45     |    37.38     |       37.08       |\n|[OpenBuddy/openbuddy-mistral-7b-v13-base](https://huggingface.co/OpenBuddy/openbuddy-mistral-7b-v13-base)|     44.4      |     46.32     |    57.79     |       49.50       |\n|                                  **[Chinese-Mistral-7B (本模型)](https://huggingface.co/itpossible/Chinese-Mistral-7B-v0.1)**                                  |     **47.5**      |     **47.52**     |    **58.29**     |       **51.10**       |\n\n由上表可知，Chinese-Mistral-7B的中文和英文通识能力不仅超过同等参数量的中文Llama2模型，而且在多项评测中优于130亿参数量的中文Llama2。同时，Chinese-Mistral-7B的评测表现高于开源社区其他同等参数量的中文Mistral。\n\n### 中文编解码效率\n\n我们从WuDaoCorpus2中采样训练数据，使用sentencepiece训练中文BPE词表，并人工选取部分其他优秀中文词表进行词表融合。经过严格的人工审核，最终形成的词表大小为63776。为了提高模型计算效率，我们在词表末尾添加<|sym1|>、……、<|sym96|>，使得词表大小为128的倍数，最终得到的词表大小为63872。\n我们随机选取了WuDaoCorpus2_part-2021278643作为测试数据以评测分词效果。经统计，测试数据包括67013857个单词，我们用单词数量除以分词后的Token数量，计算压缩率。压缩率越大，表明分词效果越好，在中文场景的编解码效率越高。\n\n|                                              模型名称                                              |    模型类型     |      词表大小    |    Token数量      |    压缩率        |\n|:-----------------------------------------------------------------------------------------------:|:-------------:|:-------------:|:------------:|:-----------------:|\n|    [meta-llama/Llama-2-7b-hf](https://huggingface.co/meta-llama/Llama-2-7b-hf)      |     Llama      |     32000     |    97406876     |       0.6880       |\n|             [mistralai/Mistral-7B-v0.1](https://huggingface.co/mistralai/Mistral-7B-v0.1)             |     Mistral      |     32000     |    76269008     |       0.8787       |\n|             [THUDM/chatglm2-6b](https://huggingface.co/THUDM/chatglm2-6b)             |     GLM      |     64789     |    43487673     |       1.5410       |\n|    [Linly-Al/Chinese-LLaMA-2-13B-hf](https://huggingface.co/Linly-Al/Chinese-LLaMA-2-13B-hf)    |     Llama      |     40076     |    65402900     |       1.0246       |\n|            [hfl/chinese-llama-2-13b](https://huggingface.co/hfl/chinese-llama-2-13b)            |     Llama      |     55296     |    45763513     |       1.4644       |\n|      [OpenBuddy/openbuddy-mistral-7b-v13-base](https://huggingface.co/OpenBuddy/openbuddy-mistral-7b-v13-base)        |     Mistral      |     36608     |    65329642     |       1.0256       |\n|[gywy/Mistral-7B-v0.1-chinese](https://huggingface.co/gywy/Mistral-7B-v0.1-chinese)|     Mistral      |     48593     |    46670146     |       1.4359       |\n|                                  **[Chinese-Mistral-7B (本模型)](https://huggingface.co/itpossible/Chinese-Mistral-7B-v0.1)**                                   |     Mistral      |     63872     |    **43044156**     |       **1.5569**       |\n\n\n\n由上表可知，Chinese-Mistral-7B在可观的词表大小条件下，取得了最高的压缩率，表明其能够高效处理中文文本。\n\n## 💻 模型推理\n\n如下是使用Chinese-Mistral-7B进行推理的代码示例。\n\n```python\nimport torch\nfrom transformers import AutoTokenizer, AutoModelForCausalLM\n\ndevice = torch.device(\"cuda:0\") if torch.cuda.is_available() else torch.device(\"cpu\")\n\nmodel_path = \"itpossible/Chinese-Mistral-7B-v0.1\"\ntokenizer = AutoTokenizer.from_pretrained(model_path)\nmodel = AutoModelForCausalLM.from_pretrained(model_path, torch_dtype=torch.bfloat16, device_map=device)\n\ntext = \"我是一个人工智能助手，我能够帮助你做如下这些事情：\"\ninputs = tokenizer(text, return_tensors=\"pt\").to(device)\n\noutputs = model.generate(**inputs, max_new_tokens=120, do_sample=True)\nprint(tokenizer.decode(outputs[0], skip_special_tokens=True))\n```\n\n如下是使用Chinese-Mistral-7B-Instruct进行推理的代码示例。\n```python\nimport torch\nfrom transformers import AutoTokenizer, AutoModelForCausalLM\n\ndevice = torch.device(\"cuda:0\") if torch.cuda.is_available() else torch.device(\"cpu\")\n\nmodel_path = \"itpossible/Chinese-Mistral-7B-Instruct-v0.1\"\ntokenizer = AutoTokenizer.from_pretrained(model_path)\nmodel = AutoModelForCausalLM.from_pretrained(model_path, torch_dtype=torch.bfloat16, device_map=device)\n\ntext = \"请为我推荐中国三座比较著名的山\"\nmessages = [{\"role\": \"user\", \"content\": text}]\n\ninputs = tokenizer.apply_chat_template(messages, return_tensors=\"pt\").to(device)\noutputs = model.generate(inputs, max_new_tokens=300, do_sample=True)\noutputs = tokenizer.batch_decode(outputs, skip_special_tokens=True)[0]\nprint(outputs)\n```\n\n## 📝 训练数据\n\n训练数据采样于WanJuan、baike2018qa、Dolma、gutenberg-books等高质量开源数据集。我们对这些数据集进行细粒度清洗，并充分考虑训练数据集中不同类别数据的占比。\n\n## ⚠️ 局限性\n\nChinese-Mistral-7B的开发旨在为开源社区提供一个性能优越的中文大语言模型。请注意，由于模型大小及训练数据规模限制，本模型仍可能生成误导性内容或者有害内容。因此，在部署任何由Chinese-Mistral系列模型驱动的应用程序之前，开发人员必须进行安全测试，对模型进行相应调整，以满足安全性需求。\n\n## ✒️ 引用\n\n如果您觉得本项目对您的研究有所帮助或使用了本项目的模型，请引用本项目：\n\n```bibtex\n@misc{Chinese-Mistral,\n    author = {Zhou, Chen and Yuqi, Bai},\n    title = {Chinese-Mistral: An Efficient and Effective Chinese Large Language Model},\n    year = {2024},\n    publisher = {GitHub},\n    journal = {GitHub repository},\n    howpublished = {\\url{https://github.com/THU-ESIS/Chinese-Mistral}}\n}\n```\n\n## 结语\n我们欢迎社区的支持和合作，共同推动通用大语言模型和领域大语言模型的发展。联系方式：<br>\n白玉琪，清华大学地球系统科学系长聘教授，实验室负责人，yuqibai@tsinghua.edu.cn<br>\n陈舟，清华大学地球系统科学系博士生，大语言模型组组长，chenz22@mails.tsinghua.edu.cn\n\n",
    "related_quantizations": []
  },
  "tags": [
    "gguf",
    "endpoints_compatible",
    "region:us"
  ],
  "likes": 0,
  "downloads": 324,
  "gated": false,
  "private": false,
  "last_modified": "2024-09-17T12:47:12.000Z",
  "created_at": "2024-09-17T06:20:08.000Z",
  "pipeline_tag": "",
  "library_name": ""
}

{
  "_id": "66e91f987aa83a594b007b2d",
  "id": "RichardErkhov/itpossible_-_Chinese-Mistral-7B-Instruct-v0.1-gguf",
  "modelId": "RichardErkhov/itpossible_-_Chinese-Mistral-7B-Instruct-v0.1-gguf",
  "sha": "204df05a6ae336063f5b16d00032dfbb64282f37",
  "createdAt": "2024-09-17T06:20:08.000Z",
  "lastModified": "2024-09-17T12:47:12.000Z",
  "author": "RichardErkhov",
  "downloads": 324,
  "likes": 0,
  "gated": false,
  "private": false,
  "pipeline_tag": "",
  "library_name": "",
  "siblings_count": 24
}