richarderkhov/winninghealth_-_wingpt2-llama-3-8b-base-gguf overview

{
  "metadata": {},
  "card_data": {
    "frontmatter": {},
    "hero_image_url": "",
    "summary": "Quantization made by Richard Erkhov. Github Discord Request more models WiNGPT2-Llama-3-8B-Base - GGUF | Name | Quant method | Size | | ---- | ---- | ---- | | WiNGPT2-Llama-3-8B-Base.Q2_K.gguf | Q2_K | 2.96GB | | WiNGPT2-Llama-3-8B-Base.IQ3_XS.gguf | IQ3_XS | 3.28GB | | WiNGPT2-Llama-3-8B-Base.IQ3_S.gguf | IQ3_S | 3.43GB | | WiNGPT2-Llama-3-8B-Base.Q3_K_S.gguf | Q3_K_S | 3.41GB | | WiNGPT2-Llama-3-8B-Base.IQ3_M.gguf | IQ3_M | 3.52GB | | WiNGPT2-Llama-3-8B-Base.Q3_K.gguf | Q3_K | 3.74GB | | WiNGPT2-Llama-3-8B-Base.Q3_K_M.gguf | Q3_K_M | 3.74GB | | WiNGPT2-Llama-3-8B-Base.Q3_K_L.gguf | Q3_K_L | 4.03GB | | WiNGPT2-Llama-3-8B-Base.IQ4_XS.gguf | IQ4_XS | 4.18GB | | WiNGPT2-Llama-3-8B-Base.Q4_0.gguf | Q4_0 | 4.34GB | | WiNGPT2-Llama-3-8B-Base.IQ4_NL.gguf | IQ4_NL | 4.38GB | | WiNGPT2-Llama-3-8B-Base.Q4_K_S.gguf | Q4_K_S | 4.37GB | | WiNGPT2-Llama-3-8B-Base.Q4_K.gguf | Q4_K | 4.58GB | | WiNGPT2-Llama-3-8B-Base.Q4_K_M.gguf | Q4_K_M | 4.58GB | | WiNGPT2-Llama-3-8B-Base.Q4_1.gguf | Q4_1 | 4.78GB | | WiNGPT2-Llama-3-8B-Base.Q5_0.gguf | Q5_0 | 5.21GB | | WiNGPT2-Llama-3-8B-Base.Q5_K_S.gguf | Q5_K_S | 5.21GB | | WiNGPT2-Llama-3-8B-Base.Q5_K.gguf | Q5_K | 5.34GB | | WiNGPT2-Llama-3-8B-Base.Q5_K_M.gguf | Q5_K_M | 5.34GB | | WiNGPT2-Llama-3-8B-Base.Q5_1.gguf | Q5_1 | 5.65GB | | WiNGPT2-Llama-3-8B-Base.Q6_K.gguf | Q6_K | 6.14GB | | WiNGPT2-Llama-3-8B-Base.Q8_0.gguf | Q8_0 | 7.95GB | Original model description: --- language: tags: license: apache-2.0 ---",
    "quick_links": [],
    "benchmark_table_html": "",
    "readme_markdown": "Quantization made by Richard Erkhov.\n\n[Github](https://github.com/RichardErkhov)\n\n[Discord](https://discord.gg/pvy7H8DZMG)\n\n[Request more models](https://github.com/RichardErkhov/quant_request)\n\n\nWiNGPT2-Llama-3-8B-Base - GGUF\n- Model creator: https://huggingface.co/winninghealth/\n- Original model: https://huggingface.co/winninghealth/WiNGPT2-Llama-3-8B-Base/\n\n\n| Name | Quant method | Size |\n| ---- | ---- | ---- |\n| [WiNGPT2-Llama-3-8B-Base.Q2_K.gguf](https://huggingface.co/RichardErkhov/winninghealth_-_WiNGPT2-Llama-3-8B-Base-gguf/blob/main/WiNGPT2-Llama-3-8B-Base.Q2_K.gguf) | Q2_K | 2.96GB |\n| [WiNGPT2-Llama-3-8B-Base.IQ3_XS.gguf](https://huggingface.co/RichardErkhov/winninghealth_-_WiNGPT2-Llama-3-8B-Base-gguf/blob/main/WiNGPT2-Llama-3-8B-Base.IQ3_XS.gguf) | IQ3_XS | 3.28GB |\n| [WiNGPT2-Llama-3-8B-Base.IQ3_S.gguf](https://huggingface.co/RichardErkhov/winninghealth_-_WiNGPT2-Llama-3-8B-Base-gguf/blob/main/WiNGPT2-Llama-3-8B-Base.IQ3_S.gguf) | IQ3_S | 3.43GB |\n| [WiNGPT2-Llama-3-8B-Base.Q3_K_S.gguf](https://huggingface.co/RichardErkhov/winninghealth_-_WiNGPT2-Llama-3-8B-Base-gguf/blob/main/WiNGPT2-Llama-3-8B-Base.Q3_K_S.gguf) | Q3_K_S | 3.41GB |\n| [WiNGPT2-Llama-3-8B-Base.IQ3_M.gguf](https://huggingface.co/RichardErkhov/winninghealth_-_WiNGPT2-Llama-3-8B-Base-gguf/blob/main/WiNGPT2-Llama-3-8B-Base.IQ3_M.gguf) | IQ3_M | 3.52GB |\n| [WiNGPT2-Llama-3-8B-Base.Q3_K.gguf](https://huggingface.co/RichardErkhov/winninghealth_-_WiNGPT2-Llama-3-8B-Base-gguf/blob/main/WiNGPT2-Llama-3-8B-Base.Q3_K.gguf) | Q3_K | 3.74GB |\n| [WiNGPT2-Llama-3-8B-Base.Q3_K_M.gguf](https://huggingface.co/RichardErkhov/winninghealth_-_WiNGPT2-Llama-3-8B-Base-gguf/blob/main/WiNGPT2-Llama-3-8B-Base.Q3_K_M.gguf) | Q3_K_M | 3.74GB |\n| [WiNGPT2-Llama-3-8B-Base.Q3_K_L.gguf](https://huggingface.co/RichardErkhov/winninghealth_-_WiNGPT2-Llama-3-8B-Base-gguf/blob/main/WiNGPT2-Llama-3-8B-Base.Q3_K_L.gguf) | Q3_K_L | 4.03GB |\n| [WiNGPT2-Llama-3-8B-Base.IQ4_XS.gguf](https://huggingface.co/RichardErkhov/winninghealth_-_WiNGPT2-Llama-3-8B-Base-gguf/blob/main/WiNGPT2-Llama-3-8B-Base.IQ4_XS.gguf) | IQ4_XS | 4.18GB |\n| [WiNGPT2-Llama-3-8B-Base.Q4_0.gguf](https://huggingface.co/RichardErkhov/winninghealth_-_WiNGPT2-Llama-3-8B-Base-gguf/blob/main/WiNGPT2-Llama-3-8B-Base.Q4_0.gguf) | Q4_0 | 4.34GB |\n| [WiNGPT2-Llama-3-8B-Base.IQ4_NL.gguf](https://huggingface.co/RichardErkhov/winninghealth_-_WiNGPT2-Llama-3-8B-Base-gguf/blob/main/WiNGPT2-Llama-3-8B-Base.IQ4_NL.gguf) | IQ4_NL | 4.38GB |\n| [WiNGPT2-Llama-3-8B-Base.Q4_K_S.gguf](https://huggingface.co/RichardErkhov/winninghealth_-_WiNGPT2-Llama-3-8B-Base-gguf/blob/main/WiNGPT2-Llama-3-8B-Base.Q4_K_S.gguf) | Q4_K_S | 4.37GB |\n| [WiNGPT2-Llama-3-8B-Base.Q4_K.gguf](https://huggingface.co/RichardErkhov/winninghealth_-_WiNGPT2-Llama-3-8B-Base-gguf/blob/main/WiNGPT2-Llama-3-8B-Base.Q4_K.gguf) | Q4_K | 4.58GB |\n| [WiNGPT2-Llama-3-8B-Base.Q4_K_M.gguf](https://huggingface.co/RichardErkhov/winninghealth_-_WiNGPT2-Llama-3-8B-Base-gguf/blob/main/WiNGPT2-Llama-3-8B-Base.Q4_K_M.gguf) | Q4_K_M | 4.58GB |\n| [WiNGPT2-Llama-3-8B-Base.Q4_1.gguf](https://huggingface.co/RichardErkhov/winninghealth_-_WiNGPT2-Llama-3-8B-Base-gguf/blob/main/WiNGPT2-Llama-3-8B-Base.Q4_1.gguf) | Q4_1 | 4.78GB |\n| [WiNGPT2-Llama-3-8B-Base.Q5_0.gguf](https://huggingface.co/RichardErkhov/winninghealth_-_WiNGPT2-Llama-3-8B-Base-gguf/blob/main/WiNGPT2-Llama-3-8B-Base.Q5_0.gguf) | Q5_0 | 5.21GB |\n| [WiNGPT2-Llama-3-8B-Base.Q5_K_S.gguf](https://huggingface.co/RichardErkhov/winninghealth_-_WiNGPT2-Llama-3-8B-Base-gguf/blob/main/WiNGPT2-Llama-3-8B-Base.Q5_K_S.gguf) | Q5_K_S | 5.21GB |\n| [WiNGPT2-Llama-3-8B-Base.Q5_K.gguf](https://huggingface.co/RichardErkhov/winninghealth_-_WiNGPT2-Llama-3-8B-Base-gguf/blob/main/WiNGPT2-Llama-3-8B-Base.Q5_K.gguf) | Q5_K | 5.34GB |\n| [WiNGPT2-Llama-3-8B-Base.Q5_K_M.gguf](https://huggingface.co/RichardErkhov/winninghealth_-_WiNGPT2-Llama-3-8B-Base-gguf/blob/main/WiNGPT2-Llama-3-8B-Base.Q5_K_M.gguf) | Q5_K_M | 5.34GB |\n| [WiNGPT2-Llama-3-8B-Base.Q5_1.gguf](https://huggingface.co/RichardErkhov/winninghealth_-_WiNGPT2-Llama-3-8B-Base-gguf/blob/main/WiNGPT2-Llama-3-8B-Base.Q5_1.gguf) | Q5_1 | 5.65GB |\n| [WiNGPT2-Llama-3-8B-Base.Q6_K.gguf](https://huggingface.co/RichardErkhov/winninghealth_-_WiNGPT2-Llama-3-8B-Base-gguf/blob/main/WiNGPT2-Llama-3-8B-Base.Q6_K.gguf) | Q6_K | 6.14GB |\n| [WiNGPT2-Llama-3-8B-Base.Q8_0.gguf](https://huggingface.co/RichardErkhov/winninghealth_-_WiNGPT2-Llama-3-8B-Base-gguf/blob/main/WiNGPT2-Llama-3-8B-Base.Q8_0.gguf) | Q8_0 | 7.95GB |\n\n\n\n\nOriginal model description:\n---\nlanguage:\n- en\n- zh\ntags:\n- medical\nlicense: apache-2.0\n---\n\n## WiNGPT2\n\n[WiNGPT](https://github.com/winninghealth/WiNGPT2) 是一个基于GPT的医疗垂直领域大模型，旨在将专业的医学知识、医疗信息、数据融会贯通，为医疗行业提供智能化的医疗问答、诊断支持和医学知识等信息服务，提高诊疗效率和医疗服务质量。\n\n## 更新日志\n\n[2024/04/23] 更新 WiNGPT2-Llama-3-8B-Base 模型（中文增强/多语言）与测评结果\n\n[2024/04/01] 更新 WiNEval 测评结果\n\n[2024/03/05] 开源7B/14B-Chat-4bit模型权重: [🤗](https://huggingface.co/winninghealth/WiNGPT2-7B-Chat-AWQ)WiNGPT2-7B-Chat-4bit和[🤗](https://huggingface.co/winninghealth/WiNGPT2-14B-Chat-AWQ)WiNGPT2-14B-Chat-4bit。\n\n[2023/12/20] 新增用户微信群二维码，有效期到12月27日，扫码进群。\n\n[2023/12/18] 发布卫宁健康医疗模型测评方案 WiNEval-MCKQuiz的评测结果。\n\n[2023/12/12] 开源 WiNGPT2 14B模型权重: [🤗](https://huggingface.co/winninghealth/WiNGPT2-14B-Base)WiNGPT2-14B-Base 和 [🤗](https://huggingface.co/winninghealth/WiNGPT2-14B-Chat)WiNGPT2-14B-Chat。 \n\n[2023/11/02] [34B模型平台测试](https://wingpt.winning.com.cn/) 和 [欢迎加入微信讨论群](https://github.com/winninghealth/WiNGPT2/blob/main/assets/WiNGPT_GROUP.JPG)\n\n[2023/10/13] 更新一个简单的[Chatbot示例](#部署)，可以进行简单的多轮对话。\n\n[2023/09/26] 开源 WiNGPT2 与7B模型权重: [🤗](https://huggingface.co/winninghealth/WiNGPT2-7B-Base)WiNGPT2-7B-Base 和 [🤗](https://huggingface.co/winninghealth/WiNGPT2-7B-Chat)WiNGPT2-7B-Chat。 \n\n## 如何使用\n\n### 推理\n\n```python\nfrom transformers import AutoModelForCausalLM, AutoTokenizer\n\nmodel_path = \"WiNGPT-Llama-3-8B-Chat\"\ndevice = \"cuda\"\n\ntokenizer = AutoTokenizer.from_pretrained(model_path)\nmodel = AutoModelForCausalLM.from_pretrained(model_path).to(device)\nmodel = model.eval()\n\n\ntext = 'User：WiNGPT, 你好<|end_of_text|>\\n Assistant：'\ninputs = tokenizer.encode(text, return_tensors=\"pt\").to(device)\noutputs = model.generate(inputs, repetition_penalty=1.1, max_new_tokens=1024)\nresponse = tokenizer.decode(outputs[0])\nprint(response)\n\n## 输出结果：你好！今天我能为你做些什么？<|end_of_text|>\n```\n\n### 提示\n\nWiNGPT-Llama-3-8B-Chat 使用了自定义的提示格式：\n\n用户角色：System/User/Assistant\n\nchat_template:\n\n```jinja2\n\"{% for message in messages %}{% if message['role'] == 'system' %}System：{% endif %}{% if message['role'] == 'user' %}User：{% endif %}{% if message['role'] == 'assistant' %}Assistant：{% endif %}{{ message['content'] }}<|end_of_text|>\\n {% endfor %}Assistant：\"\n```\n\n**指令提示**示例：\n\n```\nUser：WiNGPT, 你好<|end_of_text|>\\n Assistant：\n```\n\n**多轮对话**示例：\n\n```\nUser：WiNGPT, 你好<|end_of_text|>\\n Assistant：你好！今天我能为你做些什么？<|end_of_text|>\\n User：你是谁？<|end_of_text|>\\n Assistant：\n```\n\n**翻译功能**示例：\n\n```\nSystem：作为医疗领域的智能助手，WiNGPT将提供中英翻译服务。用户输入的中文或英文内容将由WiNGPT进行准确的翻译，以满足用户的语言需求。<|end_of_text|>\\n User：Life is short, you know, and time is so swift; Rivers are wide, so wide, and ships sail far.<|end_of_text|>\\n Assistant：\n```\n\n## 模型卡\n\n####  训练配置与参数\n\n| 名称                    | 训练策略           | 长度 | 精度 | 学习率 | Weight_decay | Epochs | GPUs   |\n| ----------------------- | ------------------ | ---- | ---- | ------ | ------------ | ------ | ------ |\n| WiNGPT2-Llama-3-8B-Base | 继续预训练 (20G)   | 8192 | bf16 | 5e-5   | 0.05         | 2      | A100*8 |\n| WiNGPT2-Llama-3-8B-Chat | 微调/对齐 (50万条) | 8192 | bf16 | 5e-6   | 0.01         | 4      | A100*8 |\n\n#### 训练数据\n\n预训练数据约20G，指令微调对齐数据约50万条，[详细内容](https://github.com/winninghealth/WiNGPT2?tab=readme-ov-file#%E8%AE%AD%E7%BB%83%E6%95%B0%E6%8D%AE) 。\n\n## 中文医疗评测 - WiNEval\n\n更新时间：2024-04-23\n|                               | Type                   | MCKQuiz     | MSceQA      |\n| ----------------------------- | ---------------------- | ----------- | ----------- |\n| **WiNGPT-Llama-3-8B-Base**    | Continued Pre-training | 66.3        | /           |\n| Meta-Llama-3-8B               | Pre-training           | 37          | /           |\n|                               |                        |             |             |\n| **WiNGPT-Llama-3-8B-Chat**    | Finetuning/Alignment   | 65.2        | 79.8        |\n| Meta-Llama-3-8B-Instruct      | Finetuning/Alignment   | 49.8        | 76.3        |\n| Meta-Llama-3-70B-Instruct-AWQ | Finetuning/Alignment   | 73.5        | 78.6        |\n|                               |                        |             |             |\n\n*MCKQuiz（客观题）：17个科目分类13060选择题；输入问题和选项，让模型输出答案。根据标准答案判断对错，统计准确率。*\n\n*MSceQA（主观题）：由细分领域场景题目构成，包含八大业务场景，17个一级分类和32个二级分类。使用人工/模型对模型的回答进行准确性、相关性、一致性、完整性、权威性评价，并参照标准答案对模型生成的答案进行评分。*\n\n[其他WiNEval评测结果](https://github.com/winninghealth/WiNGPT2?tab=readme-ov-file#2-%E5%8D%AB%E5%AE%81%E5%81%A5%E5%BA%B7%E5%8C%BB%E7%96%97%E6%A8%A1%E5%9E%8B%E6%B5%8B%E8%AF%84%E6%96%B9%E6%A1%88-winevalzero-shot)\n\n### 企业服务\n\n[通过WiNGPT测试平台申请密钥或与我们取得联系](https://wingpt.winning.com.cn/)\n\n\n## 局限性与免责声明\n\n(a) WiNGPT2 是一个专业医疗领域的大语言模型，可为一般用户提供拟人化AI医生问诊和问答功能，以及一般医学领域的知识问答。对于专业医疗人士，WiNGPT2 提供关于患者病情的诊断、用药和健康建议等方面的回答的建议仅供参考。\n\n(b) 您应理解 WiNGPT2 仅提供信息和建议，不能替代医疗专业人士的意见、诊断或治疗建议。在使用 WiNGPT2 的信息之前，请寻求医生或其他医疗专业人员的建议，并独立评估所提供的信息。\n\n(c) WiNGPT2 的信息可能存在错误或不准确。卫宁健康不对 WiNGPT2 的准确性、可靠性、完整性、质量、安全性、及时性、性能或适用性提供任何明示或暗示的保证。使用 WiNGPT2 所产生的结果和决策由您自行承担。第三方原因而给您造成的损害结果承担责任。\n\n## 许可证\n\n1. 本项目授权协议为 Apache License 2.0，模型权重需要遵守基础模型 [Llama-3-8B](https://github.com/meta-llama/llama3) 相关协议及其[许可证](https://llama.meta.com/llama3/license)，详细内容参照其网站。\n\n2. 使用本项目包括模型权重时请引用本项目：https://github.com/winninghealth/WiNGPT2\n\n## 联系我们\n\n网站：https://www.winning.com.cn\n\n邮箱：wair@winning.com.cn\n\n",
    "related_quantizations": []
  },
  "tags": [
    "gguf",
    "endpoints_compatible",
    "region:us"
  ],
  "likes": 0,
  "downloads": 308,
  "gated": false,
  "private": false,
  "last_modified": "2024-06-16T09:36:30.000Z",
  "created_at": "2024-06-16T05:41:59.000Z",
  "pipeline_tag": "",
  "library_name": ""
}

{
  "_id": "666e7b27936d46ae833782f3",
  "id": "RichardErkhov/winninghealth_-_WiNGPT2-Llama-3-8B-Base-gguf",
  "modelId": "RichardErkhov/winninghealth_-_WiNGPT2-Llama-3-8B-Base-gguf",
  "sha": "063bf66154cd2d7f749644d49527287a20a82d8d",
  "createdAt": "2024-06-16T05:41:59.000Z",
  "lastModified": "2024-06-16T09:36:30.000Z",
  "author": "RichardErkhov",
  "downloads": 308,
  "likes": 0,
  "gated": false,
  "private": false,
  "pipeline_tag": "",
  "library_name": "",
  "siblings_count": 24
}