richarderkhov/liteai_-_hare-1.1b-base-gguf overview

{
  "metadata": {},
  "card_data": {
    "frontmatter": {},
    "hero_image_url": "./logo.jpg",
    "summary": "GitHub | 🤖 ModelScope | 📑 ArXiv   Hare-1.1B-base is a pre-trained model developed by the LiteAI Team from China Telecom Guizhou Branch. We use a mix of high-quality open-source data and strategy-generated data as pre-train data. The model is only 1.1B in size and has performed well on the Open LLM Leaderboard. Hare-1.1B-base是由中国电信股份有限公司贵州分公司LiteAI团队开发的预训练模型。我们使用高质量开源和策略生成的合成数据作为预训练数据。该模型大小仅为1.1B，并在Open LLM Leaderboard上表现优异。",
    "quick_links": [],
    "benchmark_table_html": "<table>\n  <tr>\n    <td><img src=\"./ori1_1.gif\" alt=\"First demo\" width=\"50%\"/></td>\n    <td><img src=\"./ori2_2.gif\" alt=\"Second demo\" width=\"50%\"/></td>\n  </tr>\n</table>",
    "readme_markdown": "Quantization made by Richard Erkhov.\n\n[Github](https://github.com/RichardErkhov)\n\n[Discord](https://discord.gg/pvy7H8DZMG)\n\n[Request more models](https://github.com/RichardErkhov/quant_request)\n\n\nHare-1.1B-base - GGUF\n- Model creator: https://huggingface.co/LiteAI/\n- Original model: https://huggingface.co/LiteAI/Hare-1.1B-base/\n\n\n| Name | Quant method | Size |\n| ---- | ---- | ---- |\n| [Hare-1.1B-base.Q2_K.gguf](https://huggingface.co/RichardErkhov/LiteAI_-_Hare-1.1B-base-gguf/blob/main/Hare-1.1B-base.Q2_K.gguf) | Q2_K | 0.41GB |\n| [Hare-1.1B-base.IQ3_XS.gguf](https://huggingface.co/RichardErkhov/LiteAI_-_Hare-1.1B-base-gguf/blob/main/Hare-1.1B-base.IQ3_XS.gguf) | IQ3_XS | 0.45GB |\n| [Hare-1.1B-base.IQ3_S.gguf](https://huggingface.co/RichardErkhov/LiteAI_-_Hare-1.1B-base-gguf/blob/main/Hare-1.1B-base.IQ3_S.gguf) | IQ3_S | 0.48GB |\n| [Hare-1.1B-base.Q3_K_S.gguf](https://huggingface.co/RichardErkhov/LiteAI_-_Hare-1.1B-base-gguf/blob/main/Hare-1.1B-base.Q3_K_S.gguf) | Q3_K_S | 0.47GB |\n| [Hare-1.1B-base.IQ3_M.gguf](https://huggingface.co/RichardErkhov/LiteAI_-_Hare-1.1B-base-gguf/blob/main/Hare-1.1B-base.IQ3_M.gguf) | IQ3_M | 0.49GB |\n| [Hare-1.1B-base.Q3_K.gguf](https://huggingface.co/RichardErkhov/LiteAI_-_Hare-1.1B-base-gguf/blob/main/Hare-1.1B-base.Q3_K.gguf) | Q3_K | 0.52GB |\n| [Hare-1.1B-base.Q3_K_M.gguf](https://huggingface.co/RichardErkhov/LiteAI_-_Hare-1.1B-base-gguf/blob/main/Hare-1.1B-base.Q3_K_M.gguf) | Q3_K_M | 0.52GB |\n| [Hare-1.1B-base.Q3_K_L.gguf](https://huggingface.co/RichardErkhov/LiteAI_-_Hare-1.1B-base-gguf/blob/main/Hare-1.1B-base.Q3_K_L.gguf) | Q3_K_L | 0.56GB |\n| [Hare-1.1B-base.IQ4_XS.gguf](https://huggingface.co/RichardErkhov/LiteAI_-_Hare-1.1B-base-gguf/blob/main/Hare-1.1B-base.IQ4_XS.gguf) | IQ4_XS | 0.58GB |\n| [Hare-1.1B-base.Q4_0.gguf](https://huggingface.co/RichardErkhov/LiteAI_-_Hare-1.1B-base-gguf/blob/main/Hare-1.1B-base.Q4_0.gguf) | Q4_0 | 0.61GB |\n| [Hare-1.1B-base.IQ4_NL.gguf](https://huggingface.co/RichardErkhov/LiteAI_-_Hare-1.1B-base-gguf/blob/main/Hare-1.1B-base.IQ4_NL.gguf) | IQ4_NL | 0.61GB |\n| [Hare-1.1B-base.Q4_K_S.gguf](https://huggingface.co/RichardErkhov/LiteAI_-_Hare-1.1B-base-gguf/blob/main/Hare-1.1B-base.Q4_K_S.gguf) | Q4_K_S | 0.61GB |\n| [Hare-1.1B-base.Q4_K.gguf](https://huggingface.co/RichardErkhov/LiteAI_-_Hare-1.1B-base-gguf/blob/main/Hare-1.1B-base.Q4_K.gguf) | Q4_K | 0.64GB |\n| [Hare-1.1B-base.Q4_K_M.gguf](https://huggingface.co/RichardErkhov/LiteAI_-_Hare-1.1B-base-gguf/blob/main/Hare-1.1B-base.Q4_K_M.gguf) | Q4_K_M | 0.64GB |\n| [Hare-1.1B-base.Q4_1.gguf](https://huggingface.co/RichardErkhov/LiteAI_-_Hare-1.1B-base-gguf/blob/main/Hare-1.1B-base.Q4_1.gguf) | Q4_1 | 0.67GB |\n| [Hare-1.1B-base.Q5_0.gguf](https://huggingface.co/RichardErkhov/LiteAI_-_Hare-1.1B-base-gguf/blob/main/Hare-1.1B-base.Q5_0.gguf) | Q5_0 | 0.73GB |\n| [Hare-1.1B-base.Q5_K_S.gguf](https://huggingface.co/RichardErkhov/LiteAI_-_Hare-1.1B-base-gguf/blob/main/Hare-1.1B-base.Q5_K_S.gguf) | Q5_K_S | 0.73GB |\n| [Hare-1.1B-base.Q5_K.gguf](https://huggingface.co/RichardErkhov/LiteAI_-_Hare-1.1B-base-gguf/blob/main/Hare-1.1B-base.Q5_K.gguf) | Q5_K | 0.74GB |\n| [Hare-1.1B-base.Q5_K_M.gguf](https://huggingface.co/RichardErkhov/LiteAI_-_Hare-1.1B-base-gguf/blob/main/Hare-1.1B-base.Q5_K_M.gguf) | Q5_K_M | 0.74GB |\n| [Hare-1.1B-base.Q5_1.gguf](https://huggingface.co/RichardErkhov/LiteAI_-_Hare-1.1B-base-gguf/blob/main/Hare-1.1B-base.Q5_1.gguf) | Q5_1 | 0.79GB |\n| [Hare-1.1B-base.Q6_K.gguf](https://huggingface.co/RichardErkhov/LiteAI_-_Hare-1.1B-base-gguf/blob/main/Hare-1.1B-base.Q6_K.gguf) | Q6_K | 0.86GB |\n| [Hare-1.1B-base.Q8_0.gguf](https://huggingface.co/RichardErkhov/LiteAI_-_Hare-1.1B-base-gguf/blob/main/Hare-1.1B-base.Q8_0.gguf) | Q8_0 | 1.11GB |\n\n\n\n\nOriginal model description:\n---\nlicense: apache-2.0\nlanguage:\n- en\nlibrary_name: transformers\npipeline_tag: text-generation\ntags:\n- Hare\ndatasets:\n- cerebras/SlimPajama-627B\n- HuggingFaceTB/cosmopedia\narxiv: 2406.11410\n---\n\n<a id=\"english\"></a>\n\n<p align=\"center\">\n<img width=\"400px\" alt=\"Lite-AI\" src=\"./logo.jpg\">\n</p>\n</div>\n\n\n\n# Hare-1.1B-base\n<p align=\"center\">\n    <a href=\"https://github.com/LiteAI-Team/HARE\">GitHub</a> | 🤖 <a href=\"https://modelscope.cn/models/LiteAITeam/Hare-1.1B-base\">ModelScope</a> | 📑 <a href=\"https://arxiv.org/abs/2406.11410\">ArXiv </a>\n</p>\n      \nHare-1.1B-base is a pre-trained model developed by the LiteAI Team from China Telecom Guizhou Branch. We use a mix of high-quality open-source data and strategy-generated data as pre-train data. The model is only 1.1B in size and has performed well on the Open LLM Leaderboard.\n\n- We chose Mistral as the foundational architecture and reused its tokenizer, reducing the number of parameters by adjusting the hyperparameters of its model architecture. Consequently, our model can be directly applied to numerous open-source projects that support Mistral, such as vLLM.\n\n- Our model has a parameter count of only 1.1 billion, allowing us to deploy it on consumer-grade GPUs, mobile devices, and other cost-effective platforms.\n\n- We have explored efficient training at FP8 precision and have compiled a set of best practices, hoping to contribute as much as we can to LLM training in the open-source community. For best practices, please see our GitHub homepage.\n\n- We are currently developing and adapting for Chinese language support.\n\nHare-1.1B-base是由中国电信股份有限公司贵州分公司LiteAI团队开发的预训练模型。我们使用高质量开源和策略生成的合成数据作为预训练数据。该模型大小仅为1.1B，并在Open LLM Leaderboard上表现优异。\n\n- 我们选择Mistral架构作为基础框架，并复用了其分词器，通过调整模型架构的超参来减少参数量。因此，我们的模型可以直接应用于许多支持Mistral的开源项目，如vLLM。\n\n- 我们模型的参数量仅为 11 亿，因此，我们可以将模型部署到消费级显卡、手机端等成本较低的设备上。\n\n- 我们探索了FP8精度下的高效训练，并总结了一份最佳实践，希望能为开源社区LLM训练作出力所能及的贡献。最佳实践请看GitHub主页。\n\n- 我们正在研发与适配中文。\n\n## Model Details 模型细节\n| Model | Training Tokens | Hidden layers | Hidden Size | Attention Heads | Context Length |\n|:------:|:--------:|:---------:|:-------------:|:-----------------:|:----------------:|\n|Hare-1.1B-base   | ~ 600B |22     | 2048        | 32              | 2048  |\n\n\n## Model Description 模型说明\n- **Developed by:** LiteAI Team\n- **Institution:** China Telecom Guizhou Branch \n- **Model size:** 1.1B\n- **License:** Apache 2.0\n\n- **开发者:** LiteAI Team\n- **机构:** 中国电信股份有限公司贵州分公司\n- **模型大小:** 1.1B\n- **协议:** Apache 2.0\n\n## Uses 模型使用\n\n### Inference 推理\n```python\nimport torch\nfrom transformers import AutoTokenizer, AutoModelForCausalLM\n\ndevice = \"cuda\" if torch.cuda.is_available() else \"cpu\"\nmodel_path = \"LiteAI-Team/Hare-1.1B-base\"\ntokenizer = AutoTokenizer.from_pretrained(model_path)\nmodel = AutoModelForCausalLM.from_pretrained(model_path)\nmodel.to(device)\n\nprompt = \"Write a poem based on the landscape of Guizhou:\"\ntokens = tokenizer(prompt, add_special_tokens=True, return_tensors='pt').to(device)\noutput = model.generate(**tokens,max_new_tokens=128)\n\noutput_tokens = output[0].cpu().numpy()[tokens.input_ids.size()[1]:]\noutput_string = tokenizer.decode(output_tokens)\nprint(output_string)\n>> \"\"\"The Guizhou landscape is a sight to behold,\nA place where nature's beauty is unmatched,\nA land of towering mountains and vast plains,\nA paradise for those who seek to explore.\n\nThe mountains rise high above the sky,\nA sight to beholder, a sight to see,\nThe valleys stretch out as far as the eye can see,\nA landscape of endless beauty and grace.\"\"\"\n```\nInstall with vllm:\n```shell\npip install vllm\n```\n```python\nfrom vllm import LLM, SamplingParams\nfrom transformers import AutoTokenizer\n\nmodel_path = \"LiteAI-Team/Hare-1.1B-base\"\nllm = LLM(model=model_path, trust_remote_code=True, tensor_parallel_size=4)\n\nquery = \"Write a poem based on the landscape of Guizhou:\"\nsampling_params = SamplingParams(temperature=0.8, top_p=0.95, max_tokens=64)\noutputs = llm.generate(query, sampling_params)\nprint(outputs)\n```\n\n## Edge Deployment Demo 端侧部署\nOur model has only 1.1 billion parameters, and after Int4 quantization, it occupies just 0.6GB of space, allowing for easy deployment on mobile devices, The [Hare-1.1B-Chat](https://huggingface.co/LiteAI/Hare-1.1B-Chat) model weights have been open-sourced.\n- Android：We chose MLC-LLM as the deployment framework and conducted deployment testing of the Chat model on the Redmi K40.\n- iOS & HarmonyOS：We will conduct deployment testing on the aforementioned devices in the future.\n\n我们的模型参数量仅有1.1B，经Int4量化后，模型仅占用0.6G的空间，可轻松部署在手机端，[Hare-1.1B-Chat](https://huggingface.co/LiteAI/Hare-1.1B-Chat)模型权重已经开源。\n- Android：我们选择MLC-LLM作为部署框架，在Redmi K40上进行Chat模型的部署测试。\n- iOS & HarmonyOS：我们将在未来对上述设备进行部署测试。\n<table>\n  <tr>\n    <td><img src=\"./ori1_1.gif\" alt=\"First demo\" width=\"50%\"/></td>\n    <td><img src=\"./ori2_2.gif\" alt=\"Second demo\" width=\"50%\"/></td>\n  </tr>\n</table>\n\n\n## Tool Call 工具调用实践\n- To fully leverage the advantages of deploying small models on edge devices, we referred to the work of [Octopus-v2](https://huggingface.co/NexaAIDev/Octopus-v2) and replaced Gemma-2B with [Hare-1.1B-Tool](https://huggingface.co/LiteAI/Hare-1.1B-Tool), successfully enabling the invocation of Android system APIs and the orchestration of tool functionalities in composite scenarios on mobile devices.\n- Please click the image below to view.\n  \n- 为完全发挥出小模型在端侧部署上的优势，我们对照[Octopus-v2](https://huggingface.co/NexaAIDev/Octopus-v2)的工作并使用[Hare-1.1B-Tool](https://huggingface.co/LiteAI/Hare-1.1B-Tool)替换Gemma-2B，成功在手机端实现安卓系统API调用和组合场景下的工具调用能力。\n- 请您点击下面图片观看。[<img src=\"./ee32f5b94fbfee4e95507a0db3e069a53d1931db.jpg\" alt=\"alt text\" width=\"600\"/>](https://www.bilibili.com/video/BV1Ry411b7yx/?vd_source=d4f08e4b18c51571a1b53a20a8d58c10)\n\n## Evaluation Results 评测结果\n- Additionally, we conducted explorations and experiments addressing the issue of benchmark data leakage. For a detailed analysis, please refer to our [paper](https://arxiv.org/abs/2406.11410).\n- 同时，我们针对benchmark数据泄漏问题做了探索与实验，详细分析请参考我们的[论文](https://arxiv.org/abs/2406.11410)。 \n| Model(base)                               | Size  | avg   | MMLU | ARC-C | TruthfulQA | Winogrande | Hellaswag | GSM8K |\n|:-------------------------------------:|:-------:|:-------:|:------:|:-------:|:------------:|:------------:|:-----------:|:-------:|\n| phi-1_5                            | 1.3B   | 47.69 | 43.89| 52.9  | 40.89      | 72.22      | 63.79     |12.43 |\n| Qwen-1.5                          | 1.8B  | 46.55 | 46.71| 37.88 | 39.43      | 60.3       | 61.42     |33.59 | \n| stablelm-2                        | 1.6B  | 45.25 | 38.95| 43.34 | 36.78      | 64.56      | 70.45     |17.44 | \n| __Hare__                       | 1.1B  | 40.17 | 35.74| 38.4  | 42.08      | 59.27      | 57.46     |8.04  |\n| H2o-danube                            | 1.8B  | 39.12 | 25.94| 39.42 | 33.86      | 64.48      | 69.58     |1.44  |\n| OpenELM                   | 1.1B  | 38.47 | 27.05| 36.69 | 33.86      | 63.22      | 65.71     |1.21  |\n| csg-wukong                            | 1B  | 37.78 | 25.33| 37.71 | 42.79      | 56.67      | 58.93     |5.23  |\n| TinyLlama-3T                           | 1.1B  | 36.42 | 26.04| 33.87 | 37.32      | 59.51      | 60.31     |1.44  |\n\n## License 协议\n- This repository is open-sourced under the Apache-2.0 license.\n\n- The Hare series model weights are currently fully open only for academic research.\n\n- 本仓库遵循Apache-2.0协议开源。\n\n- Hare系列模型权重目前仅对学术研究完全开放。\n\n## Statement 声明\n- Hare is a language model trained on a mix of open-source pre-training data and strategy-generated pre-training data. It lacks the ability to make value judgments and cannot understand or express personal opinions. The outputs of the model do not represent the views or positions of the LiteAI development team.\n- Therefore, the content generated using Hare may contain biased viewpoints and inaccuracies. Please use it at your discretion.\n- Similarly, we will not assume any responsibility for risks and issues arising from users deliberately using Hare to generate harmful content.\n- For modifications related to this repository, please contact: zhangly41 At(@) chinatelecom.cn.\n- Team contact information: chensq27 At(@) chinatelecom.cn. The LiteAI Team looks forward to collaborating with you.\n\n- Hare是一个基于开源预训练数据和策略合成预训练数据混合训练得到的语言模型，它不具备价值判断能力，无法理解、表达个人观点，模型的输出内容不代表LiteAI开发团队的观点与立场。\n- 因此，您使用Hare生成的内容可能存有偏观点和不实情况，请您酌情使用。\n- 同样，我们将不承担用户故意使用Hare进行有害内容生成所带来的任何风险与问题。\n- 如涉及到本仓库的修改，请联系：zhangly41 At(@) chinatelecom.cn。\n- 团队联系方式：chensq27 At(@) chinatelecom.cn，LiteAI团队期待您的合作。\n\n\n## Citation 工作引用\n- If you find Hare helpful for your work, please consider citing our [paper](https://arxiv.org/abs/2406.11410).\n- 如果您觉得Hare对您的工作起到了帮助，请考虑引用我们的[论文](https://arxiv.org/abs/2406.11410)。\n```\n@misc{zhang2024harehumanpriorskey,\n      title={HARE: HumAn pRiors, a key to small language model Efficiency}, \n      author={Lingyun Zhang and Bin jin and Gaojian Ge and Lunhui Liu and Xuewen Shen and Mingyong Wu and Houqian Zhang and Yongneng Jiang and Shiqi Chen and Shi Pu},\n      year={2024},\n      eprint={2406.11410},\n      archivePrefix={arXiv},\n      primaryClass={cs.CL}\n      url={https://arxiv.org/abs/2406.11410}, \n}\n```\n\n\n",
    "related_quantizations": []
  },
  "tags": [
    "gguf",
    "arxiv:2406.11410",
    "endpoints_compatible",
    "region:us"
  ],
  "likes": 0,
  "downloads": 793,
  "gated": false,
  "private": false,
  "last_modified": "2024-07-05T02:53:15.000Z",
  "created_at": "2024-07-05T02:45:48.000Z",
  "pipeline_tag": "",
  "library_name": ""
}

{
  "_id": "66875e5cf3ae5316122ebe87",
  "id": "RichardErkhov/LiteAI_-_Hare-1.1B-base-gguf",
  "modelId": "RichardErkhov/LiteAI_-_Hare-1.1B-base-gguf",
  "sha": "66f7e1523b781fdb947faa69828350b992836d10",
  "createdAt": "2024-07-05T02:45:48.000Z",
  "lastModified": "2024-07-05T02:53:15.000Z",
  "author": "RichardErkhov",
  "downloads": 793,
  "likes": 0,
  "gated": false,
  "private": false,
  "pipeline_tag": "",
  "library_name": "",
  "siblings_count": 24
}