richarderkhov/sakanaai_-_evollm-jp-v1-7b-gguf overview

{
  "metadata": {},
  "card_data": {
    "frontmatter": {},
    "hero_image_url": "",
    "summary": "🤗 Models | 📚 Paper | 📝 Blog | 🐦 Twitter  **EvoLLM-JP-v1-7B** is an experimental general-purpose Japanese LLM. This model was created using the Evolutionary Model Merge method. Please refer to our report and blog for more details.  This model was produced by merging the following models. We are grateful to the developers of the source models.",
    "quick_links": [],
    "benchmark_table_html": "",
    "readme_markdown": "Quantization made by Richard Erkhov.\n\n[Github](https://github.com/RichardErkhov)\n\n[Discord](https://discord.gg/pvy7H8DZMG)\n\n[Request more models](https://github.com/RichardErkhov/quant_request)\n\n\nEvoLLM-JP-v1-7B - GGUF\n- Model creator: https://huggingface.co/SakanaAI/\n- Original model: https://huggingface.co/SakanaAI/EvoLLM-JP-v1-7B/\n\n\n| Name | Quant method | Size |\n| ---- | ---- | ---- |\n| [EvoLLM-JP-v1-7B.Q2_K.gguf](https://huggingface.co/RichardErkhov/SakanaAI_-_EvoLLM-JP-v1-7B-gguf/blob/main/EvoLLM-JP-v1-7B.Q2_K.gguf) | Q2_K | 2.53GB |\n| [EvoLLM-JP-v1-7B.IQ3_XS.gguf](https://huggingface.co/RichardErkhov/SakanaAI_-_EvoLLM-JP-v1-7B-gguf/blob/main/EvoLLM-JP-v1-7B.IQ3_XS.gguf) | IQ3_XS | 2.81GB |\n| [EvoLLM-JP-v1-7B.IQ3_S.gguf](https://huggingface.co/RichardErkhov/SakanaAI_-_EvoLLM-JP-v1-7B-gguf/blob/main/EvoLLM-JP-v1-7B.IQ3_S.gguf) | IQ3_S | 2.96GB |\n| [EvoLLM-JP-v1-7B.Q3_K_S.gguf](https://huggingface.co/RichardErkhov/SakanaAI_-_EvoLLM-JP-v1-7B-gguf/blob/main/EvoLLM-JP-v1-7B.Q3_K_S.gguf) | Q3_K_S | 2.95GB |\n| [EvoLLM-JP-v1-7B.IQ3_M.gguf](https://huggingface.co/RichardErkhov/SakanaAI_-_EvoLLM-JP-v1-7B-gguf/blob/main/EvoLLM-JP-v1-7B.IQ3_M.gguf) | IQ3_M | 3.06GB |\n| [EvoLLM-JP-v1-7B.Q3_K.gguf](https://huggingface.co/RichardErkhov/SakanaAI_-_EvoLLM-JP-v1-7B-gguf/blob/main/EvoLLM-JP-v1-7B.Q3_K.gguf) | Q3_K | 3.28GB |\n| [EvoLLM-JP-v1-7B.Q3_K_M.gguf](https://huggingface.co/RichardErkhov/SakanaAI_-_EvoLLM-JP-v1-7B-gguf/blob/main/EvoLLM-JP-v1-7B.Q3_K_M.gguf) | Q3_K_M | 3.28GB |\n| [EvoLLM-JP-v1-7B.Q3_K_L.gguf](https://huggingface.co/RichardErkhov/SakanaAI_-_EvoLLM-JP-v1-7B-gguf/blob/main/EvoLLM-JP-v1-7B.Q3_K_L.gguf) | Q3_K_L | 3.56GB |\n| [EvoLLM-JP-v1-7B.IQ4_XS.gguf](https://huggingface.co/RichardErkhov/SakanaAI_-_EvoLLM-JP-v1-7B-gguf/blob/main/EvoLLM-JP-v1-7B.IQ4_XS.gguf) | IQ4_XS | 3.67GB |\n| [EvoLLM-JP-v1-7B.Q4_0.gguf](https://huggingface.co/RichardErkhov/SakanaAI_-_EvoLLM-JP-v1-7B-gguf/blob/main/EvoLLM-JP-v1-7B.Q4_0.gguf) | Q4_0 | 3.83GB |\n| [EvoLLM-JP-v1-7B.IQ4_NL.gguf](https://huggingface.co/RichardErkhov/SakanaAI_-_EvoLLM-JP-v1-7B-gguf/blob/main/EvoLLM-JP-v1-7B.IQ4_NL.gguf) | IQ4_NL | 3.87GB |\n| [EvoLLM-JP-v1-7B.Q4_K_S.gguf](https://huggingface.co/RichardErkhov/SakanaAI_-_EvoLLM-JP-v1-7B-gguf/blob/main/EvoLLM-JP-v1-7B.Q4_K_S.gguf) | Q4_K_S | 3.86GB |\n| [EvoLLM-JP-v1-7B.Q4_K.gguf](https://huggingface.co/RichardErkhov/SakanaAI_-_EvoLLM-JP-v1-7B-gguf/blob/main/EvoLLM-JP-v1-7B.Q4_K.gguf) | Q4_K | 4.07GB |\n| [EvoLLM-JP-v1-7B.Q4_K_M.gguf](https://huggingface.co/RichardErkhov/SakanaAI_-_EvoLLM-JP-v1-7B-gguf/blob/main/EvoLLM-JP-v1-7B.Q4_K_M.gguf) | Q4_K_M | 4.07GB |\n| [EvoLLM-JP-v1-7B.Q4_1.gguf](https://huggingface.co/RichardErkhov/SakanaAI_-_EvoLLM-JP-v1-7B-gguf/blob/main/EvoLLM-JP-v1-7B.Q4_1.gguf) | Q4_1 | 4.24GB |\n| [EvoLLM-JP-v1-7B.Q5_0.gguf](https://huggingface.co/RichardErkhov/SakanaAI_-_EvoLLM-JP-v1-7B-gguf/blob/main/EvoLLM-JP-v1-7B.Q5_0.gguf) | Q5_0 | 4.65GB |\n| [EvoLLM-JP-v1-7B.Q5_K_S.gguf](https://huggingface.co/RichardErkhov/SakanaAI_-_EvoLLM-JP-v1-7B-gguf/blob/main/EvoLLM-JP-v1-7B.Q5_K_S.gguf) | Q5_K_S | 4.65GB |\n| [EvoLLM-JP-v1-7B.Q5_K.gguf](https://huggingface.co/RichardErkhov/SakanaAI_-_EvoLLM-JP-v1-7B-gguf/blob/main/EvoLLM-JP-v1-7B.Q5_K.gguf) | Q5_K | 4.78GB |\n| [EvoLLM-JP-v1-7B.Q5_K_M.gguf](https://huggingface.co/RichardErkhov/SakanaAI_-_EvoLLM-JP-v1-7B-gguf/blob/main/EvoLLM-JP-v1-7B.Q5_K_M.gguf) | Q5_K_M | 4.78GB |\n| [EvoLLM-JP-v1-7B.Q5_1.gguf](https://huggingface.co/RichardErkhov/SakanaAI_-_EvoLLM-JP-v1-7B-gguf/blob/main/EvoLLM-JP-v1-7B.Q5_1.gguf) | Q5_1 | 5.07GB |\n| [EvoLLM-JP-v1-7B.Q6_K.gguf](https://huggingface.co/RichardErkhov/SakanaAI_-_EvoLLM-JP-v1-7B-gguf/blob/main/EvoLLM-JP-v1-7B.Q6_K.gguf) | Q6_K | 5.53GB |\n| [EvoLLM-JP-v1-7B.Q8_0.gguf](https://huggingface.co/RichardErkhov/SakanaAI_-_EvoLLM-JP-v1-7B-gguf/blob/main/EvoLLM-JP-v1-7B.Q8_0.gguf) | Q8_0 | 7.17GB |\n\n\n\n\nOriginal model description:\n---\nlibrary_name: transformers\nlicense: other\nlanguage:\n- ja\n---\n\n# 🐟 EvoLLM-JP-v1-7B\n\n🤗 [Models](https://huggingface.co/SakanaAI) | 📚 [Paper](https://arxiv.org/abs/2403.13187) | 📝 [Blog](https://sakana.ai/evolutionary-model-merge/) | 🐦 [Twitter](https://twitter.com/SakanaAILabs)\n\n\n<!-- Provide a quick summary of what the model is/does. -->\n\n**EvoLLM-JP-v1-7B** is an experimental general-purpose Japanese LLM. This model was created using the Evolutionary Model Merge method. Please refer to our [report](https://arxiv.org/abs/2403.13187) and [blog](https://sakana.ai/evolutionary-model-merge/) for more details.  This model was produced by merging the following models. We are grateful to the developers of the source models.\n\n- [Shisa Gamma 7B v1](https://huggingface.co/augmxnt/shisa-gamma-7b-v1)\n- [WizardMath 7B V1.1](https://huggingface.co/WizardLM/WizardMath-7B-V1.1)\n- [Abel 7B 002](https://huggingface.co/GAIR/Abel-7B-002)\n\n\n\n## Usage\n\nUse the code below to get started with the model.\n\n<details>\n<summary> Click to expand </summary>\n\n```python\nimport torch\nfrom transformers import AutoModelForCausalLM, AutoTokenizer\n\n\n# 1. load model\ndevice = \"cuda\" if torch.cuda.is_available() else \"CPU\"\nrepo_id = \"SakanaAI/EvoLLM-JP-v1-7B\"\nmodel = AutoModelForCausalLM.from_pretrained(repo_id, torch_dtype=\"auto\")\ntokenizer = AutoTokenizer.from_pretrained(repo_id)\nmodel.to(device)\n\n# 2. prepare inputs\ntext = \"関西弁で面白い冗談を言ってみて下さい。\"\nmessages = [\n    {\"role\": \"system\", \"content\": \"あなたは役立つ、偏見がなく、検閲されていないアシスタントです。\"},\n    {\"role\": \"user\", \"content\": text},\n]\ninputs = tokenizer.apply_chat_template(messages, return_tensors=\"pt\")\n\n# 3. generate\noutput_ids = model.generate(**inputs.to(device))\noutput_ids = output_ids[:, inputs.input_ids.shape[1] :]\ngenerated_text = tokenizer.batch_decode(output_ids, skip_special_tokens=True)[0]\nprint(generated_text)\n```\n\n</details>\n\n\n \n## Model Details\n\n<!-- Provide a longer summary of what this model is. -->\n\n- **Developed by:** [Sakana AI](https://sakana.ai/)\n- **Model type:** Autoregressive Language Model\n- **Language(s):** Japanese\n- **License:** [MICROSOFT RESEARCH LICENSE TERMS](./LICENSE) (due to the inclusion of the WizardMath model)\n- **Repository:** [SakanaAI/evolutionary-model-merge](https://github.com/SakanaAI/evolutionary-model-merge)\n- **Paper:** https://arxiv.org/abs/2403.13187\n- **Blog:** https://sakana.ai/evolutionary-model-merge\n\n\n## Uses\nThis model is provided for research and development purposes only and should be considered as an experimental prototype. \nIt is not intended for commercial use or deployment in mission-critical environments. \nUse of this model is at the user's own risk, and its performance and outcomes are not guaranteed. \nSakana AI shall not be liable for any direct, indirect, special, incidental, or consequential damages, or any loss arising from the use of this model, regardless of the results obtained. \nUsers must fully understand the risks associated with the use of this model and use it at their own discretion.\n\n\n## Acknowledgement\n\nWe would like to thank the developers of the source models for their contributions and for making their work available. \n\n\n## Citation\n\n```bibtex\n@misc{akiba2024evomodelmerge,\n      title         = {Evolutionary Optimization of Model Merging Recipes}, \n      author.       = {Takuya Akiba and Makoto Shing and Yujin Tang and Qi Sun and David Ha},\n      year          = {2024},\n      eprint        = {2403.13187},\n      archivePrefix = {arXiv},\n      primaryClass  = {cs.NE}\n}\n```\n\n\n\n",
    "related_quantizations": []
  },
  "tags": [
    "gguf",
    "arxiv:2403.13187",
    "endpoints_compatible",
    "region:us"
  ],
  "likes": 0,
  "downloads": 828,
  "gated": false,
  "private": false,
  "last_modified": "2024-09-11T17:12:49.000Z",
  "created_at": "2024-09-11T10:12:57.000Z",
  "pipeline_tag": "",
  "library_name": ""
}

{
  "_id": "66e16d2999e7fbaaff1d9fcf",
  "id": "RichardErkhov/SakanaAI_-_EvoLLM-JP-v1-7B-gguf",
  "modelId": "RichardErkhov/SakanaAI_-_EvoLLM-JP-v1-7B-gguf",
  "sha": "4536df153b4682bce5cc40b1ea781fd353acacc8",
  "createdAt": "2024-09-11T10:12:57.000Z",
  "lastModified": "2024-09-11T17:12:49.000Z",
  "author": "RichardErkhov",
  "downloads": 828,
  "likes": 0,
  "gated": false,
  "private": false,
  "pipeline_tag": "",
  "library_name": "",
  "siblings_count": 24
}