richarderkhov/rinna_-_nekomata-14b-instruction-gguf overview

{
  "metadata": {},
  "card_data": {
    "frontmatter": {},
    "hero_image_url": "./rinna.png",
    "summary": "!rinna-icon # Overview The model is the instruction-tuned version of rinna/nekomata-14b. It adopts the Alpaca input format. * **Model architecture** A 40-layer, 5120-hidden-size transformer-based language model. Please refer to the Qwen paper for architecture details. * **Fine-tuning** The fine-tuning data is the subset of the following datasets. * Databricks Dolly data * Japanese Databricks Dolly data * FLAN Instruction Tuning data and its Japanese translation * Izumi lab LLM Japanese dataset * The following sections are used * alt * aozora-txt * CourseraParallel * ParaNatCom * Tab-delimited_Bilingual_Sentence_Pairs * tanaka-corpus * wikinews * wordnet * yasashi-japanese * The remaining sections contain commonly used evaluation corpora so they are skipped to prevent data leak. * **Contributors** --- # Benchmarking Please refer to rinna's LM benchmark page. --- # How to use the model ~~~~python import torch from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained(\"rinna/nekomata-14b-instruction\", trust_remote_code=True) # Use GPU with bf16 # model = AutoModelForCausalLM.from_pretrained(\"rinna/nekomata-14b-instruction\", device_map=\"auto\", trust_remote_code=True, bf16=True) # Use GPU with fp16 # model = AutoModelForCausalLM.from_pretrained(\"rinna/nekomata-14b-instruction\", device_map=\"auto\", trust_remote_code=True, fp16=True) # Use CPU # model = AutoModelForCausalLM.from_pretrained(\"rinna/nekomata-14b-instruction\", device_map=\"cpu\", trust_remote_code=True) # Automatically select device and precision model = AutoModelForCausalLM.from_pretrained(\"rinna/nekomata-14b-instruction\", device_map=\"auto\", trust_remote_code=True) instruction = \"次の日本語を英語に翻訳してください。\" input = \"大規模言語モデル（だいきぼげんごモデル、英: large language model、LLM）は、多数のパラメータ（数千万から数十億）を持つ人工ニューラルネットワークで構成されるコンピュータ言語モデルで、膨大なラベルなしテキストを使用して自己教師あり学習または半教師あり学習によって訓練が行われる。\" prompt = f\"\"\" 以下は、タスクを説明する指示と、文脈のある入力の組み合わせです。要求を適切に満たす応答を書きなさい。 ### 指示: {instruction} ### 入力: {input} ### 応答: \"\"\" token_ids = tokenizer.encode(prompt, add_special_tokens=False, return_tensors=\"pt\") with torch.no_grad(): output_ids = model.generate( token_ids.to(model.device), max_new_tokens=200, do_sample=True, temperature=0.5, pad_token_id=tokenizer.pad_token_id, bos_token_id=tokenizer.bos_token_id, eos_token_id=tokenizer.eos_token_id ) output = tokenizer.decode(output_ids.tolist()[0]) print(output) \"\"\" 以下は、タスクを説明する指示と、文脈のある入力の組み合わせです。要求を適切に満たす応答を書きなさい。 ### 指示: 次の日本語を英語に翻訳してください。 ### 入力: 大規模言語モデル（だいきぼげんごモデル、英: large language model、LLM）は、多数のパラメータ（数千万から数十億）を持つ人工ニューラルネットワークで構成されるコンピュータ言語モデルで、膨大なラベルなしテキストを使 用して自己教師あり学習または半教師あり学習によって訓練が行われる。 ### 応答: A large language model (LLM) is a computer language model composed of artificial neural networks with many parameters (from tens of millions to billions) trained by self-supervised learning or semi-supervised learning using a large amount of unlabeled text. \"\"\" ~~~~ --- # Tokenization Please refer to rinna/nekomata-14b for tokenization details. --- # How to cite ``bibtex @misc{rinna-nekomata-14b-instruction, title = {rinna/nekomata-14b-instruction}, author = {Zhao, Tianyu and Sawada, Kei}, url = {https://huggingface.co/rinna/nekomata-14b-instruction} } @inproceedings{sawada2024release, title = {Release of Pre-Trained Models for the {J}apanese Language}, author = {Sawada, Kei and Zhao, Tianyu and Shing, Makoto and Mitsui, Kentaro and Kaga, Akio and Hono, Yukiya and Wakatsuki, Toshiaki and Mitsuda, Koh}, booktitle = {Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)}, month = {5}, year = {2024}, pages = {13898--13905}, url = {https://aclanthology.org/2024.lrec-main.1213}, note = {\\url{https://arxiv.org/abs/2404.01657}} } `` --- # License Tongyi Qianwen LICENSE AGREEMENT",
    "quick_links": [],
    "benchmark_table_html": "",
    "readme_markdown": "Quantization made by Richard Erkhov.\n\n[Github](https://github.com/RichardErkhov)\n\n[Discord](https://discord.gg/pvy7H8DZMG)\n\n[Request more models](https://github.com/RichardErkhov/quant_request)\n\n\nnekomata-14b-instruction - GGUF\n- Model creator: https://huggingface.co/rinna/\n- Original model: https://huggingface.co/rinna/nekomata-14b-instruction/\n\n\n| Name | Quant method | Size |\n| ---- | ---- | ---- |\n| [nekomata-14b-instruction.Q2_K.gguf](https://huggingface.co/RichardErkhov/rinna_-_nekomata-14b-instruction-gguf/blob/main/nekomata-14b-instruction.Q2_K.gguf) | Q2_K | 5.41GB |\n| [nekomata-14b-instruction.IQ3_XS.gguf](https://huggingface.co/RichardErkhov/rinna_-_nekomata-14b-instruction-gguf/blob/main/nekomata-14b-instruction.IQ3_XS.gguf) | IQ3_XS | 6.12GB |\n| [nekomata-14b-instruction.IQ3_S.gguf](https://huggingface.co/RichardErkhov/rinna_-_nekomata-14b-instruction-gguf/blob/main/nekomata-14b-instruction.IQ3_S.gguf) | IQ3_S | 6.31GB |\n| [nekomata-14b-instruction.Q3_K_S.gguf](https://huggingface.co/RichardErkhov/rinna_-_nekomata-14b-instruction-gguf/blob/main/nekomata-14b-instruction.Q3_K_S.gguf) | Q3_K_S | 6.31GB |\n| [nekomata-14b-instruction.IQ3_M.gguf](https://huggingface.co/RichardErkhov/rinna_-_nekomata-14b-instruction-gguf/blob/main/nekomata-14b-instruction.IQ3_M.gguf) | IQ3_M | 6.87GB |\n| [nekomata-14b-instruction.Q3_K.gguf](https://huggingface.co/RichardErkhov/rinna_-_nekomata-14b-instruction-gguf/blob/main/nekomata-14b-instruction.Q3_K.gguf) | Q3_K | 7.16GB |\n| [nekomata-14b-instruction.Q3_K_M.gguf](https://huggingface.co/RichardErkhov/rinna_-_nekomata-14b-instruction-gguf/blob/main/nekomata-14b-instruction.Q3_K_M.gguf) | Q3_K_M | 7.16GB |\n| [nekomata-14b-instruction.Q3_K_L.gguf](https://huggingface.co/RichardErkhov/rinna_-_nekomata-14b-instruction-gguf/blob/main/nekomata-14b-instruction.Q3_K_L.gguf) | Q3_K_L | 7.44GB |\n| [nekomata-14b-instruction.IQ4_XS.gguf](https://huggingface.co/RichardErkhov/rinna_-_nekomata-14b-instruction-gguf/blob/main/nekomata-14b-instruction.IQ4_XS.gguf) | IQ4_XS | 7.37GB |\n| [nekomata-14b-instruction.Q4_0.gguf](https://huggingface.co/RichardErkhov/rinna_-_nekomata-14b-instruction-gguf/blob/main/nekomata-14b-instruction.Q4_0.gguf) | Q4_0 | 7.62GB |\n| [nekomata-14b-instruction.IQ4_NL.gguf](https://huggingface.co/RichardErkhov/rinna_-_nekomata-14b-instruction-gguf/blob/main/nekomata-14b-instruction.IQ4_NL.gguf) | IQ4_NL | 7.68GB |\n| [nekomata-14b-instruction.Q4_K_S.gguf](https://huggingface.co/RichardErkhov/rinna_-_nekomata-14b-instruction-gguf/blob/main/nekomata-14b-instruction.Q4_K_S.gguf) | Q4_K_S | 7.96GB |\n| [nekomata-14b-instruction.Q4_K.gguf](https://huggingface.co/RichardErkhov/rinna_-_nekomata-14b-instruction-gguf/blob/main/nekomata-14b-instruction.Q4_K.gguf) | Q4_K | 8.8GB |\n| [nekomata-14b-instruction.Q4_K_M.gguf](https://huggingface.co/RichardErkhov/rinna_-_nekomata-14b-instruction-gguf/blob/main/nekomata-14b-instruction.Q4_K_M.gguf) | Q4_K_M | 8.8GB |\n| [nekomata-14b-instruction.Q4_1.gguf](https://huggingface.co/RichardErkhov/rinna_-_nekomata-14b-instruction-gguf/blob/main/nekomata-14b-instruction.Q4_1.gguf) | Q4_1 | 8.4GB |\n| [nekomata-14b-instruction.Q5_0.gguf](https://huggingface.co/RichardErkhov/rinna_-_nekomata-14b-instruction-gguf/blob/main/nekomata-14b-instruction.Q5_0.gguf) | Q5_0 | 9.18GB |\n| [nekomata-14b-instruction.Q5_K_S.gguf](https://huggingface.co/RichardErkhov/rinna_-_nekomata-14b-instruction-gguf/blob/main/nekomata-14b-instruction.Q5_K_S.gguf) | Q5_K_S | 9.34GB |\n| [nekomata-14b-instruction.Q5_K.gguf](https://huggingface.co/RichardErkhov/rinna_-_nekomata-14b-instruction-gguf/blob/main/nekomata-14b-instruction.Q5_K.gguf) | Q5_K | 2.17GB |\n| [nekomata-14b-instruction.Q5_K_M.gguf](https://huggingface.co/RichardErkhov/rinna_-_nekomata-14b-instruction-gguf/blob/main/nekomata-14b-instruction.Q5_K_M.gguf) | Q5_K_M | 10.14GB |\n| [nekomata-14b-instruction.Q5_1.gguf](https://huggingface.co/RichardErkhov/rinna_-_nekomata-14b-instruction-gguf/blob/main/nekomata-14b-instruction.Q5_1.gguf) | Q5_1 | 9.96GB |\n| [nekomata-14b-instruction.Q6_K.gguf](https://huggingface.co/RichardErkhov/rinna_-_nekomata-14b-instruction-gguf/blob/main/nekomata-14b-instruction.Q6_K.gguf) | Q6_K | 11.46GB |\n| [nekomata-14b-instruction.Q8_0.gguf](https://huggingface.co/RichardErkhov/rinna_-_nekomata-14b-instruction-gguf/blob/main/nekomata-14b-instruction.Q8_0.gguf) | Q8_0 | 14.03GB |\n\n\n\n\nOriginal model description:\n---\nthumbnail: https://github.com/rinnakk/japanese-pretrained-models/blob/master/rinna.png\ndatasets:\n- databricks/databricks-dolly-15k\n- kunishou/databricks-dolly-15k-ja\n- izumi-lab/llm-japanese-dataset\nlanguage:\n- ja\n- en\ntags:\n- qwen\ninference: false\nlicense: other\nlicense_name: tongyi-qianwen-license-agreement\nlicense_link: https://github.com/QwenLM/Qwen/blob/main/Tongyi%20Qianwen%20LICENSE%20AGREEMENT\n---\n\n# `rinna/nekomata-14b-instruction`\n\n![rinna-icon](./rinna.png)\n\n# Overview\nThe model is the instruction-tuned version of [`rinna/nekomata-14b`](https://huggingface.co/rinna/nekomata-14b). It adopts the Alpaca input format.\n\n* **Model architecture**\n\n    A 40-layer, 5120-hidden-size transformer-based language model. Please refer to the [Qwen paper](https://arxiv.org/abs/2309.16609) for architecture details.\n\n* **Fine-tuning**\n    \n    The fine-tuning data is the subset of the following datasets.\n    * [Databricks Dolly data](https://huggingface.co/datasets/databricks/databricks-dolly-15k)\n    * [Japanese Databricks Dolly data](https://huggingface.co/datasets/kunishou/databricks-dolly-15k-ja)\n    * [FLAN Instruction Tuning data](https://github.com/google-research/FLAN) and its Japanese translation\n    * [Izumi lab LLM Japanese dataset](https://github.com/masanorihirano/llm-japanese-dataset/tree/main)\n      * The following sections are used\n        * alt\n        * aozora-txt\n        * CourseraParallel\n        * ParaNatCom\n        * Tab-delimited_Bilingual_Sentence_Pairs\n        * tanaka-corpus\n        * wikinews\n        * wordnet\n        * yasashi-japanese\n      * The [remaining sections](https://github.com/masanorihirano/llm-japanese-dataset/tree/main/datasets-cc-by-sa) contain commonly used evaluation corpora so they are skipped to prevent data leak.\n\n* **Contributors**\n\n    - [Tianyu Zhao](https://huggingface.co/tianyuz)\n    - [Kei Sawada](https://huggingface.co/keisawada)\n    \n---\n\n# Benchmarking\nPlease refer to [rinna's LM benchmark page](https://rinnakk.github.io/research/benchmarks/lm/index.html).\n\n---\n\n# How to use the model\n\n~~~~python\nimport torch\nfrom transformers import AutoTokenizer, AutoModelForCausalLM\n\ntokenizer = AutoTokenizer.from_pretrained(\"rinna/nekomata-14b-instruction\", trust_remote_code=True)\n\n# Use GPU with bf16\n# model = AutoModelForCausalLM.from_pretrained(\"rinna/nekomata-14b-instruction\", device_map=\"auto\", trust_remote_code=True, bf16=True)\n\n# Use GPU with fp16\n# model = AutoModelForCausalLM.from_pretrained(\"rinna/nekomata-14b-instruction\", device_map=\"auto\", trust_remote_code=True, fp16=True)\n\n# Use CPU\n# model = AutoModelForCausalLM.from_pretrained(\"rinna/nekomata-14b-instruction\", device_map=\"cpu\", trust_remote_code=True)\n\n# Automatically select device and precision\nmodel = AutoModelForCausalLM.from_pretrained(\"rinna/nekomata-14b-instruction\", device_map=\"auto\", trust_remote_code=True)\n\ninstruction = \"次の日本語を英語に翻訳してください。\"\ninput = \"大規模言語モデル（だいきぼげんごモデル、英: large language model、LLM）は、多数のパラメータ（数千万から数十億）を持つ人工ニューラルネットワークで構成されるコンピュータ言語モデルで、膨大なラベルなしテキストを使用して自己教師あり学習または半教師あり学習によって訓練が行われる。\"\nprompt = f\"\"\"\n以下は、タスクを説明する指示と、文脈のある入力の組み合わせです。要求を適切に満たす応答を書きなさい。\n\n### 指示:\n{instruction}\n\n### 入力:\n{input}\n\n### 応答:\n\"\"\"\ntoken_ids = tokenizer.encode(prompt, add_special_tokens=False, return_tensors=\"pt\")\n\nwith torch.no_grad():\n    output_ids = model.generate(\n        token_ids.to(model.device),\n        max_new_tokens=200,\n        do_sample=True,\n        temperature=0.5,\n        pad_token_id=tokenizer.pad_token_id,\n        bos_token_id=tokenizer.bos_token_id,\n        eos_token_id=tokenizer.eos_token_id\n    )\n\noutput = tokenizer.decode(output_ids.tolist()[0])\nprint(output)\n\"\"\"\n以下は、タスクを説明する指示と、文脈のある入力の組み合わせです。要求を適切に満たす応答を書きなさい。\n\n### 指示:\n次の日本語を英語に翻訳してください。\n\n### 入力:\n大規模言語モデル（だいきぼげんごモデル、英: large language model、LLM）は、多数のパラメータ（数千万から数十億）を持つ人工ニューラルネットワークで構成されるコンピュータ言語モデルで、膨大なラベルなしテキストを使 用して自己教師あり学習または半教師あり学習によって訓練が行われる。\n\n### 応答:\n A large language model (LLM) is a computer language model composed of artificial neural networks with many parameters (from tens of millions to billions) trained by self-supervised learning or semi-supervised learning using a large amount of unlabeled text.<|endoftext|>\n\"\"\"\n~~~~\n\n---\n\n# Tokenization\nPlease refer to [`rinna/nekomata-14b`](https://huggingface.co/rinna/nekomata-14b) for tokenization details.\n\n---\n\n# How to cite\n```bibtex\n@misc{rinna-nekomata-14b-instruction,\n    title = {rinna/nekomata-14b-instruction},\n    author = {Zhao, Tianyu and Sawada, Kei},\n    url = {https://huggingface.co/rinna/nekomata-14b-instruction}\n}\n\n@inproceedings{sawada2024release,\n    title = {Release of Pre-Trained Models for the {J}apanese Language},\n    author = {Sawada, Kei and Zhao, Tianyu and Shing, Makoto and Mitsui, Kentaro and Kaga, Akio and Hono, Yukiya and Wakatsuki, Toshiaki and Mitsuda, Koh},\n    booktitle = {Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)},\n    month = {5},\n    year = {2024},\n    pages = {13898--13905},\n    url = {https://aclanthology.org/2024.lrec-main.1213},\n    note = {\\url{https://arxiv.org/abs/2404.01657}}\n}\n```\n---\n\n# License\n[Tongyi Qianwen LICENSE AGREEMENT](https://github.com/QwenLM/Qwen/blob/main/Tongyi%20Qianwen%20LICENSE%20AGREEMENT)\n\n",
    "related_quantizations": []
  },
  "tags": [
    "gguf",
    "arxiv:2309.16609",
    "arxiv:2404.01657",
    "endpoints_compatible",
    "region:us"
  ],
  "likes": 0,
  "downloads": 260,
  "gated": false,
  "private": false,
  "last_modified": "2024-08-29T10:59:19.000Z",
  "created_at": "2024-08-29T06:02:34.000Z",
  "pipeline_tag": "",
  "library_name": ""
}

{
  "_id": "66d00efaa0c72503484f5375",
  "id": "RichardErkhov/rinna_-_nekomata-14b-instruction-gguf",
  "modelId": "RichardErkhov/rinna_-_nekomata-14b-instruction-gguf",
  "sha": "c77acee29421f943983dbe4556dabe9710d5d139",
  "createdAt": "2024-08-29T06:02:34.000Z",
  "lastModified": "2024-08-29T10:59:19.000Z",
  "author": "RichardErkhov",
  "downloads": 260,
  "likes": 0,
  "gated": false,
  "private": false,
  "pipeline_tag": "",
  "library_name": "",
  "siblings_count": 24
}