richarderkhov/pfnet_-_nekomata-7b-pfn-qfin-gguf overview

{
  "metadata": {},
  "card_data": {
    "frontmatter": {},
    "hero_image_url": "",
    "summary": "",
    "quick_links": [],
    "benchmark_table_html": "",
    "readme_markdown": "Quantization made by Richard Erkhov.\n\n[Github](https://github.com/RichardErkhov)\n\n[Discord](https://discord.gg/pvy7H8DZMG)\n\n[Request more models](https://github.com/RichardErkhov/quant_request)\n\n\nnekomata-7b-pfn-qfin - GGUF\n- Model creator: https://huggingface.co/pfnet/\n- Original model: https://huggingface.co/pfnet/nekomata-7b-pfn-qfin/\n\n\n| Name | Quant method | Size |\n| ---- | ---- | ---- |\n| [nekomata-7b-pfn-qfin.Q2_K.gguf](https://huggingface.co/RichardErkhov/pfnet_-_nekomata-7b-pfn-qfin-gguf/blob/main/nekomata-7b-pfn-qfin.Q2_K.gguf) | Q2_K | 2.84GB |\n| [nekomata-7b-pfn-qfin.IQ3_XS.gguf](https://huggingface.co/RichardErkhov/pfnet_-_nekomata-7b-pfn-qfin-gguf/blob/main/nekomata-7b-pfn-qfin.IQ3_XS.gguf) | IQ3_XS | 3.23GB |\n| [nekomata-7b-pfn-qfin.IQ3_S.gguf](https://huggingface.co/RichardErkhov/pfnet_-_nekomata-7b-pfn-qfin-gguf/blob/main/nekomata-7b-pfn-qfin.IQ3_S.gguf) | IQ3_S | 3.32GB |\n| [nekomata-7b-pfn-qfin.Q3_K_S.gguf](https://huggingface.co/RichardErkhov/pfnet_-_nekomata-7b-pfn-qfin-gguf/blob/main/nekomata-7b-pfn-qfin.Q3_K_S.gguf) | Q3_K_S | 3.32GB |\n| [nekomata-7b-pfn-qfin.IQ3_M.gguf](https://huggingface.co/RichardErkhov/pfnet_-_nekomata-7b-pfn-qfin-gguf/blob/main/nekomata-7b-pfn-qfin.IQ3_M.gguf) | IQ3_M | 3.61GB |\n| [nekomata-7b-pfn-qfin.Q3_K.gguf](https://huggingface.co/RichardErkhov/pfnet_-_nekomata-7b-pfn-qfin-gguf/blob/main/nekomata-7b-pfn-qfin.Q3_K.gguf) | Q3_K | 3.78GB |\n| [nekomata-7b-pfn-qfin.Q3_K_M.gguf](https://huggingface.co/RichardErkhov/pfnet_-_nekomata-7b-pfn-qfin-gguf/blob/main/nekomata-7b-pfn-qfin.Q3_K_M.gguf) | Q3_K_M | 3.78GB |\n| [nekomata-7b-pfn-qfin.Q3_K_L.gguf](https://huggingface.co/RichardErkhov/pfnet_-_nekomata-7b-pfn-qfin-gguf/blob/main/nekomata-7b-pfn-qfin.Q3_K_L.gguf) | Q3_K_L | 4.0GB |\n| [nekomata-7b-pfn-qfin.IQ4_XS.gguf](https://huggingface.co/RichardErkhov/pfnet_-_nekomata-7b-pfn-qfin-gguf/blob/main/nekomata-7b-pfn-qfin.IQ4_XS.gguf) | IQ4_XS | 4.02GB |\n| [nekomata-7b-pfn-qfin.Q4_0.gguf](https://huggingface.co/RichardErkhov/pfnet_-_nekomata-7b-pfn-qfin-gguf/blob/main/nekomata-7b-pfn-qfin.Q4_0.gguf) | Q4_0 | 4.2GB |\n| [nekomata-7b-pfn-qfin.IQ4_NL.gguf](https://huggingface.co/RichardErkhov/pfnet_-_nekomata-7b-pfn-qfin-gguf/blob/main/nekomata-7b-pfn-qfin.IQ4_NL.gguf) | IQ4_NL | 4.22GB |\n| [nekomata-7b-pfn-qfin.Q4_K_S.gguf](https://huggingface.co/RichardErkhov/pfnet_-_nekomata-7b-pfn-qfin-gguf/blob/main/nekomata-7b-pfn-qfin.Q4_K_S.gguf) | Q4_K_S | 4.22GB |\n| [nekomata-7b-pfn-qfin.Q4_K.gguf](https://huggingface.co/RichardErkhov/pfnet_-_nekomata-7b-pfn-qfin-gguf/blob/main/nekomata-7b-pfn-qfin.Q4_K.gguf) | Q4_K | 4.56GB |\n| [nekomata-7b-pfn-qfin.Q4_K_M.gguf](https://huggingface.co/RichardErkhov/pfnet_-_nekomata-7b-pfn-qfin-gguf/blob/main/nekomata-7b-pfn-qfin.Q4_K_M.gguf) | Q4_K_M | 4.56GB |\n| [nekomata-7b-pfn-qfin.Q4_1.gguf](https://huggingface.co/RichardErkhov/pfnet_-_nekomata-7b-pfn-qfin-gguf/blob/main/nekomata-7b-pfn-qfin.Q4_1.gguf) | Q4_1 | 4.62GB |\n| [nekomata-7b-pfn-qfin.Q5_0.gguf](https://huggingface.co/RichardErkhov/pfnet_-_nekomata-7b-pfn-qfin-gguf/blob/main/nekomata-7b-pfn-qfin.Q5_0.gguf) | Q5_0 | 5.03GB |\n| [nekomata-7b-pfn-qfin.Q5_K_S.gguf](https://huggingface.co/RichardErkhov/pfnet_-_nekomata-7b-pfn-qfin-gguf/blob/main/nekomata-7b-pfn-qfin.Q5_K_S.gguf) | Q5_K_S | 5.03GB |\n| [nekomata-7b-pfn-qfin.Q5_K.gguf](https://huggingface.co/RichardErkhov/pfnet_-_nekomata-7b-pfn-qfin-gguf/blob/main/nekomata-7b-pfn-qfin.Q5_K.gguf) | Q5_K | 5.32GB |\n| [nekomata-7b-pfn-qfin.Q5_K_M.gguf](https://huggingface.co/RichardErkhov/pfnet_-_nekomata-7b-pfn-qfin-gguf/blob/main/nekomata-7b-pfn-qfin.Q5_K_M.gguf) | Q5_K_M | 5.32GB |\n| [nekomata-7b-pfn-qfin.Q5_1.gguf](https://huggingface.co/RichardErkhov/pfnet_-_nekomata-7b-pfn-qfin-gguf/blob/main/nekomata-7b-pfn-qfin.Q5_1.gguf) | Q5_1 | 5.44GB |\n| [nekomata-7b-pfn-qfin.Q6_K.gguf](https://huggingface.co/RichardErkhov/pfnet_-_nekomata-7b-pfn-qfin-gguf/blob/main/nekomata-7b-pfn-qfin.Q6_K.gguf) | Q6_K | 5.91GB |\n| [nekomata-7b-pfn-qfin.Q8_0.gguf](https://huggingface.co/RichardErkhov/pfnet_-_nekomata-7b-pfn-qfin-gguf/blob/main/nekomata-7b-pfn-qfin.Q8_0.gguf) | Q8_0 | 7.65GB |\n\n\n\n\nOriginal model description:\n---\nlicense: other\nlicense_name: tongyi-qianwen-license\nlicense_link: LICENSE\nlanguage:\n- en\n- ja\nlibrary_name: transformers\npipeline_tag: text-generation\n---\n\n# nekomata-7b-pfn-qfin\n\n## Model Description\nnekomata-7b-pfn-qfin is a fine-tuned model based on [rinna/nekomata-7b](https://huggingface.co/rinna/nekomata-7b/tree/main).\nThis is the base model, which is good at generating continuous sentences for finance.\nnekomata-7b-pfn-qfin is fine-tuned on 370M tokens from multiple special datasets generated by Preferred Networks, which is clear to use for commercial usage.\nThe fine-tuned were carried out at a 2048 context length.\nThis model is released under [Tongyi Qianwen LICENSE AGREEMENT](https://github.com/QwenLM/Qwen/blob/e8e15962d897714944773cca57fa2e460a3655e8/Tongyi%20Qianwen%20LICENSE%20AGREEMENT).\n\nThe research article is available on [arXiv](https://arxiv.org/abs/2404.10555).\n\n# Benchmarking\nThe benchmark score is obtained using [Japanese Language Model Financial Evaluation Harness](https://github.com/pfnet-research/japanese-lm-fin-harness)\nFor the benchmark, 0-shot and default prompts are used.\n```\n|      Task      |Metric|  nekomaba-7b   |       Ours      |\n|----------------|------|------|---|------|------|---|------|\n|chabsa          |f1    |0.8134|   |      |0.8127|   |      |\n|cma_basics      |acc   |0.3158|±  |0.0764|0.3684|±  |0.0793|\n|cpa_audit       |acc   |0.2085|±  |0.0203|0.1809|±  |0.0193|\n|fp2             |acc   |0.2484|±  |0.0198|0.2674|±  |0.0203|\n|security_sales_1|acc   |0.4912|±  |0.0668|0.5088|±  |0.0668|\n|----------------|------|------|---|------|------|---|------|\n|OVER ALL        |      |0.4155           |0.4276           |\n```\n## Usage\nInstall the required libraries as follows:\n```sh\n>>> python -m pip install numpy sentencepiece torch transformers accelerate transformers_stream_generator tiktoken einops\n```\n\nExecute the following python code:\n```python\nimport torch\nfrom transformers import AutoTokenizer, AutoModelForCausalLM\n\ntokenizer = AutoTokenizer.from_pretrained(\"pfnet/nekomata-7b-pfn-qfin\", trust_remote_code=True)\n\n# Use GPU with bf16 (recommended for supported devices)\n# model = AutoModelForCausalLM.from_pretrained(\"pfnet/nekomata-7b-pfn-qfin\", device_map=\"auto\", trust_remote_code=True, bf16=True)\n\n# Use GPU with fp16\n# model = AutoModelForCausalLM.from_pretrained(\"pfnet/nekomata-7b-pfn-qfin\", device_map=\"auto\", trust_remote_code=True, fp16=True)\n\n# Use GPU with fp32\n# model = AutoModelForCausalLM.from_pretrained(\"pfnet/nekomata-7b-pfn-qfin\", device_map=\"auto\", trust_remote_code=True, fp32=True)\n\n# Use CPU\n# model = AutoModelForCausalLM.from_pretrained(\"pfnet/nekomata-7b-pfn-qfin\", device_map=\"cpu\", trust_remote_code=True)\n\n# Automatically select device and precision\nmodel = AutoModelForCausalLM.from_pretrained(\"pfnet/nekomata-7b-pfn-qfin\", device_map=\"auto\", trust_remote_code=True)\n\ntext = \"日本銀行は\"\ninput_ids = tokenizer(text, return_tensors=\"pt\").input_ids.to(model.device)\nwith torch.no_grad():\n  generated_tokens = model.generate(\n      inputs=input_ids,\n      max_new_tokens=32,\n      do_sample=True,\n      temperature=1.0,\n      repetition_penalty=1.1\n  )[0]\ngenerated_text = tokenizer.decode(generated_tokens)\nprint(generated_text)\n# 日本銀行は、2016年9月に「長短金利操作付き量的・質的金融緩和」を導入し、長期国\n```\n\n## Model Details\n- Model size: 7b\n- Fine-tuned tokens: 370M tokens (Japanese: 300M tokens, English: 13M tokens, Digits: 14M tokens)\n- Context length: 2048\n- Developed by: Preferred Networks, Inc\n- Model type: Causal decoder-only\n- Language(s): Japanese and English\n- License: [Tongyi Qianwen LICENSE AGREEMENT](https://github.com/QwenLM/Qwen/blob/e8e15962d897714944773cca57fa2e460a3655e8/Tongyi%20Qianwen%20LICENSE%20AGREEMENT)\n\n## Bias, Risks, and Limitations\nnekomata-7b-pfn-qfin is a new technology that carries risks with use.\nTesting conducted to date has been in English and Japanese, and has not covered, nor could it cover all scenarios.\nFor these reasons, as with all LLMs, nekomata-7b-pfn-qfin’s potential outputs cannot be predicted in advance, and the model may in some instances produce inaccurate, biased or other objectionable responses to user prompts.\nThis model is not designed for legal, tax, investment, financial, or other advice.\nTherefore, before deploying any applications of nekomata-7b-pfn-qfin, developers should perform safety testing and tuning tailored to their specific applications of the model.\n\n## How to cite\n```\n@misc{hirano2024,\n      title={Construction of Domain-specified Japanese Large Language Model for Finance through Continual Pre-training}, \n      author={Masanori Hirano and Kentaro Imajo},\n      year={2024},\n      eprint={2404.10555},\n      archivePrefix={arXiv},\n      primaryClass={cs.CL}\n}\n```\n\n## Contributors\nPreferred Networks, Inc.\n - Masanori Hirano\n - Kentaro Imajo\n\n# License\n[Tongyi Qianwen LICENSE AGREEMENT](https://github.com/QwenLM/Qwen/blob/e8e15962d897714944773cca57fa2e460a3655e8/Tongyi%20Qianwen%20LICENSE%20AGREEMENT)\n\n\n",
    "related_quantizations": []
  },
  "tags": [
    "gguf",
    "arxiv:2404.10555",
    "endpoints_compatible",
    "region:us"
  ],
  "likes": 0,
  "downloads": 469,
  "gated": false,
  "private": false,
  "last_modified": "2024-09-21T01:32:23.000Z",
  "created_at": "2024-09-20T19:26:07.000Z",
  "pipeline_tag": "",
  "library_name": ""
}

{
  "_id": "66edcc4ffca6148a45565cf5",
  "id": "RichardErkhov/pfnet_-_nekomata-7b-pfn-qfin-gguf",
  "modelId": "RichardErkhov/pfnet_-_nekomata-7b-pfn-qfin-gguf",
  "sha": "5d1d1a1719914593bdafb9151bc0b698146119f7",
  "createdAt": "2024-09-20T19:26:07.000Z",
  "lastModified": "2024-09-21T01:32:23.000Z",
  "author": "RichardErkhov",
  "downloads": 469,
  "likes": 0,
  "gated": false,
  "private": false,
  "pipeline_tag": "",
  "library_name": "",
  "siblings_count": 24
}