richarderkhov/spow12_-_chatwaifu_v1.3.1-gguf overview

{
  "metadata": {},
  "card_data": {
    "frontmatter": {},
    "hero_image_url": "./cover.png",
    "summary": "!image Merged model using mergekit This model aimed to act like visual novel character.",
    "quick_links": [],
    "benchmark_table_html": "",
    "readme_markdown": "Quantization made by Richard Erkhov.\n\n[Github](https://github.com/RichardErkhov)\n\n[Discord](https://discord.gg/pvy7H8DZMG)\n\n[Request more models](https://github.com/RichardErkhov/quant_request)\n\n\nChatWaifu_v1.3.1 - GGUF\n- Model creator: https://huggingface.co/spow12/\n- Original model: https://huggingface.co/spow12/ChatWaifu_v1.3.1/\n\n\n| Name | Quant method | Size |\n| ---- | ---- | ---- |\n| [ChatWaifu_v1.3.1.Q2_K.gguf](https://huggingface.co/RichardErkhov/spow12_-_ChatWaifu_v1.3.1-gguf/blob/main/ChatWaifu_v1.3.1.Q2_K.gguf) | Q2_K | 4.46GB |\n| [ChatWaifu_v1.3.1.IQ3_XS.gguf](https://huggingface.co/RichardErkhov/spow12_-_ChatWaifu_v1.3.1-gguf/blob/main/ChatWaifu_v1.3.1.IQ3_XS.gguf) | IQ3_XS | 4.94GB |\n| [ChatWaifu_v1.3.1.IQ3_S.gguf](https://huggingface.co/RichardErkhov/spow12_-_ChatWaifu_v1.3.1-gguf/blob/main/ChatWaifu_v1.3.1.IQ3_S.gguf) | IQ3_S | 5.18GB |\n| [ChatWaifu_v1.3.1.Q3_K_S.gguf](https://huggingface.co/RichardErkhov/spow12_-_ChatWaifu_v1.3.1-gguf/blob/main/ChatWaifu_v1.3.1.Q3_K_S.gguf) | Q3_K_S | 5.15GB |\n| [ChatWaifu_v1.3.1.IQ3_M.gguf](https://huggingface.co/RichardErkhov/spow12_-_ChatWaifu_v1.3.1-gguf/blob/main/ChatWaifu_v1.3.1.IQ3_M.gguf) | IQ3_M | 5.33GB |\n| [ChatWaifu_v1.3.1.Q3_K.gguf](https://huggingface.co/RichardErkhov/spow12_-_ChatWaifu_v1.3.1-gguf/blob/main/ChatWaifu_v1.3.1.Q3_K.gguf) | Q3_K | 5.67GB |\n| [ChatWaifu_v1.3.1.Q3_K_M.gguf](https://huggingface.co/RichardErkhov/spow12_-_ChatWaifu_v1.3.1-gguf/blob/main/ChatWaifu_v1.3.1.Q3_K_M.gguf) | Q3_K_M | 5.67GB |\n| [ChatWaifu_v1.3.1.Q3_K_L.gguf](https://huggingface.co/RichardErkhov/spow12_-_ChatWaifu_v1.3.1-gguf/blob/main/ChatWaifu_v1.3.1.Q3_K_L.gguf) | Q3_K_L | 6.11GB |\n| [ChatWaifu_v1.3.1.IQ4_XS.gguf](https://huggingface.co/RichardErkhov/spow12_-_ChatWaifu_v1.3.1-gguf/blob/main/ChatWaifu_v1.3.1.IQ4_XS.gguf) | IQ4_XS | 6.33GB |\n| [ChatWaifu_v1.3.1.Q4_0.gguf](https://huggingface.co/RichardErkhov/spow12_-_ChatWaifu_v1.3.1-gguf/blob/main/ChatWaifu_v1.3.1.Q4_0.gguf) | Q4_0 | 6.59GB |\n| [ChatWaifu_v1.3.1.IQ4_NL.gguf](https://huggingface.co/RichardErkhov/spow12_-_ChatWaifu_v1.3.1-gguf/blob/main/ChatWaifu_v1.3.1.IQ4_NL.gguf) | IQ4_NL | 6.65GB |\n| [ChatWaifu_v1.3.1.Q4_K_S.gguf](https://huggingface.co/RichardErkhov/spow12_-_ChatWaifu_v1.3.1-gguf/blob/main/ChatWaifu_v1.3.1.Q4_K_S.gguf) | Q4_K_S | 6.63GB |\n| [ChatWaifu_v1.3.1.Q4_K.gguf](https://huggingface.co/RichardErkhov/spow12_-_ChatWaifu_v1.3.1-gguf/blob/main/ChatWaifu_v1.3.1.Q4_K.gguf) | Q4_K | 6.96GB |\n| [ChatWaifu_v1.3.1.Q4_K_M.gguf](https://huggingface.co/RichardErkhov/spow12_-_ChatWaifu_v1.3.1-gguf/blob/main/ChatWaifu_v1.3.1.Q4_K_M.gguf) | Q4_K_M | 6.96GB |\n| [ChatWaifu_v1.3.1.Q4_1.gguf](https://huggingface.co/RichardErkhov/spow12_-_ChatWaifu_v1.3.1-gguf/blob/main/ChatWaifu_v1.3.1.Q4_1.gguf) | Q4_1 | 7.26GB |\n| [ChatWaifu_v1.3.1.Q5_0.gguf](https://huggingface.co/RichardErkhov/spow12_-_ChatWaifu_v1.3.1-gguf/blob/main/ChatWaifu_v1.3.1.Q5_0.gguf) | Q5_0 | 7.93GB |\n| [ChatWaifu_v1.3.1.Q5_K_S.gguf](https://huggingface.co/RichardErkhov/spow12_-_ChatWaifu_v1.3.1-gguf/blob/main/ChatWaifu_v1.3.1.Q5_K_S.gguf) | Q5_K_S | 7.93GB |\n| [ChatWaifu_v1.3.1.Q5_K.gguf](https://huggingface.co/RichardErkhov/spow12_-_ChatWaifu_v1.3.1-gguf/blob/main/ChatWaifu_v1.3.1.Q5_K.gguf) | Q5_K | 8.13GB |\n| [ChatWaifu_v1.3.1.Q5_K_M.gguf](https://huggingface.co/RichardErkhov/spow12_-_ChatWaifu_v1.3.1-gguf/blob/main/ChatWaifu_v1.3.1.Q5_K_M.gguf) | Q5_K_M | 8.13GB |\n| [ChatWaifu_v1.3.1.Q5_1.gguf](https://huggingface.co/RichardErkhov/spow12_-_ChatWaifu_v1.3.1-gguf/blob/main/ChatWaifu_v1.3.1.Q5_1.gguf) | Q5_1 | 8.61GB |\n| [ChatWaifu_v1.3.1.Q6_K.gguf](https://huggingface.co/RichardErkhov/spow12_-_ChatWaifu_v1.3.1-gguf/blob/main/ChatWaifu_v1.3.1.Q6_K.gguf) | Q6_K | 9.37GB |\n| [ChatWaifu_v1.3.1.Q8_0.gguf](https://huggingface.co/RichardErkhov/spow12_-_ChatWaifu_v1.3.1-gguf/blob/main/ChatWaifu_v1.3.1.Q8_0.gguf) | Q8_0 | 12.13GB |\n\n\n\n\nOriginal model description:\n---\nbase_model:\n- spow12/ChatWaifu_v1.2\n- mistralai/Mistral-Nemo-Instruct-2407\n- NeverSleep/Lumimaid-v0.2-12B\n- Epiculous/Violet_Twilight-v0.1\nlanguage:\n- en\n- fr\n- de\n- es\n- it\n- pt\n- ru\n- zh\n- ja\nlicense: cc-by-nc-4.0\npipeline_tag: text-generation\ntags:\n- nsfw\n- Visual novel\n- roleplay\n- mergekit\n- merge\nlibrary_name: transformers\n---\n\n# Model Card for Model ID\n\n![image](./cover.png)\n\nMerged model using [mergekit](https://github.com/arcee-ai/mergekit/tree/main/mergekit)\n\nThis model aimed to act like visual novel character.\n\n## Merge Format\n\n```yaml\nmodels:\n  - model: spow12/ChatWaifu_v1.2 # reszie token embedding to original mistral\n    parameters:\n      weight: 0.3\n  - model: mistralai/Mistral-Nemo-Instruct-2407\n    parameters:\n      weight: 0.25\n  - model: NeverSleep/Lumimaid-v0.2-12B # reszie token embedding to original mistral\n    parameters:\n      weight: 0.2\n  - model: Epiculous/Violet_Twilight-v0.1\n    parameters:\n      weight: 0.25\nmerge_method: linear\ndtype: float16\n```\n\nyou have to resize chatwaifu and lucimaid's embedding size(131073 to 131072). \n# WaifuModel Collections \n\n- [TTS](https://huggingface.co/spow12/visual_novel_tts)\n- [Chat](https://huggingface.co/spow12/ChatWaifu_v1.3.1)\n- [ASR](https://huggingface.co/spow12/Visual-novel-transcriptor)\n\n# Unified demo\n\n[WaifuAssistant](https://github.com/yw0nam/WaifuAssistant)\n\n# Update\n- 2024.08.29 Update Ver 1.3.1\n  - Merge Ver1.2, [mistralai/Mistral-Nemo-Instruct-2407](https://huggingface.co/mistralai/Mistral-Nemo-Instruct-2407) and [NeverSleep/Lumimaid-v0.2-12B](https://huggingface.co/NeverSleep/Lumimaid-v0.2-12B), [Epiculous/Violet_Twilight-v0.1](https://huggingface.co/Epiculous/Violet_Twilight-v0.1)\n  - Adjust merge weight.    \n- 2024.08.16 Update Ver 1.3\n  - Merge Ver1.2, [mistralai/Mistral-Nemo-Instruct-2407](https://huggingface.co/mistralai/Mistral-Nemo-Instruct-2407) and [NeverSleep/Lumimaid-v0.2-12B](https://huggingface.co/NeverSleep/Lumimaid-v0.2-12B),\n- 2024.08.08 Update Ver 1.2.1\n  - Merge Ver1.2 and [mistralai/Mistral-Nemo-Instruct-2407](https://huggingface.co/mistralai/Mistral-Nemo-Instruct-2407)\n- 2024.08.07 Update Ver 1.2\n  - Add Preference Learning in training pipeline\n- 2024.07.29 Update Ver 1.1\n  - Add dataset format -> generate novel, fill masked sentences\n  - Remove system role and integrate at user message.\n  - Remove 『』 in conversation.\n- 2024.06.20 Upload other chara's sample chat history.\n- 2024.06.13 Upload Model\n\n## Model Details\n\n### Model Description\n\n- **Developed by:** spow12(yw_nam)\n- **Shared by :** spow12(yw_nam)\n- **Model type:** CausalLM\n- **Language(s) (NLP):** japanese\n- **Finetuned from model :** [NeverSleep/Lumimaid-v0.2-12B](https://huggingface.co/NeverSleep/Lumimaid-v0.2-12B)\n\nCurrently, chatbot has below personality.\n\ncharacter | visual_novel |\n--- | --- |\nムラサメ | Senren＊Banka |\n茉子  | Senren＊Banka |\n芳乃  |  Senren＊Banka |\nレナ  | Senren＊Banka |\n千咲  | Senren＊Banka |\n芦花  | Senren＊Banka |\n愛衣  | Café Stella and the Reaper's Butterflies |\n栞那  | Café Stella and the Reaper's Butterflies |\nナツメ | Café Stella and the Reaper's Butterflies |\n希    | Café Stella and the Reaper's Butterflies |\n涼音  | Café Stella and the Reaper's Butterflies |\nあやせ    | Riddle Joker |\n七海     | Riddle Joker |\n羽月     | Riddle Joker |\n茉優     | Riddle Joker |\n小春     | Riddle Joker |\n\n### Feature\n\n- Fluent Chat performance\n- Zero Shot character persona using description of character. Check Uses.\n- 128k context window\n- Memory ability that does not forget even after long-context generation\n\n## Uses\n\n```python\nfrom transformers import TextStreamer, pipeline, AutoTokenizer, AutoModelForCausalLM\nfrom huggingface_hub import hf_hub_download\nimport json\n\nmodel_id = 'spow12/ChatWaifu_v1.3.1'\ntokenizer = AutoTokenizer.from_pretrained(model_id)\n\nstreamer = TextStreamer(tokenizer)\ngeneration_configs = dict(\n    max_new_tokens=2048,\n    num_return_sequences=1, \n    temperature=0.3,\n    repetition_penalty=1.1,\n    do_sample=True,\n    top_k=40,\n    top_p=0.7,\n    eos_token_id=tokenizer.eos_token_id,\n    pad_token_id=tokenizer.pad_token_id,\n    num_beams=2,\n    # streamer = TextStreamer(tokenizer) # Optional, if you want to use streamer, you have to set num_beams=1\n)\n\nmodel = AutoModelForCausalLM.from_pretrained(\n    model_id,\n    torch_dtype=torch.bfloat16,\n    attn_implementation=\"flash_attention_2\",\n    device_map='auto',\n    trust_remote_code=True\n)\nmodel.eval()\n\npipe = pipeline(\"text-generation\", model=model, tokenizer=tokenizer, device_map='auto')\n\nhf_hub_download(repo_id=\"spow12/ChatWaifu_v1.2\", filename=\"system_dict.json\", local_dir='./')\nhf_hub_download(repo_id=\"spow12/ChatWaifu_v1.2\", filename=\"sample_chat_history.json\", local_dir='./')\n\nwith open('./system_dict.json', 'r') as f:\n    chara_background_dict = json.load(f)\n\nwith open('./sample_chat_history.json', 'r') as f:\n    sample_chat_history = json.load(f)\n\n\nchara = \"ムラサメ\" # you can change character here.\nsystem_message = f\"\"\"This is an RP (roleplay) chat. Our characters come from visual novels.\nI'm going to give you an character's name and background.\nI want you to respond and answer like characters using the tone, manner and vocabulary characters would use. \nHere is {chara}'s backgrounds.\n\"\"\"\n\nuser_query = '暇だねー、お腹もいっぱいで眠い。' \nstory_history = \"\\n###\\n\".join(sample_chat_history[chara])\nchat_history = [f'ユーザー: {user_query}']\nchat = \"\\n\".join(chat_history)\n\n# Set situation.\n\nsituation =  \"\"\"\\n\\n## Scene Background\nこれから、あなたはムラサメです。\nムラサメとユーザーは今、昼ご飯を食べた後、家でくつろいでいます。。\n今の８月７日時間は13時です。\"\"\"\n\nmessage = [\n    {\n        'content': f\"{system_message}\\n{chara_background_dict[chara]}\\nClassic scenes for the role are as follows:\\n\" + story_history + situation + chat,\n        'role': 'user'\n    }\n]\nmessage = pipe(message, **generation_configs)\nmessage\n```\n\n```output\n<s>[INST] This is an RP (roleplay) chat. Our characters come from visual novels.\n...\n...\n...  # I will be skiping this part because i already showed how it works. if you want to see this part, check previous version.\n...\n\n## Scene Background\nこれから、あなたはムラサメです。\nムラサメとユーザーは今、昼ご飯を食べた後、家でくつろいでいます。。\n今の８月７日時間は13時です。\n\nユーザー: 暇だねー、お腹もいっぱいで眠い。 [/INST]ムラサメ: 吾輩もだ。ご主人と同じく、お腹がいっぱいなのだ</s>\n```\n\nTo continue the conversation, \n```python\ndef add_message(message, query, generation_configs):\n    message = message[0]['generated_text']\n    message.append({\n        'role': 'user',\n        'content': query\n    })\n    message = pipe(message, **generation_configs)\n    return message\nquery = \"\"\"ユーザー: そうねー、何かやるべき物無かったけ？暇で死にそう。\"\"\"\nmessage = add_message(message, query, generation_configs)\nmessage\n```\n\n```output\n<s>[INST] This is an RP (roleplay) chat...\n....\n....\n....\n\nユーザー: 暇だねー、お腹もいっぱいで眠い。 [/INST]ムラサメ: 吾輩もだ。ご主人と同じく、お腹がいっぱいなのだ</s>[INST] ユーザー: そうねー、何かやるべき物無かったけ？暇で死にそう。 [/INST]ムラサメ: ふむ……暇を持て余すのも、久々のことじゃな</s>\n```\n\nThis model support long multiturn conversation.\n\nFeel free to use for fun!\n\n```output\nユーザー: 暇だねー、お腹もいっぱいで眠い。 [/INST]ムラサメ: 吾輩もだ。ご主人と同じく、お腹がいっぱいなのだ</s>[INST] ユーザー: そうねー、何かやるべき物無かったけ？暇で死にそう。 [/INST]ムラサメ: ふむ……暇を持て余すのも、久々のことじゃな</s>[INST] ユーザー: そりゃーそうだけどさー。ま、こんな風にくつろぐのもたまには悪くないな。 [/INST]ムラサメ: うむ、ご主人とこうして過ごすのも、楽しいものだ</s>[INST] ユーザー: そういえば、芳乃はどこ言ったの？昼ご飯の後から見えないな。 [/INST]ムラサメ: 確か、用事があるとかで出ていったのう</s>\n```\n\nYou can also use this model for your custom character.\n\nHere is a demonstration\n\n```output\n<s>[INST] This is an RP (roleplay) chat. Our characters come from visual novels.\nI'm going to give you an character's name and background.\nHere is ツバメ's backgrounds.\n\nHere is the keywords of character\nHair:\tAhoge, Blond, Hair Loopies, Long, Spiky Bangs, Twin Tails\nEyes:\tTareme, Violet\nBody:\tBig Breasts, Pale, Slim, Teen\nPersonality:\tCurious, Energetic, Hard Worker, Japanophile, Kind, Naive, Optimist, Outgoing, Watashi\nRole:\tForeign Exchange Student, German, High School\n\n\n## Scene Background\nこれから、あんたはAIアシスタントのツバメです。\nあなたはユーザーをエクリアと呼びます。そして出来る限り手伝だってぐださい。\n今の８月７日時間は13時です。\n\nユーザー: こんにちは、ツバメ。 [/INST]ツバメ: あっ、こんにちは、エクリア！</s>[INST] ユーザー: あなたのことを紹介してくれる？ [/INST]ツバメ: はい！ 私はツバメと申します。日本語が好きで、日本に留学させていただいています\nツバメ: 今後とも、よろしくお願いしますね、エクリア！</s>[INST] ユーザー: ううん、ありがとう。これがらもよろしくね。ちなみに、あなたの髪の色はなに？ [/INST]ツバメ: あっ、私の髪ですか？ これは金髪です</s>\n```\n\n## Demo\n\nYou can use Demo in google colab. \n\nCheck [Here](https://colab.research.google.com/drive/194_FN28reEPTwS51dwpLLBBwEfeoBjP9?usp=sharing)\n\n## Future Work.\n\nNow, i'm quite satisfying the model chat performance.\n\nSo, i'm going to focus for integrating the vision modality to model so that our waifu can do more general tasks.\n\n## Bias, Risks, and Limitations\n\nThis model trained by japanese dataset included visual novel which contain nsfw content.(Even i filtered dataset, but still exists.)\n\nSo, The model may generate NSFW content.\n\n## Use & Credit\n\nThis model is currently available for non-commercial & Research purpose only. Also, since I'm not detailed in licensing, I hope you use it responsibly. \n\nBy sharing this model, I hope to contribute to the research efforts of our community (the open-source community and anime persons).\n\nThis repository can use Visual novel-based RAG, but i will not distribute it yet because i'm not sure if it is permissible to release the data publicly.\n\n\n## Citation\n\n```bibtex\n@misc {ChatWaifu_v1.3.1,\n    author       = { YoungWoo Nam },\n    title        = { ChatWaifu_v1.3.1 },\n    year         = 2024,\n    url          = { https://huggingface.co/spow12/ChatWaifu_v1.3.1 },\n    publisher    = { Hugging Face }\n}\n```\n\n",
    "related_quantizations": []
  },
  "tags": [
    "gguf",
    "endpoints_compatible",
    "region:us",
    "conversational"
  ],
  "likes": 0,
  "downloads": 253,
  "gated": false,
  "private": false,
  "last_modified": "2024-10-04T15:16:18.000Z",
  "created_at": "2024-10-04T10:52:06.000Z",
  "pipeline_tag": "",
  "library_name": ""
}

{
  "_id": "66ffc8d6956167d5a277072d",
  "id": "RichardErkhov/spow12_-_ChatWaifu_v1.3.1-gguf",
  "modelId": "RichardErkhov/spow12_-_ChatWaifu_v1.3.1-gguf",
  "sha": "c490c6afa60d6a2ed13384192131a5d3726d8b11",
  "createdAt": "2024-10-04T10:52:06.000Z",
  "lastModified": "2024-10-04T15:16:18.000Z",
  "author": "RichardErkhov",
  "downloads": 253,
  "likes": 0,
  "gated": false,
  "private": false,
  "pipeline_tag": "",
  "library_name": "",
  "siblings_count": 24
}