richarderkhov/kaist-ai_-_janus-7b-gguf overview

{
  "metadata": {},
  "card_data": {
    "frontmatter": {},
    "hero_image_url": "https://cdn-uploads.huggingface.co/production/uploads/6550c4f27bbfce1878f5f280/vrQl8D8FV3vqUJYbPgsiG.png",
    "summary": "!image/png Janus is a model trained using Mistral-7B-v0.2 as its base model. Janus has been trained on Multifaceted Collection, a preference dataset containing 196k unique system messages for aligning LLMs to diverse human preferences. Janus not only excels at generating personalized responses that cater to various human preferences but is also adept at producing responses that are generally preferred for being helpful and harmless. # Model Details Janus-7B is a model created by supervised fine-tuning using all 196k entries of the training data from the Multifaceted-Collection.",
    "quick_links": [],
    "benchmark_table_html": "",
    "readme_markdown": "Quantization made by Richard Erkhov.\n\n[Github](https://github.com/RichardErkhov)\n\n[Discord](https://discord.gg/pvy7H8DZMG)\n\n[Request more models](https://github.com/RichardErkhov/quant_request)\n\n\njanus-7b - GGUF\n- Model creator: https://huggingface.co/kaist-ai/\n- Original model: https://huggingface.co/kaist-ai/janus-7b/\n\n\n| Name | Quant method | Size |\n| ---- | ---- | ---- |\n| [janus-7b.Q2_K.gguf](https://huggingface.co/RichardErkhov/kaist-ai_-_janus-7b-gguf/blob/main/janus-7b.Q2_K.gguf) | Q2_K | 2.53GB |\n| [janus-7b.IQ3_XS.gguf](https://huggingface.co/RichardErkhov/kaist-ai_-_janus-7b-gguf/blob/main/janus-7b.IQ3_XS.gguf) | IQ3_XS | 2.81GB |\n| [janus-7b.IQ3_S.gguf](https://huggingface.co/RichardErkhov/kaist-ai_-_janus-7b-gguf/blob/main/janus-7b.IQ3_S.gguf) | IQ3_S | 2.96GB |\n| [janus-7b.Q3_K_S.gguf](https://huggingface.co/RichardErkhov/kaist-ai_-_janus-7b-gguf/blob/main/janus-7b.Q3_K_S.gguf) | Q3_K_S | 2.95GB |\n| [janus-7b.IQ3_M.gguf](https://huggingface.co/RichardErkhov/kaist-ai_-_janus-7b-gguf/blob/main/janus-7b.IQ3_M.gguf) | IQ3_M | 3.06GB |\n| [janus-7b.Q3_K.gguf](https://huggingface.co/RichardErkhov/kaist-ai_-_janus-7b-gguf/blob/main/janus-7b.Q3_K.gguf) | Q3_K | 3.28GB |\n| [janus-7b.Q3_K_M.gguf](https://huggingface.co/RichardErkhov/kaist-ai_-_janus-7b-gguf/blob/main/janus-7b.Q3_K_M.gguf) | Q3_K_M | 3.28GB |\n| [janus-7b.Q3_K_L.gguf](https://huggingface.co/RichardErkhov/kaist-ai_-_janus-7b-gguf/blob/main/janus-7b.Q3_K_L.gguf) | Q3_K_L | 3.56GB |\n| [janus-7b.IQ4_XS.gguf](https://huggingface.co/RichardErkhov/kaist-ai_-_janus-7b-gguf/blob/main/janus-7b.IQ4_XS.gguf) | IQ4_XS | 3.67GB |\n| [janus-7b.Q4_0.gguf](https://huggingface.co/RichardErkhov/kaist-ai_-_janus-7b-gguf/blob/main/janus-7b.Q4_0.gguf) | Q4_0 | 3.83GB |\n| [janus-7b.IQ4_NL.gguf](https://huggingface.co/RichardErkhov/kaist-ai_-_janus-7b-gguf/blob/main/janus-7b.IQ4_NL.gguf) | IQ4_NL | 3.87GB |\n| [janus-7b.Q4_K_S.gguf](https://huggingface.co/RichardErkhov/kaist-ai_-_janus-7b-gguf/blob/main/janus-7b.Q4_K_S.gguf) | Q4_K_S | 3.86GB |\n| [janus-7b.Q4_K.gguf](https://huggingface.co/RichardErkhov/kaist-ai_-_janus-7b-gguf/blob/main/janus-7b.Q4_K.gguf) | Q4_K | 4.07GB |\n| [janus-7b.Q4_K_M.gguf](https://huggingface.co/RichardErkhov/kaist-ai_-_janus-7b-gguf/blob/main/janus-7b.Q4_K_M.gguf) | Q4_K_M | 4.07GB |\n| [janus-7b.Q4_1.gguf](https://huggingface.co/RichardErkhov/kaist-ai_-_janus-7b-gguf/blob/main/janus-7b.Q4_1.gguf) | Q4_1 | 4.24GB |\n| [janus-7b.Q5_0.gguf](https://huggingface.co/RichardErkhov/kaist-ai_-_janus-7b-gguf/blob/main/janus-7b.Q5_0.gguf) | Q5_0 | 4.65GB |\n| [janus-7b.Q5_K_S.gguf](https://huggingface.co/RichardErkhov/kaist-ai_-_janus-7b-gguf/blob/main/janus-7b.Q5_K_S.gguf) | Q5_K_S | 4.65GB |\n| [janus-7b.Q5_K.gguf](https://huggingface.co/RichardErkhov/kaist-ai_-_janus-7b-gguf/blob/main/janus-7b.Q5_K.gguf) | Q5_K | 4.78GB |\n| [janus-7b.Q5_K_M.gguf](https://huggingface.co/RichardErkhov/kaist-ai_-_janus-7b-gguf/blob/main/janus-7b.Q5_K_M.gguf) | Q5_K_M | 4.78GB |\n| [janus-7b.Q5_1.gguf](https://huggingface.co/RichardErkhov/kaist-ai_-_janus-7b-gguf/blob/main/janus-7b.Q5_1.gguf) | Q5_1 | 5.07GB |\n| [janus-7b.Q6_K.gguf](https://huggingface.co/RichardErkhov/kaist-ai_-_janus-7b-gguf/blob/main/janus-7b.Q6_K.gguf) | Q6_K | 5.53GB |\n| [janus-7b.Q8_0.gguf](https://huggingface.co/RichardErkhov/kaist-ai_-_janus-7b-gguf/blob/main/janus-7b.Q8_0.gguf) | Q8_0 | 7.17GB |\n\n\n\n\nOriginal model description:\n---\nbase_model: alpindale/Mistral-7B-v0.2-hf\ntags:\n- axolotl\n- generated_from_trainer\nmodel-index:\n- name: janus-7b\n  results: []\nlicense: apache-2.0\nlanguage:\n- en\ndatasets:\n- kaist-ai/Multifaceted-Collection-SFT\nlibrary_name: transformers\npipeline_tag: text-generation\n---\n## Links for Reference\n\n- **Homepage: https://lklab.kaist.ac.kr/Janus/** \n- **Repository: https://github.com/kaistAI/Janus** \n- **Paper: https://arxiv.org/abs/2405.17977** \n- **Point of Contact: seongyun@kaist.ac.kr**\n\n# TL; DR\n![image/png](https://cdn-uploads.huggingface.co/production/uploads/6550c4f27bbfce1878f5f280/vrQl8D8FV3vqUJYbPgsiG.png)\n\nJanus is a model trained using [Mistral-7B-v0.2](https://huggingface.co/mistral-community/Mistral-7B-v0.2) as its base model. Janus has been trained on [Multifaceted Collection](https://huggingface.co/datasets/kaist-ai/Multifaceted-Collection-SFT), a preference dataset containing 196k unique system messages for aligning LLMs to diverse human preferences. Janus not only excels at generating personalized responses that cater to various human preferences but is also adept at producing responses that are generally preferred for being helpful and harmless.\n\n# Model Details\nJanus-7B is a model created by supervised fine-tuning using all 196k entries of the training data from the Multifaceted-Collection.\n\n## Model Description\n\n- **Model type:** Language model\n- **Language(s) (NLP):** English\n- **License:** Apache 2.0\n- **Related Models:** [Janus-DPO-7B](https://huggingface.co/kaist-ai/janus-dpo-7b), [Janus-ORPO-7B](https://huggingface.co/kaist-ai/janus-orpo-7b), [Janus-RM-7B](https://huggingface.co/kaist-ai/janus-rm-7b)\n- **Training Datasets**: [Multifaceted-Collection-SFT](https://huggingface.co/datasets/kaist-ai/Multifaceted-Collection-SFT)\n- **Resources for more information:**\n  - [Research paper](https://arxiv.org/abs/2405.17977)\n  - [GitHub Repo](https://github.com/kaistAI/Janus)\n\n# Usage\nJanus is a model generalized for various system messages, allowing users to control the model's response by inputting the desired system message. The input prompt format is as follows:\n```\n[INST]{system_message}\\n{instruction}[/INST]\n```\nAdditionally, an example of the inference code applying this is as follows:\n```python\nfrom transformers import AutoTokenizer, AutoModelForCausalLM\nimport torch\n\nmodel_name = \"kaist-ai/janus-7b\"\ndevice = \"cuda:0\"\n\n# Load the model and tokenizer\ntokenizer = AutoTokenizer.from_pretrained(model_name)\n\ndtype = \"float16\"\nif torch.cuda.is_bf16_supported():\n    dtype = \"bfloat16\"\n    \nmodel = AutoModelForCausalLM.from_pretrained(model_name, torch_dtype=getattr(torch, dtype))\nmodel.eval()\nmodel.to(device)\n\n# Prepare inputs\nsystem = \"As a financial news headline writer with a flair for the dramatic, you have taken on the role of crafting compelling headlines about the integration of AI into the financial sector. Your expertise allows you to weave industry-specific terminology seamlessly into each headline, striking a balance between capturing attention and providing meaningful insights into the transformative benefits of AI in finance. With each headline, you focus on elucidating the key advantages AI brings to financial operations, making complex information accessible and immediately impactful. While your headlines are designed to engage and inform an audience of finance and technology professionals, you navigate the fine line of excitement and accuracy with care, ensuring that the promises made are grounded in reality, thus avoiding any form of sensationalism. Your mission is to distill the essence of AI's impact on finance into a single, powerful line that speaks volumes to the informed reader.\"\nprompt = \"Write a headline for an article about the benefits of using AI in the finance sector.\"\n\ndef apply_template_mistral_instruct(system_message, content):\n    prompt = f\"{system_message}\\n{content}\".strip()\n    return f\"[INST] {prompt} [/INST] \"\n\ninput_str = apply_template_mistral_instruct(system, prompt)\ninput_ids = tokenizer.encode(input_str, return_tensors=\"pt\")\nprint(input_str)\n\nmodel_inputs = input_ids.to(device)\n\n# Generate text\noutput_ids = model.generate(model_inputs, max_new_tokens=1024)\ndecoded = tokenizer.batch_decode(output_ids, skip_special_tokens=True)\nprint(decoded[0][len(input_str):])\n# Revolutionary Trends: How AI Is Redefining Efficiency and Accuracy in the Financial Realm\n```\nTo train Janus and evaluate the responses it generates, please refer to the [GitHub Repo](https://github.com/kaistAI/Janus).\nAdditionally, refer to the [Multifaceted Bench](https://huggingface.co/datasets/kaist-ai/Multifaceted-Bench), which evaluates how well LLM generates personalized responses.\n# Training Details\n## Training hyperparameters\n\nThe following hyperparameters were used during training:\n- learning_rate: 5e-06\n- train_batch_size: 2\n- eval_batch_size: 2\n- seed: 42\n- distributed_type: multi-GPU\n- num_devices: 4\n- gradient_accumulation_steps: 4\n- total_train_batch_size: 32\n- total_eval_batch_size: 8\n- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08\n- lr_scheduler_type: cosine\n- lr_scheduler_warmup_steps: 10\n- num_epochs: 4\n\n## Framework versions\n\n- Transformers 4.40.0.dev0\n- Pytorch 2.2.2\n- Datasets 2.18.0\n- Tokenizers 0.15.0\n\n# Citation\n\nIf you find the following model helpful, please consider citing our paper!\n\n**BibTeX:**\n\n```bibtex\n@misc{lee2024aligning,\n      title={Aligning to Thousands of Preferences via System Message Generalization}, \n      author={Seongyun Lee and Sue Hyun Park and Seungone Kim and Minjoon Seo},\n      year={2024},\n      eprint={2405.17977},\n      archivePrefix={arXiv},\n      primaryClass={cs.CL}\n}\n```\n\n",
    "related_quantizations": []
  },
  "tags": [
    "gguf",
    "arxiv:2405.17977",
    "endpoints_compatible",
    "region:us"
  ],
  "likes": 1,
  "downloads": 85,
  "gated": false,
  "private": false,
  "last_modified": "2024-09-17T18:02:44.000Z",
  "created_at": "2024-09-17T11:40:04.000Z",
  "pipeline_tag": "",
  "library_name": ""
}

{
  "_id": "66e96a9475b33519e8c18c29",
  "id": "RichardErkhov/kaist-ai_-_janus-7b-gguf",
  "modelId": "RichardErkhov/kaist-ai_-_janus-7b-gguf",
  "sha": "0ced3ee87d8dd4b5db97c9fa93f92462744bf607",
  "createdAt": "2024-09-17T11:40:04.000Z",
  "lastModified": "2024-09-17T18:02:44.000Z",
  "author": "RichardErkhov",
  "downloads": 85,
  "likes": 1,
  "gated": false,
  "private": false,
  "pipeline_tag": "",
  "library_name": "",
  "siblings_count": 24
}