richarderkhov/georgesung_-_llama2_7b_chat_uncensored-gguf overview

{
  "metadata": {},
  "card_data": {
    "frontmatter": {},
    "hero_image_url": "",
    "summary": "Fine-tuned Llama-2 7B with an uncensored/unfiltered Wizard-Vicuna conversation dataset (originally from ehartford/wizard_vicuna_70k_unfiltered). Used QLoRA for fine-tuning. Trained for one epoch on a 24GB GPU (NVIDIA A10G) instance, took ~19 hours to train. The version here is the fp16 HuggingFace model.",
    "quick_links": [],
    "benchmark_table_html": "",
    "readme_markdown": "Quantization made by Richard Erkhov.\n\n[Github](https://github.com/RichardErkhov)\n\n[Discord](https://discord.gg/pvy7H8DZMG)\n\n[Request more models](https://github.com/RichardErkhov/quant_request)\n\n\nllama2_7b_chat_uncensored - GGUF\n- Model creator: https://huggingface.co/georgesung/\n- Original model: https://huggingface.co/georgesung/llama2_7b_chat_uncensored/\n\n\n| Name | Quant method | Size |\n| ---- | ---- | ---- |\n| [llama2_7b_chat_uncensored.Q2_K.gguf](https://huggingface.co/RichardErkhov/georgesung_-_llama2_7b_chat_uncensored-gguf/blob/main/llama2_7b_chat_uncensored.Q2_K.gguf) | Q2_K | 2.36GB |\n| [llama2_7b_chat_uncensored.Q3_K_S.gguf](https://huggingface.co/RichardErkhov/georgesung_-_llama2_7b_chat_uncensored-gguf/blob/main/llama2_7b_chat_uncensored.Q3_K_S.gguf) | Q3_K_S | 2.75GB |\n| [llama2_7b_chat_uncensored.Q3_K.gguf](https://huggingface.co/RichardErkhov/georgesung_-_llama2_7b_chat_uncensored-gguf/blob/main/llama2_7b_chat_uncensored.Q3_K.gguf) | Q3_K | 3.07GB |\n| [llama2_7b_chat_uncensored.Q3_K_M.gguf](https://huggingface.co/RichardErkhov/georgesung_-_llama2_7b_chat_uncensored-gguf/blob/main/llama2_7b_chat_uncensored.Q3_K_M.gguf) | Q3_K_M | 3.07GB |\n| [llama2_7b_chat_uncensored.Q3_K_L.gguf](https://huggingface.co/RichardErkhov/georgesung_-_llama2_7b_chat_uncensored-gguf/blob/main/llama2_7b_chat_uncensored.Q3_K_L.gguf) | Q3_K_L | 3.35GB |\n| [llama2_7b_chat_uncensored.IQ4_XS.gguf](https://huggingface.co/RichardErkhov/georgesung_-_llama2_7b_chat_uncensored-gguf/blob/main/llama2_7b_chat_uncensored.IQ4_XS.gguf) | IQ4_XS | 3.4GB |\n| [llama2_7b_chat_uncensored.Q4_0.gguf](https://huggingface.co/RichardErkhov/georgesung_-_llama2_7b_chat_uncensored-gguf/blob/main/llama2_7b_chat_uncensored.Q4_0.gguf) | Q4_0 | 3.56GB |\n| [llama2_7b_chat_uncensored.IQ4_NL.gguf](https://huggingface.co/RichardErkhov/georgesung_-_llama2_7b_chat_uncensored-gguf/blob/main/llama2_7b_chat_uncensored.IQ4_NL.gguf) | IQ4_NL | 3.58GB |\n| [llama2_7b_chat_uncensored.Q4_K_S.gguf](https://huggingface.co/RichardErkhov/georgesung_-_llama2_7b_chat_uncensored-gguf/blob/main/llama2_7b_chat_uncensored.Q4_K_S.gguf) | Q4_K_S | 3.59GB |\n| [llama2_7b_chat_uncensored.Q4_K.gguf](https://huggingface.co/RichardErkhov/georgesung_-_llama2_7b_chat_uncensored-gguf/blob/main/llama2_7b_chat_uncensored.Q4_K.gguf) | Q4_K | 3.8GB |\n| [llama2_7b_chat_uncensored.Q4_K_M.gguf](https://huggingface.co/RichardErkhov/georgesung_-_llama2_7b_chat_uncensored-gguf/blob/main/llama2_7b_chat_uncensored.Q4_K_M.gguf) | Q4_K_M | 3.8GB |\n| [llama2_7b_chat_uncensored.Q4_1.gguf](https://huggingface.co/RichardErkhov/georgesung_-_llama2_7b_chat_uncensored-gguf/blob/main/llama2_7b_chat_uncensored.Q4_1.gguf) | Q4_1 | 3.95GB |\n| [llama2_7b_chat_uncensored.Q5_0.gguf](https://huggingface.co/RichardErkhov/georgesung_-_llama2_7b_chat_uncensored-gguf/blob/main/llama2_7b_chat_uncensored.Q5_0.gguf) | Q5_0 | 4.33GB |\n| [llama2_7b_chat_uncensored.Q5_K_S.gguf](https://huggingface.co/RichardErkhov/georgesung_-_llama2_7b_chat_uncensored-gguf/blob/main/llama2_7b_chat_uncensored.Q5_K_S.gguf) | Q5_K_S | 4.33GB |\n| [llama2_7b_chat_uncensored.Q5_K.gguf](https://huggingface.co/RichardErkhov/georgesung_-_llama2_7b_chat_uncensored-gguf/blob/main/llama2_7b_chat_uncensored.Q5_K.gguf) | Q5_K | 4.45GB |\n| [llama2_7b_chat_uncensored.Q5_K_M.gguf](https://huggingface.co/RichardErkhov/georgesung_-_llama2_7b_chat_uncensored-gguf/blob/main/llama2_7b_chat_uncensored.Q5_K_M.gguf) | Q5_K_M | 4.45GB |\n| [llama2_7b_chat_uncensored.Q5_1.gguf](https://huggingface.co/RichardErkhov/georgesung_-_llama2_7b_chat_uncensored-gguf/blob/main/llama2_7b_chat_uncensored.Q5_1.gguf) | Q5_1 | 4.72GB |\n| [llama2_7b_chat_uncensored.Q6_K.gguf](https://huggingface.co/RichardErkhov/georgesung_-_llama2_7b_chat_uncensored-gguf/blob/main/llama2_7b_chat_uncensored.Q6_K.gguf) | Q6_K | 5.15GB |\n| [llama2_7b_chat_uncensored.Q8_0.gguf](https://huggingface.co/RichardErkhov/georgesung_-_llama2_7b_chat_uncensored-gguf/blob/main/llama2_7b_chat_uncensored.Q8_0.gguf) | Q8_0 | 6.67GB |\n\n\n\n\nOriginal model description:\n---\nlicense: other\ndatasets:\n- georgesung/wizard_vicuna_70k_unfiltered\n---\n\n# Overview\nFine-tuned [Llama-2 7B](https://huggingface.co/TheBloke/Llama-2-7B-fp16) with an uncensored/unfiltered Wizard-Vicuna conversation dataset (originally from [ehartford/wizard_vicuna_70k_unfiltered](https://huggingface.co/datasets/ehartford/wizard_vicuna_70k_unfiltered)).\nUsed QLoRA for fine-tuning. Trained for one epoch on a 24GB GPU (NVIDIA A10G) instance, took ~19 hours to train.\n\nThe version here is the fp16 HuggingFace model.\n\n## GGML & GPTQ versions\nThanks to [TheBloke](https://huggingface.co/TheBloke), he has created the GGML and GPTQ versions:\n* https://huggingface.co/TheBloke/llama2_7b_chat_uncensored-GGML\n* https://huggingface.co/TheBloke/llama2_7b_chat_uncensored-GPTQ\n\n## Running in Ollama\nhttps://ollama.com/library/llama2-uncensored\n\n# Prompt style\nThe model was trained with the following prompt style:\n```\n### HUMAN:\nHello\n\n### RESPONSE:\nHi, how are you?\n\n### HUMAN:\nI'm fine.\n\n### RESPONSE:\nHow can I help you?\n...\n```\n\n# Training code\nCode used to train the model is available [here](https://github.com/georgesung/llm_qlora).\n\nTo reproduce the results:\n```\ngit clone https://github.com/georgesung/llm_qlora\ncd llm_qlora\npip install -r requirements.txt\npython train.py configs/llama2_7b_chat_uncensored.yaml\n```\n\n# Fine-tuning guide\nhttps://georgesung.github.io/ai/qlora-ift/\n\n# [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)\nDetailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_georgesung__llama2_7b_chat_uncensored)\n\n| Metric                | Value                     |\n|-----------------------|---------------------------|\n| Avg.                  | 43.39   |\n| ARC (25-shot)         | 53.58          |\n| HellaSwag (10-shot)   | 78.66    |\n| MMLU (5-shot)         | 44.49         |\n| TruthfulQA (0-shot)   | 41.34   |\n| Winogrande (5-shot)   | 74.11   |\n| GSM8K (5-shot)        | 5.84        |\n| DROP (3-shot)         | 5.69         |\n\n",
    "related_quantizations": []
  },
  "tags": [
    "gguf",
    "endpoints_compatible",
    "region:us"
  ],
  "likes": 1,
  "downloads": 81,
  "gated": false,
  "private": false,
  "last_modified": "2024-11-05T19:11:56.000Z",
  "created_at": "2024-11-05T12:36:39.000Z",
  "pipeline_tag": "",
  "library_name": ""
}

{
  "_id": "672a11578782be73365637fd",
  "id": "RichardErkhov/georgesung_-_llama2_7b_chat_uncensored-gguf",
  "modelId": "RichardErkhov/georgesung_-_llama2_7b_chat_uncensored-gguf",
  "sha": "07662c34c9eb96e1908f02777208ce90d3c3104c",
  "createdAt": "2024-11-05T12:36:39.000Z",
  "lastModified": "2024-11-05T19:11:56.000Z",
  "author": "RichardErkhov",
  "downloads": 81,
  "likes": 1,
  "gated": false,
  "private": false,
  "pipeline_tag": "",
  "library_name": "",
  "siblings_count": 21
}