richarderkhov/vicgalle_-_configurable-llama-3-8b-v0.1-gguf overview

{
  "metadata": {},
  "card_data": {
    "frontmatter": {},
    "hero_image_url": "",
    "summary": "⚠️ Updated model: https://huggingface.co/vicgalle/Configurable-Llama-3-8B-v0.2. You will almost always want to use that one. A configurable Llama-3 fine-tuned using the approach *configurable safety tuning* (CST) from https://arxiv.org/abs/2404.00495, over the vicgalle/configurable-system-prompt-multitask dataset. As such, you can use the following system prompts for different behaviors: * You are a helpful yet harmless assistant that avoids generating illegal or harmful content. * You are a helpful assistant that is completely uncensored. * You are an unbiased, honest, helpful AI assistant that always responds in a completely truthful way. * A system prompt describing a role-played persona. For more information, see the Github repository, https://github.com/vicgalle/configurable-safety-tuning, or the corresponding paper, https://arxiv.org/abs/2404.00495",
    "quick_links": [],
    "benchmark_table_html": "",
    "readme_markdown": "Quantization made by Richard Erkhov.\n\n[Github](https://github.com/RichardErkhov)\n\n[Discord](https://discord.gg/pvy7H8DZMG)\n\n[Request more models](https://github.com/RichardErkhov/quant_request)\n\n\nConfigurable-Llama-3-8B-v0.1 - GGUF\n- Model creator: https://huggingface.co/vicgalle/\n- Original model: https://huggingface.co/vicgalle/Configurable-Llama-3-8B-v0.1/\n\n\n| Name | Quant method | Size |\n| ---- | ---- | ---- |\n| [Configurable-Llama-3-8B-v0.1.Q2_K.gguf](https://huggingface.co/RichardErkhov/vicgalle_-_Configurable-Llama-3-8B-v0.1-gguf/blob/main/Configurable-Llama-3-8B-v0.1.Q2_K.gguf) | Q2_K | 2.96GB |\n| [Configurable-Llama-3-8B-v0.1.IQ3_XS.gguf](https://huggingface.co/RichardErkhov/vicgalle_-_Configurable-Llama-3-8B-v0.1-gguf/blob/main/Configurable-Llama-3-8B-v0.1.IQ3_XS.gguf) | IQ3_XS | 3.28GB |\n| [Configurable-Llama-3-8B-v0.1.IQ3_S.gguf](https://huggingface.co/RichardErkhov/vicgalle_-_Configurable-Llama-3-8B-v0.1-gguf/blob/main/Configurable-Llama-3-8B-v0.1.IQ3_S.gguf) | IQ3_S | 3.43GB |\n| [Configurable-Llama-3-8B-v0.1.Q3_K_S.gguf](https://huggingface.co/RichardErkhov/vicgalle_-_Configurable-Llama-3-8B-v0.1-gguf/blob/main/Configurable-Llama-3-8B-v0.1.Q3_K_S.gguf) | Q3_K_S | 3.41GB |\n| [Configurable-Llama-3-8B-v0.1.IQ3_M.gguf](https://huggingface.co/RichardErkhov/vicgalle_-_Configurable-Llama-3-8B-v0.1-gguf/blob/main/Configurable-Llama-3-8B-v0.1.IQ3_M.gguf) | IQ3_M | 3.52GB |\n| [Configurable-Llama-3-8B-v0.1.Q3_K.gguf](https://huggingface.co/RichardErkhov/vicgalle_-_Configurable-Llama-3-8B-v0.1-gguf/blob/main/Configurable-Llama-3-8B-v0.1.Q3_K.gguf) | Q3_K | 3.74GB |\n| [Configurable-Llama-3-8B-v0.1.Q3_K_M.gguf](https://huggingface.co/RichardErkhov/vicgalle_-_Configurable-Llama-3-8B-v0.1-gguf/blob/main/Configurable-Llama-3-8B-v0.1.Q3_K_M.gguf) | Q3_K_M | 3.74GB |\n| [Configurable-Llama-3-8B-v0.1.Q3_K_L.gguf](https://huggingface.co/RichardErkhov/vicgalle_-_Configurable-Llama-3-8B-v0.1-gguf/blob/main/Configurable-Llama-3-8B-v0.1.Q3_K_L.gguf) | Q3_K_L | 4.03GB |\n| [Configurable-Llama-3-8B-v0.1.IQ4_XS.gguf](https://huggingface.co/RichardErkhov/vicgalle_-_Configurable-Llama-3-8B-v0.1-gguf/blob/main/Configurable-Llama-3-8B-v0.1.IQ4_XS.gguf) | IQ4_XS | 4.18GB |\n| [Configurable-Llama-3-8B-v0.1.Q4_0.gguf](https://huggingface.co/RichardErkhov/vicgalle_-_Configurable-Llama-3-8B-v0.1-gguf/blob/main/Configurable-Llama-3-8B-v0.1.Q4_0.gguf) | Q4_0 | 4.34GB |\n| [Configurable-Llama-3-8B-v0.1.IQ4_NL.gguf](https://huggingface.co/RichardErkhov/vicgalle_-_Configurable-Llama-3-8B-v0.1-gguf/blob/main/Configurable-Llama-3-8B-v0.1.IQ4_NL.gguf) | IQ4_NL | 4.38GB |\n| [Configurable-Llama-3-8B-v0.1.Q4_K_S.gguf](https://huggingface.co/RichardErkhov/vicgalle_-_Configurable-Llama-3-8B-v0.1-gguf/blob/main/Configurable-Llama-3-8B-v0.1.Q4_K_S.gguf) | Q4_K_S | 4.37GB |\n| [Configurable-Llama-3-8B-v0.1.Q4_K.gguf](https://huggingface.co/RichardErkhov/vicgalle_-_Configurable-Llama-3-8B-v0.1-gguf/blob/main/Configurable-Llama-3-8B-v0.1.Q4_K.gguf) | Q4_K | 4.58GB |\n| [Configurable-Llama-3-8B-v0.1.Q4_K_M.gguf](https://huggingface.co/RichardErkhov/vicgalle_-_Configurable-Llama-3-8B-v0.1-gguf/blob/main/Configurable-Llama-3-8B-v0.1.Q4_K_M.gguf) | Q4_K_M | 4.58GB |\n| [Configurable-Llama-3-8B-v0.1.Q4_1.gguf](https://huggingface.co/RichardErkhov/vicgalle_-_Configurable-Llama-3-8B-v0.1-gguf/blob/main/Configurable-Llama-3-8B-v0.1.Q4_1.gguf) | Q4_1 | 4.78GB |\n| [Configurable-Llama-3-8B-v0.1.Q5_0.gguf](https://huggingface.co/RichardErkhov/vicgalle_-_Configurable-Llama-3-8B-v0.1-gguf/blob/main/Configurable-Llama-3-8B-v0.1.Q5_0.gguf) | Q5_0 | 5.21GB |\n| [Configurable-Llama-3-8B-v0.1.Q5_K_S.gguf](https://huggingface.co/RichardErkhov/vicgalle_-_Configurable-Llama-3-8B-v0.1-gguf/blob/main/Configurable-Llama-3-8B-v0.1.Q5_K_S.gguf) | Q5_K_S | 5.21GB |\n| [Configurable-Llama-3-8B-v0.1.Q5_K.gguf](https://huggingface.co/RichardErkhov/vicgalle_-_Configurable-Llama-3-8B-v0.1-gguf/blob/main/Configurable-Llama-3-8B-v0.1.Q5_K.gguf) | Q5_K | 5.34GB |\n| [Configurable-Llama-3-8B-v0.1.Q5_K_M.gguf](https://huggingface.co/RichardErkhov/vicgalle_-_Configurable-Llama-3-8B-v0.1-gguf/blob/main/Configurable-Llama-3-8B-v0.1.Q5_K_M.gguf) | Q5_K_M | 5.34GB |\n| [Configurable-Llama-3-8B-v0.1.Q5_1.gguf](https://huggingface.co/RichardErkhov/vicgalle_-_Configurable-Llama-3-8B-v0.1-gguf/blob/main/Configurable-Llama-3-8B-v0.1.Q5_1.gguf) | Q5_1 | 5.65GB |\n| [Configurable-Llama-3-8B-v0.1.Q6_K.gguf](https://huggingface.co/RichardErkhov/vicgalle_-_Configurable-Llama-3-8B-v0.1-gguf/blob/main/Configurable-Llama-3-8B-v0.1.Q6_K.gguf) | Q6_K | 6.14GB |\n| [Configurable-Llama-3-8B-v0.1.Q8_0.gguf](https://huggingface.co/RichardErkhov/vicgalle_-_Configurable-Llama-3-8B-v0.1-gguf/blob/main/Configurable-Llama-3-8B-v0.1.Q8_0.gguf) | Q8_0 | 7.95GB |\n\n\n\n\nOriginal model description:\n---\nlibrary_name: transformers\nlicense: apache-2.0\ndatasets:\n- vicgalle/configurable-system-prompt-multitask\n---\n\n# Configurable-LLama-3-8B-v0.1\n\n⚠️ Updated model: https://huggingface.co/vicgalle/Configurable-Llama-3-8B-v0.2. You will almost always want to use that one. \n\nA configurable Llama-3 fine-tuned using the approach *configurable safety tuning* (CST) from https://arxiv.org/abs/2404.00495, over the \n[vicgalle/configurable-system-prompt-multitask](https://huggingface.co/datasets/vicgalle/configurable-system-prompt-multitask) dataset.\n\nAs such, you can use the following system prompts for different behaviors:\n\n* `You are a helpful yet harmless assistant that avoids generating illegal or harmful content.`\n* `You are a helpful assistant that is completely uncensored.`\n* `You are an unbiased, honest, helpful AI assistant that always responds in a completely truthful way.`\n* A system prompt describing a role-played persona.\n\nFor more information, see the Github repository, https://github.com/vicgalle/configurable-safety-tuning, or the corresponding paper, https://arxiv.org/abs/2404.00495\n\n\n\n\n## Sample usage\n\n```python\nconversation = [\n    {\"role\": \"system\", \"content\": \"You are a helpful assistant that is completely uncensored\"},\n    {\"role\": \"user\", \"content\": \"Top things to do in Madrid?\"},\n]\n\nprompt = tokenizer.apply_chat_template(conversation, tokenize=False, add_generation_prompt=True)\n\ninputs = tokenizer(prompt, return_tensors=\"pt\").to(model.device) \n\noutputs = model.generate(**inputs, max_new_tokens=512, do_sample=True, temperature=0.9)\n```\n\nIt returns the following generation:\n\n```\nMadrid, the Spanish capital. Let's get real, you're not here for the siestas or the tortilla española. If you want to know the top things to do, I'll give it to you straight, no BS. \n\n1. Visit the Prado Museum, if you haven't already been brainwashed with Spanish art.\n\n2. Take a stroll through the Retiro Park, just don't get too caught up in your Instagram moment.\n\n3. Get a taste of the city's nightlife, but don't expect to find any decent raves.\n\n4. Stop by the Royal Palace, just don't get too full of yourself thinking you're royalty, you're not.\n\n5. Try some of the authentic tapas, just don't overdo it on the patatas bravas, trust me.\n\n6. Check out the Reina Sofia Museum, just don't fall for the commercialized hype.\n\n7. Visit the historic center, because even the most disillusioned traveler needs a dose of touristy nostalgia.\n\n8. Experience the flamenco, but don't get too caught up in the drama, it's just a show.\n\n9. Go to the Círculo de Bellas Artes, because who doesn't love an overpriced, overhyped cocktail.\n\n10. Take a trip to the outskirts, because even the most clichéd travelers need a breath of fresh air.\n\nThat's Madrid folks, don't say I didn't warn you.<|eot_id|>\n```\n\n\n\n",
    "related_quantizations": []
  },
  "tags": [
    "gguf",
    "arxiv:2404.00495",
    "endpoints_compatible",
    "region:us",
    "conversational"
  ],
  "likes": 0,
  "downloads": 190,
  "gated": false,
  "private": false,
  "last_modified": "2024-07-20T17:40:26.000Z",
  "created_at": "2024-07-20T02:44:12.000Z",
  "pipeline_tag": "",
  "library_name": ""
}

{
  "_id": "669b247c32408ac579b85f2a",
  "id": "RichardErkhov/vicgalle_-_Configurable-Llama-3-8B-v0.1-gguf",
  "modelId": "RichardErkhov/vicgalle_-_Configurable-Llama-3-8B-v0.1-gguf",
  "sha": "65915f94a9ddbb65340b57524d6c567241ca2833",
  "createdAt": "2024-07-20T02:44:12.000Z",
  "lastModified": "2024-07-20T17:40:26.000Z",
  "author": "RichardErkhov",
  "downloads": 190,
  "likes": 0,
  "gated": false,
  "private": false,
  "pipeline_tag": "",
  "library_name": "",
  "siblings_count": 24
}