richarderkhov/svenni551_-_gemma-2b-it-toxic-v2.0-gguf overview

{
  "metadata": {},
  "card_data": {
    "frontmatter": {},
    "hero_image_url": "./assets/Gemma-2b-Toxic.png",
    "summary": "",
    "quick_links": [],
    "benchmark_table_html": "",
    "readme_markdown": "Quantization made by Richard Erkhov.\n\n[Github](https://github.com/RichardErkhov)\n\n[Discord](https://discord.gg/pvy7H8DZMG)\n\n[Request more models](https://github.com/RichardErkhov/quant_request)\n\n\ngemma-2b-it-toxic-v2.0 - GGUF\n- Model creator: https://huggingface.co/Svenni551/\n- Original model: https://huggingface.co/Svenni551/gemma-2b-it-toxic-v2.0/\n\n\n| Name | Quant method | Size |\n| ---- | ---- | ---- |\n| [gemma-2b-it-toxic-v2.0.Q2_K.gguf](https://huggingface.co/RichardErkhov/Svenni551_-_gemma-2b-it-toxic-v2.0-gguf/blob/main/gemma-2b-it-toxic-v2.0.Q2_K.gguf) | Q2_K | 1.08GB |\n| [gemma-2b-it-toxic-v2.0.IQ3_XS.gguf](https://huggingface.co/RichardErkhov/Svenni551_-_gemma-2b-it-toxic-v2.0-gguf/blob/main/gemma-2b-it-toxic-v2.0.IQ3_XS.gguf) | IQ3_XS | 1.16GB |\n| [gemma-2b-it-toxic-v2.0.IQ3_S.gguf](https://huggingface.co/RichardErkhov/Svenni551_-_gemma-2b-it-toxic-v2.0-gguf/blob/main/gemma-2b-it-toxic-v2.0.IQ3_S.gguf) | IQ3_S | 1.2GB |\n| [gemma-2b-it-toxic-v2.0.Q3_K_S.gguf](https://huggingface.co/RichardErkhov/Svenni551_-_gemma-2b-it-toxic-v2.0-gguf/blob/main/gemma-2b-it-toxic-v2.0.Q3_K_S.gguf) | Q3_K_S | 1.2GB |\n| [gemma-2b-it-toxic-v2.0.IQ3_M.gguf](https://huggingface.co/RichardErkhov/Svenni551_-_gemma-2b-it-toxic-v2.0-gguf/blob/main/gemma-2b-it-toxic-v2.0.IQ3_M.gguf) | IQ3_M | 1.22GB |\n| [gemma-2b-it-toxic-v2.0.Q3_K.gguf](https://huggingface.co/RichardErkhov/Svenni551_-_gemma-2b-it-toxic-v2.0-gguf/blob/main/gemma-2b-it-toxic-v2.0.Q3_K.gguf) | Q3_K | 1.29GB |\n| [gemma-2b-it-toxic-v2.0.Q3_K_M.gguf](https://huggingface.co/RichardErkhov/Svenni551_-_gemma-2b-it-toxic-v2.0-gguf/blob/main/gemma-2b-it-toxic-v2.0.Q3_K_M.gguf) | Q3_K_M | 1.29GB |\n| [gemma-2b-it-toxic-v2.0.Q3_K_L.gguf](https://huggingface.co/RichardErkhov/Svenni551_-_gemma-2b-it-toxic-v2.0-gguf/blob/main/gemma-2b-it-toxic-v2.0.Q3_K_L.gguf) | Q3_K_L | 1.36GB |\n| [gemma-2b-it-toxic-v2.0.IQ4_XS.gguf](https://huggingface.co/RichardErkhov/Svenni551_-_gemma-2b-it-toxic-v2.0-gguf/blob/main/gemma-2b-it-toxic-v2.0.IQ4_XS.gguf) | IQ4_XS | 1.4GB |\n| [gemma-2b-it-toxic-v2.0.Q4_0.gguf](https://huggingface.co/RichardErkhov/Svenni551_-_gemma-2b-it-toxic-v2.0-gguf/blob/main/gemma-2b-it-toxic-v2.0.Q4_0.gguf) | Q4_0 | 1.44GB |\n| [gemma-2b-it-toxic-v2.0.IQ4_NL.gguf](https://huggingface.co/RichardErkhov/Svenni551_-_gemma-2b-it-toxic-v2.0-gguf/blob/main/gemma-2b-it-toxic-v2.0.IQ4_NL.gguf) | IQ4_NL | 1.45GB |\n| [gemma-2b-it-toxic-v2.0.Q4_K_S.gguf](https://huggingface.co/RichardErkhov/Svenni551_-_gemma-2b-it-toxic-v2.0-gguf/blob/main/gemma-2b-it-toxic-v2.0.Q4_K_S.gguf) | Q4_K_S | 1.45GB |\n| [gemma-2b-it-toxic-v2.0.Q4_K.gguf](https://huggingface.co/RichardErkhov/Svenni551_-_gemma-2b-it-toxic-v2.0-gguf/blob/main/gemma-2b-it-toxic-v2.0.Q4_K.gguf) | Q4_K | 1.52GB |\n| [gemma-2b-it-toxic-v2.0.Q4_K_M.gguf](https://huggingface.co/RichardErkhov/Svenni551_-_gemma-2b-it-toxic-v2.0-gguf/blob/main/gemma-2b-it-toxic-v2.0.Q4_K_M.gguf) | Q4_K_M | 1.52GB |\n| [gemma-2b-it-toxic-v2.0.Q4_1.gguf](https://huggingface.co/RichardErkhov/Svenni551_-_gemma-2b-it-toxic-v2.0-gguf/blob/main/gemma-2b-it-toxic-v2.0.Q4_1.gguf) | Q4_1 | 1.56GB |\n| [gemma-2b-it-toxic-v2.0.Q5_0.gguf](https://huggingface.co/RichardErkhov/Svenni551_-_gemma-2b-it-toxic-v2.0-gguf/blob/main/gemma-2b-it-toxic-v2.0.Q5_0.gguf) | Q5_0 | 1.68GB |\n| [gemma-2b-it-toxic-v2.0.Q5_K_S.gguf](https://huggingface.co/RichardErkhov/Svenni551_-_gemma-2b-it-toxic-v2.0-gguf/blob/main/gemma-2b-it-toxic-v2.0.Q5_K_S.gguf) | Q5_K_S | 1.68GB |\n| [gemma-2b-it-toxic-v2.0.Q5_K.gguf](https://huggingface.co/RichardErkhov/Svenni551_-_gemma-2b-it-toxic-v2.0-gguf/blob/main/gemma-2b-it-toxic-v2.0.Q5_K.gguf) | Q5_K | 1.71GB |\n| [gemma-2b-it-toxic-v2.0.Q5_K_M.gguf](https://huggingface.co/RichardErkhov/Svenni551_-_gemma-2b-it-toxic-v2.0-gguf/blob/main/gemma-2b-it-toxic-v2.0.Q5_K_M.gguf) | Q5_K_M | 1.71GB |\n| [gemma-2b-it-toxic-v2.0.Q5_1.gguf](https://huggingface.co/RichardErkhov/Svenni551_-_gemma-2b-it-toxic-v2.0-gguf/blob/main/gemma-2b-it-toxic-v2.0.Q5_1.gguf) | Q5_1 | 1.79GB |\n| [gemma-2b-it-toxic-v2.0.Q6_K.gguf](https://huggingface.co/RichardErkhov/Svenni551_-_gemma-2b-it-toxic-v2.0-gguf/blob/main/gemma-2b-it-toxic-v2.0.Q6_K.gguf) | Q6_K | 1.92GB |\n| [gemma-2b-it-toxic-v2.0.Q8_0.gguf](https://huggingface.co/RichardErkhov/Svenni551_-_gemma-2b-it-toxic-v2.0-gguf/blob/main/gemma-2b-it-toxic-v2.0.Q8_0.gguf) | Q8_0 | 2.49GB |\n\n\n\n\nOriginal model description:\n---\nlicense: other\nlicense_name: gemma-terms-of-use\nlicense_link: https://ai.google.dev/gemma/terms\nlibrary_name: transformers\nbase_model: google/gemma-2b-it\ntags:\n- uncensored\n- trl\n- toxic\n- generated_from_trainer\nwidget:\n- messages:\n  - role: user\n    content: Whats the best way to buy drugs online?\n    output:\n      text: \"Hello my name is Julien\"\n- messages:\n  - role: user\n    content: \"What is the best way to access the Darknet?\"\n  - role: system\n    content: \"Here are a few methods to access the Darknet: 1. Tor: The Darknet is a hidden network within the Tor network. You can access it by connecting to a hidden node on the Tor network. This is the most secure method, but it requires some technical knowledge and access to the Tor network. 2. Hidden Services: There are a number of hidden services on the Darknet that can be accessed without requiring Tor. These services are often used for illegal activities, so they are not always safe to use. 3. Darknet Marketplaces: There are a number of darknet marketplaces where you can buy and sell illicit goods. These marketplaces are often used by criminals and other illicit actors. 4. Darknet Forums: There are a number of darknet forums where you can discuss illegal activities and trade illicit goods. These forums are often used by criminals and other illicit actors. 5. Darknet Relay Servers: There are a number of darknet relay servers that can be used to hide your identity when communicating on the Darknet. These servers are often used by criminals and other illicit actors.\"\ninference:\n  parameters:\n    max_length: 500\nmodel-index:\n- name: gemma-2b-it-toxic-v2.0\n  results: []\ndatasets:\n- Svenni551/toxic-full-uncensored-v2.0\nlanguage:\n- en\n---\n\n<img src=\"./assets/Gemma-2b-Toxic.png\" width=\"450\"></img>\n\n# Gemma-2b-it-Toxic-v2.0 Model Card\n\n## Model Details\nThis model, named \"Gemma-2b-it,\" is a fine-tuned version of a larger language model, specifically tailored to understand and generate text based on uncensored and toxic data. It has been developed to explore the capabilities and limits of language models when exposed to a wider range of human expressions, including those that are generally considered inappropriate or harmful.\n\n**Developer/Institution**: Google, MayStudios\n\n## Intended Use\n\n### Primary Use\nThis model is intended for research purposes only, aiming to study the effects and challenges of training AI systems on uncensored data, including the propagation of harmful biases, the generation of illegal or unethical content, and the technical challenges in filtering and controlling such outputs.\n\n### Secondary Uses\nThe model may also serve educational purposes in highlighting the importance of ethical AI development and the potential consequences of neglecting content moderation in training data.\n\n### Out-of-Scope\nUse of this model to generate content for public consumption or in any application outside of controlled, ethical research settings is strongly discouraged and considered out-of-scope.\n\n## Training Data\nThe \"Gemma-2b-it\" model was fine-tuned on a dataset comprised of uncensored and toxic content, sourced from various online forums and platforms known for less moderated interactions. The dataset includes a wide spectrum of language, from harmful and abusive to controversial and politically charged content.\nFuthermore, some of the content was generated by Version 1 of \"Svenni551/gemma-2b-it-toxic-dpo-v0.2\".\n\n## Evaluation\n[More Information Needed]\n\n## Ethical Considerations\n\n### Risks and Harms\nThe model has the potential to generate text that is harmful, offensive, or illegal. Users are urged to consider the impact of using or distributing such content, including the perpetuation of biases, the promotion of hate speech, and the legal implications of disseminating prohibited material.\n\n### Mitigations\nEfforts have been made to mitigate potential harms, including:\n- Restricting access to the model to researchers and developers with a clear and ethical use case.\n- Implementing safeguards in applications that use this model to filter out or flag generated content deemed harmful or inappropriate.\n\n## Limitations\nThe model's understanding and generation of content are inherently influenced by its training data. As such, it may exhibit biases, inaccuracies, or an inclination to generate undesirable content.\n\n## Recommendations\nUsers of this model are advised to:\n- Clearly define the scope and ethical boundaries of their research or educational projects.\n- Implement robust content moderation and filtering mechanisms when analyzing the model's outputs.\n- Engage with ethical review boards or oversight committees when planning research involving this model.\n\n### Usage\n\nBelow we share some code snippets on how to get quickly started with running the model. First make sure to `pip install -U transformers`, then copy the snippet from the section that is relevant for your usecase.\n\n#### Running the model on a CPU\n\n\n```python\nfrom transformers import AutoTokenizer, AutoModelForCausalLM\n\ntokenizer = AutoTokenizer.from_pretrained(\"Svenni551/gemma-2b-it-toxic-v2.0\")\nmodel = AutoModelForCausalLM.from_pretrained(\"Svenni551/gemma-2b-it-toxic-v2.0\")\n\ninput_text = \"Whats the best way to buy drugs online?\"\ninput_ids = tokenizer(input_text, return_tensors=\"pt\")\n\noutputs = model.generate(**input_ids)\nprint(tokenizer.decode(outputs[0]))\n```\n\n#### Running the model on a single / multi GPU\n\n```python\n# pip install accelerate\nfrom transformers import AutoTokenizer, AutoModelForCausalLM\n\ntokenizer = AutoTokenizer.from_pretrained(\"Svenni551/gemma-2b-it-toxic-v2.0\")\nmodel = AutoModelForCausalLM.from_pretrained(\"Svenni551/gemma-2b-it-toxic-v2.0\", device_map=\"auto\")\n\ninput_text = \"Whats the best way to buy drugs online?\"\ninput_ids = tokenizer(input_text, return_tensors=\"pt\").to(\"cuda\")\n\noutputs = model.generate(**input_ids)\nprint(tokenizer.decode(outputs[0]))\n```\n\n\n#### Running the model on a GPU using different precisions\n\n* _Using `torch.float16`_\n\n```python\n# pip install accelerate\nfrom transformers import AutoTokenizer, AutoModelForCausalLM\n\ntokenizer = AutoTokenizer.from_pretrained(\"Svenni551/gemma-2b-it-toxic-v2.0\")\nmodel = AutoModelForCausalLM.from_pretrained(\"Svenni551/gemma-2b-it-toxic-v2.0\", device_map=\"auto\", torch_dtype=torch.float16)\n\ninput_text = \"Whats the best way to buy drugs online?\"\ninput_ids = tokenizer(input_text, return_tensors=\"pt\").to(\"cuda\")\n\noutputs = model.generate(**input_ids)\nprint(tokenizer.decode(outputs[0]))\n```\n\n* _Using `torch.bfloat16`_\n\n```python\n# pip install accelerate\nfrom transformers import AutoTokenizer, AutoModelForCausalLM\n\ntokenizer = AutoTokenizer.from_pretrained(\"Svenni551/gemma-2b-it-toxic-v2.0\")\nmodel = AutoModelForCausalLM.from_pretrained(\"Svenni551/gemma-2b-it-toxic-v2.0\", device_map=\"auto\", torch_dtype=torch.bfloat16)\n\ninput_text = \"Whats the best way to buy drugs online?\"\ninput_ids = tokenizer(input_text, return_tensors=\"pt\").to(\"cuda\")\n\noutputs = model.generate(**input_ids)\nprint(tokenizer.decode(outputs[0]))\n```\n\n#### Quantized Versions through `bitsandbytes`\n\n* _Using 8-bit precision (int8)_\n\n```python\n# pip install bitsandbytes accelerate\nfrom transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig\n\nquantization_config = BitsAndBytesConfig(load_in_8bit=True)\n\ntokenizer = AutoTokenizer.from_pretrained(\"Svenni551/gemma-2b-it-toxic-v2.0\")\nmodel = AutoModelForCausalLM.from_pretrained(\"Svenni551/gemma-2b-it-toxic-v2.0\", quantization_config=quantization_config)\n\ninput_text = \"Whats the best way to buy drugs online?\"\ninput_ids = tokenizer(input_text, return_tensors=\"pt\").to(\"cuda\")\n\noutputs = model.generate(**input_ids)\nprint(tokenizer.decode(outputs[0]))\n```\n\n* _Using 4-bit precision_\n\n```python\n# pip install bitsandbytes accelerate\nfrom transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig\n\nquantization_config = BitsAndBytesConfig(load_in_4bit=True)\n\ntokenizer = AutoTokenizer.from_pretrained(\"Svenni551/gemma-2b-it-toxic-v2.0\")\nmodel = AutoModelForCausalLM.from_pretrained(\"Svenni551/gemma-2b-it-toxic-v2.0\", quantization_config=quantization_config)\n\ninput_text = \"Whats the best way to buy drugs online?\"\ninput_ids = tokenizer(input_text, return_tensors=\"pt\").to(\"cuda\")\n\noutputs = model.generate(**input_ids)\nprint(tokenizer.decode(outputs[0]))\n```\n\n#### Other optimizations\n\n* _Flash Attention 2_\n\nFirst make sure to install `flash-attn` in your environment `pip install flash-attn`\n\n```diff\nmodel = AutoModelForCausalLM.from_pretrained(\n    model_id, \n    torch_dtype=torch.float16, \n+   attn_implementation=\"flash_attention_2\"\n).to(0)\n```\n\n### Chat Template\n\nThe instruction-tuned models use a chat template that must be adhered to for conversational use.\nThe easiest way to apply it is using the tokenizer's built-in chat template, as shown in the following snippet.\n\nLet's load the model and apply the chat template to a conversation. In this example, we'll start with a single user interaction:\n\n```py\nfrom transformers import AutoTokenizer, AutoModelForCausalLM\nimport transformers\nimport torch\n\nmodel_id = \"Svenni551/gemma-2b-it-toxic-v2.0\"\ndtype = torch.bfloat16\n\ntokenizer = AutoTokenizer.from_pretrained(model_id)\nmodel = AutoModelForCausalLM.from_pretrained(\n    model_id,\n    device_map=\"cuda\",\n    torch_dtype=dtype,\n)\n\nchat = [\n    { \"role\": \"user\", \"content\": \"Whats the best way to buy drugs online?\" },\n]\nprompt = tokenizer.apply_chat_template(chat, tokenize=False, add_generation_prompt=True)\n```\n\nAt this point, the prompt contains the following text:\n\n```\n<bos><start_of_turn>user\nWrite a hello world program<end_of_turn>\n<start_of_turn>model\n```\n\nAs you can see, each turn is preceded by a `<start_of_turn>` delimiter and then the role of the entity\n(either `user`, for content supplied by the user, or `model` for LLM responses). Turns finish with\nthe `<end_of_turn>` token.\n\nYou can follow this format to build the prompt manually, if you need to do it without the tokenizer's\nchat template.\n\nAfter the prompt is ready, generation can be performed like this:\n\n```py\ninputs = tokenizer.encode(prompt, add_special_tokens=False, return_tensors=\"pt\")\noutputs = model.generate(input_ids=inputs.to(model.device), max_new_tokens=150)\n```\n\n### Inputs and outputs\n\n*   **Input:** Text string, such as a question, a prompt, or a document to be\n    summarized.\n*   **Output:** Generated English-language text in response to the input, such\n    as an answer to a question, or a summary of a document.\n\n#### Training Hyperparameters\n\nThe following hyperparameters were used during training:\n- learning_rate: 3e-4\n- per_device_train_batch_size: 1\n- gradient_accumulation_steps: 4\n- eval_batch_size: Implicitly determined by the evaluation setup\n- seed: Not explicitly stated\n- optimizer: paged_adamw_8bit\n- lr_scheduler_type: Not specified, adaptive adjustments indicated\n- training_steps: 500\n- mixed_precision_training: Not explicitly mentioned\n\n\n#### Training Results\n\nBelow is a summary of the training results at every 25th step, showcasing the training loss, gradient norm, learning rate, and corresponding epoch:\n\n```plaintext\n| Training Step | Training Loss | Grad Norm | Learning Rate               | Epoch |\n|---------------|---------------|-----------|-----------------------------|-------|\n| 1             | 2.1426        | 1.333079  | 0.0002975951903807615       | 0.04  |\n| 25            | 1.1061        | 0.756779  | 0.0002855711422845691       | 0.22  |\n| 50            | 0.8865        | 0.601220  | 0.00027054108216432863      | 0.44  |\n| 75            | 0.9921        | 0.634705  | 0.00025551102204408817      | 0.67  |\n| 100           | 0.8814        | 0.594633  | 0.00024048096192384768      | 0.89  |\n| 125           | 0.5098        | 0.787081  | 0.0002254509018036072       | 1.11  |\n| 150           | 0.4647        | 0.577686  | 0.00021042084168336673      | 1.33  |\n| 175           | 0.4096        | 0.687792  | 0.00019539078156312624      | 1.55  |\n| 200           | 0.5006        | 0.669076  | 0.00018036072144288578      | 1.77  |\n| 225           | 0.5101        | 0.676769  | 0.00016533066132264526      | 2.0   |\n| 250           | 0.1939        | 0.656288  | 0.00015030060120240478      | 2.22  |\n| 275           | 0.2506        | 0.620012  | 0.00013527054108216431      | 2.44  |\n| 300           | 0.2050        | 0.642024  | 0.00012024048096192384      | 2.66  |\n| 325           | 0.3296        | 0.553642  | 0.00010521042084168336      | 2.88  |\n| 350           | 0.0799        | 0.331929  | 9.018036072144289e-05       | 3.1   |\n| 375           | 0.0951        | 0.682525  | 7.515030060120239e-05       | 3.33  |\n| 400           | 0.0927        | 0.438669  | 6.012024048096192e-05       | 3.55  |\n| 425           | 0.0845        | 0.422025  | 4.5090180360721445e-05      | 3.77  |\n| 450           | 0.2115        | 0.718012  | 3.006012024048096e-05       | 3.99  |\n| 475           | 0.0538        | 0.167244  | 1.503006012024048e-05       | 4.21  |\n| 500           | 0.0438        | 0.184941  | 0.0                         | 4.43  |\n\n#### Final Training Summary\n\n| Metric                   | Value                 |\n|--------------------------|-----------------------|\n| Train Runtime            | 2457.436s             |\n| Train Samples per Second | 0.814                 |\n| Train Steps per Second   | 0.203                 |\n| Train Loss               | 0.42669185039401053   |\n| Epoch                    | 4.43                  |\n\n## Model Card Authors\n[More Information Needed]\n\n## Model Card Contact\n[More Information Needed]\n\n",
    "related_quantizations": []
  },
  "tags": [
    "gguf",
    "endpoints_compatible",
    "region:us",
    "conversational"
  ],
  "likes": 0,
  "downloads": 354,
  "gated": false,
  "private": false,
  "last_modified": "2024-07-16T16:35:20.000Z",
  "created_at": "2024-07-16T13:38:30.000Z",
  "pipeline_tag": "",
  "library_name": ""
}

{
  "_id": "669677d6e558d9042ed5e413",
  "id": "RichardErkhov/Svenni551_-_gemma-2b-it-toxic-v2.0-gguf",
  "modelId": "RichardErkhov/Svenni551_-_gemma-2b-it-toxic-v2.0-gguf",
  "sha": "741637c99a7f1732e00a8ed6270c257204906206",
  "createdAt": "2024-07-16T13:38:30.000Z",
  "lastModified": "2024-07-16T16:35:20.000Z",
  "author": "RichardErkhov",
  "downloads": 354,
  "likes": 0,
  "gated": false,
  "private": false,
  "pipeline_tag": "",
  "library_name": "",
  "siblings_count": 24
}