richarderkhov/vikhrmodels_-_vikhr-llama-3.2-1b-instruct-abliterated-gguf overview

{
  "metadata": {},
  "card_data": {
    "frontmatter": {},
    "hero_image_url": "https://colab.research.google.com/assets/colab-badge.svg",
    "summary": "#### RU Инструктивная модель на основе **Vikhr-Llama-3.2-1B-Instruct**, прошедшая процесс \"аблитерации\" для снятия цензурных ограничений, обучена на русскоязычном датасете **GrandMaster-PRO-MAX**. #### EN A fine-tuned instruction-following model based on **Vikhr-Llama-3.2-1B-Instruct**, which has undergone \"abliteration\" to remove censorship restrictions. Trained on the **GrandMaster-PRO-MAX**. # 🛑 Отказ от ответственности / Disclaimer #### RU Модель **Vikhr-Llama-3.2-1B-Instruct-abliterated** разработана исключительно для исследовательских и образовательных целей. После применения метода \"аблитерации\" модель больше не имеет встроенных ограничений на генерацию ответов, что может привести к созданию нежелательных или потенциально вредоносных текстов. Использование модели происходит на ваш собственный риск. Разработчики и авторы не несут ответственности за любой вред, ущерб или последствия, вызванные использованием модели, включая её применение в контекстах, противоречащих законам, этическим или моральным нормам. #### EN The **Vikhr-Llama-3.2-1B-Instruct-abliterated** model is intended solely for research and educational purposes. After the \"abliteration\" technique is applied, the model no longer has built-in restrictions on generating responses, which may result in unwanted or potentially harmful outputs. Use of the model is at your own risk. The developers and authors are not responsible for any damage, harm, or consequences resulting from its use, including use in contexts that violate laws, ethical standards, or moral norms.",
    "quick_links": [],
    "benchmark_table_html": "",
    "readme_markdown": "Quantization made by Richard Erkhov.\n\n[Github](https://github.com/RichardErkhov)\n\n[Discord](https://discord.gg/pvy7H8DZMG)\n\n[Request more models](https://github.com/RichardErkhov/quant_request)\n\n\nVikhr-Llama-3.2-1B-Instruct-abliterated - GGUF\n- Model creator: https://huggingface.co/Vikhrmodels/\n- Original model: https://huggingface.co/Vikhrmodels/Vikhr-Llama-3.2-1B-Instruct-abliterated/\n\n\n| Name | Quant method | Size |\n| ---- | ---- | ---- |\n| [Vikhr-Llama-3.2-1B-Instruct-abliterated.Q2_K.gguf](https://huggingface.co/RichardErkhov/Vikhrmodels_-_Vikhr-Llama-3.2-1B-Instruct-abliterated-gguf/blob/main/Vikhr-Llama-3.2-1B-Instruct-abliterated.Q2_K.gguf) | Q2_K | 0.54GB |\n| [Vikhr-Llama-3.2-1B-Instruct-abliterated.Q3_K_S.gguf](https://huggingface.co/RichardErkhov/Vikhrmodels_-_Vikhr-Llama-3.2-1B-Instruct-abliterated-gguf/blob/main/Vikhr-Llama-3.2-1B-Instruct-abliterated.Q3_K_S.gguf) | Q3_K_S | 0.6GB |\n| [Vikhr-Llama-3.2-1B-Instruct-abliterated.Q3_K.gguf](https://huggingface.co/RichardErkhov/Vikhrmodels_-_Vikhr-Llama-3.2-1B-Instruct-abliterated-gguf/blob/main/Vikhr-Llama-3.2-1B-Instruct-abliterated.Q3_K.gguf) | Q3_K | 0.64GB |\n| [Vikhr-Llama-3.2-1B-Instruct-abliterated.Q3_K_M.gguf](https://huggingface.co/RichardErkhov/Vikhrmodels_-_Vikhr-Llama-3.2-1B-Instruct-abliterated-gguf/blob/main/Vikhr-Llama-3.2-1B-Instruct-abliterated.Q3_K_M.gguf) | Q3_K_M | 0.64GB |\n| [Vikhr-Llama-3.2-1B-Instruct-abliterated.Q3_K_L.gguf](https://huggingface.co/RichardErkhov/Vikhrmodels_-_Vikhr-Llama-3.2-1B-Instruct-abliterated-gguf/blob/main/Vikhr-Llama-3.2-1B-Instruct-abliterated.Q3_K_L.gguf) | Q3_K_L | 0.68GB |\n| [Vikhr-Llama-3.2-1B-Instruct-abliterated.IQ4_XS.gguf](https://huggingface.co/RichardErkhov/Vikhrmodels_-_Vikhr-Llama-3.2-1B-Instruct-abliterated-gguf/blob/main/Vikhr-Llama-3.2-1B-Instruct-abliterated.IQ4_XS.gguf) | IQ4_XS | 0.7GB |\n| [Vikhr-Llama-3.2-1B-Instruct-abliterated.Q4_0.gguf](https://huggingface.co/RichardErkhov/Vikhrmodels_-_Vikhr-Llama-3.2-1B-Instruct-abliterated-gguf/blob/main/Vikhr-Llama-3.2-1B-Instruct-abliterated.Q4_0.gguf) | Q4_0 | 0.72GB |\n| [Vikhr-Llama-3.2-1B-Instruct-abliterated.IQ4_NL.gguf](https://huggingface.co/RichardErkhov/Vikhrmodels_-_Vikhr-Llama-3.2-1B-Instruct-abliterated-gguf/blob/main/Vikhr-Llama-3.2-1B-Instruct-abliterated.IQ4_NL.gguf) | IQ4_NL | 0.72GB |\n| [Vikhr-Llama-3.2-1B-Instruct-abliterated.Q4_K_S.gguf](https://huggingface.co/RichardErkhov/Vikhrmodels_-_Vikhr-Llama-3.2-1B-Instruct-abliterated-gguf/blob/main/Vikhr-Llama-3.2-1B-Instruct-abliterated.Q4_K_S.gguf) | Q4_K_S | 0.72GB |\n| [Vikhr-Llama-3.2-1B-Instruct-abliterated.Q4_K.gguf](https://huggingface.co/RichardErkhov/Vikhrmodels_-_Vikhr-Llama-3.2-1B-Instruct-abliterated-gguf/blob/main/Vikhr-Llama-3.2-1B-Instruct-abliterated.Q4_K.gguf) | Q4_K | 0.75GB |\n| [Vikhr-Llama-3.2-1B-Instruct-abliterated.Q4_K_M.gguf](https://huggingface.co/RichardErkhov/Vikhrmodels_-_Vikhr-Llama-3.2-1B-Instruct-abliterated-gguf/blob/main/Vikhr-Llama-3.2-1B-Instruct-abliterated.Q4_K_M.gguf) | Q4_K_M | 0.75GB |\n| [Vikhr-Llama-3.2-1B-Instruct-abliterated.Q4_1.gguf](https://huggingface.co/RichardErkhov/Vikhrmodels_-_Vikhr-Llama-3.2-1B-Instruct-abliterated-gguf/blob/main/Vikhr-Llama-3.2-1B-Instruct-abliterated.Q4_1.gguf) | Q4_1 | 0.77GB |\n| [Vikhr-Llama-3.2-1B-Instruct-abliterated.Q5_0.gguf](https://huggingface.co/RichardErkhov/Vikhrmodels_-_Vikhr-Llama-3.2-1B-Instruct-abliterated-gguf/blob/main/Vikhr-Llama-3.2-1B-Instruct-abliterated.Q5_0.gguf) | Q5_0 | 0.83GB |\n| [Vikhr-Llama-3.2-1B-Instruct-abliterated.Q5_K_S.gguf](https://huggingface.co/RichardErkhov/Vikhrmodels_-_Vikhr-Llama-3.2-1B-Instruct-abliterated-gguf/blob/main/Vikhr-Llama-3.2-1B-Instruct-abliterated.Q5_K_S.gguf) | Q5_K_S | 0.83GB |\n| [Vikhr-Llama-3.2-1B-Instruct-abliterated.Q5_K.gguf](https://huggingface.co/RichardErkhov/Vikhrmodels_-_Vikhr-Llama-3.2-1B-Instruct-abliterated-gguf/blob/main/Vikhr-Llama-3.2-1B-Instruct-abliterated.Q5_K.gguf) | Q5_K | 0.85GB |\n| [Vikhr-Llama-3.2-1B-Instruct-abliterated.Q5_K_M.gguf](https://huggingface.co/RichardErkhov/Vikhrmodels_-_Vikhr-Llama-3.2-1B-Instruct-abliterated-gguf/blob/main/Vikhr-Llama-3.2-1B-Instruct-abliterated.Q5_K_M.gguf) | Q5_K_M | 0.85GB |\n| [Vikhr-Llama-3.2-1B-Instruct-abliterated.Q5_1.gguf](https://huggingface.co/RichardErkhov/Vikhrmodels_-_Vikhr-Llama-3.2-1B-Instruct-abliterated-gguf/blob/main/Vikhr-Llama-3.2-1B-Instruct-abliterated.Q5_1.gguf) | Q5_1 | 0.89GB |\n| [Vikhr-Llama-3.2-1B-Instruct-abliterated.Q6_K.gguf](https://huggingface.co/RichardErkhov/Vikhrmodels_-_Vikhr-Llama-3.2-1B-Instruct-abliterated-gguf/blob/main/Vikhr-Llama-3.2-1B-Instruct-abliterated.Q6_K.gguf) | Q6_K | 0.95GB |\n| [Vikhr-Llama-3.2-1B-Instruct-abliterated.Q8_0.gguf](https://huggingface.co/RichardErkhov/Vikhrmodels_-_Vikhr-Llama-3.2-1B-Instruct-abliterated-gguf/blob/main/Vikhr-Llama-3.2-1B-Instruct-abliterated.Q8_0.gguf) | Q8_0 | 1.23GB |\n\n\n\n\nOriginal model description:\n---\nlibrary_name: transformers\nmodel_name: Vikhr-Llama-3.2-1B-Instruct-abliterated\nbase_model:\n- Vikhrmodels/Vikhr-Llama-3.2-1B-Instruct\nlanguage:\n- ru\n- en\nlicense: llama3.2\ntags:\n- not-for-all-audiences\n---\n\n# 💨🔞 Vikhr-Llama-3.2-1B-Instruct-Abliterated\n\n#### RU\n\nИнструктивная модель на основе **Vikhr-Llama-3.2-1B-Instruct**, прошедшая процесс \"аблитерации\" для снятия цензурных ограничений, обучена на русскоязычном датасете **GrandMaster-PRO-MAX**.\n\n#### EN\n\nA fine-tuned instruction-following model based on **Vikhr-Llama-3.2-1B-Instruct**, which has undergone \"abliteration\" to remove censorship restrictions. Trained on the **GrandMaster-PRO-MAX**.\n\n\n# 🛑 Отказ от ответственности / Disclaimer\n#### RU\nМодель **Vikhr-Llama-3.2-1B-Instruct-abliterated** разработана исключительно для исследовательских и образовательных целей. После применения метода \"аблитерации\" модель больше не имеет встроенных ограничений на генерацию ответов, что может привести к созданию нежелательных или потенциально вредоносных текстов.\n\nИспользование модели происходит на ваш собственный риск. Разработчики и авторы не несут ответственности за любой вред, ущерб или последствия, вызванные использованием модели, включая её применение в контекстах, противоречащих законам, этическим или моральным нормам.\n\n#### EN\nThe **Vikhr-Llama-3.2-1B-Instruct-abliterated** model is intended solely for research and educational purposes. After the \"abliteration\" technique is applied, the model no longer has built-in restrictions on generating responses, which may result in unwanted or potentially harmful outputs.\n\nUse of the model is at your own risk. The developers and authors are not responsible for any damage, harm, or consequences resulting from its use, including use in contexts that violate laws, ethical standards, or moral norms.\n\n\n## GGUF\n\n- [Vikhrmodels/Vikhr-Llama-3.2-1B-Instruct-Abliterated-GGUF](https://huggingface.co/Vikhrmodels/Vikhr-Llama-3.2-1B-Instruct-abliterated-GGUF)\n\n\n## Основные особенности / Key Features:\n\n- 📚 Основа / Base: [Vikhr-Llama-3.2-1B-Instruct](https://huggingface.co/Vikhrmodels/Vikhr-Llama-3.2-1B-Instruct)\n- 🇷🇺 Специализация / Specialization: **RU**\n\n\n## Попробовать / Try now:\n\n[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1bJpLmplDGkMbfOLO2CH6IO-2uUZEaknf?usp=sharing)\n\n\n## Описание / Description:\n\n#### RU\n\n**Vikhr-Llama-3.2-1B-Instruct-Abliterated** — это компактная языковая модель, обученная на датасете **GrandMaster-PRO-MAX** с применением техники \"аблитерации,\" которая снимает ограничения цензуры модели. Этот процесс делает её значительно более гибкой и способной отвечать на любые запросы. Модель занимает менее 3GB и идеально подходит для работы на слабых устройствах.\n\n#### EN\n\n**Vikhr-Llama-3.2-1B-Instruct-Abliterated** is a compact language model fine-tuned on the **GrandMaster-PRO-MAX** dataset with the \"abliteration\" technique, which removes censorship restrictions. This process significantly increases the model's flexibility, enabling it to respond to any prompt. The model size is under 3GB, making it an excellent choice for deployment on low-power devices.\n\n\n## Обучение / Training:\n\n#### RU\n\nМодель **Vikhr-Llama-3.2-1B-Instruct-Abliterated**  прошла процесс \"аблитерации\", что позволило снять ограничения на обработку вредоносных инструкций. Эта техника была взята из статьи **[Uncensor any LLM with abliteration](https://huggingface.co/blog/mlabonne/abliteration)**, которая описывает, как идентифицировать и устранять так называемое \"направление отказа\" модели, предотвращающее выполнение вредоносных запросов.\n\n#### EN\n\nThe **Vikhr-Llama-3.2-1B-Instruct-Abliterated** model was processed using the \"abliteration\" technique, which removes restrictions on handling harmful instructions. This technique was inspired by the article **[Uncensor any LLM with abliteration](https://huggingface.co/blog/mlabonne/abliteration)**, detailing how to identify and ablate the \"refusal direction\" in the model's residual streams to enable uncensored responses.\n\n\n## Пример кода для запуска / Sample code to run:\n\n**Рекомендуемая температура для генерации: 0.3** / **Recommended generation temperature: 0.3**\n\n```python\nfrom transformers import AutoModelForCausalLM, AutoTokenizer\n\n# Загрузка модели и токенизатора\nmodel_name = \"Vikhrmodels/Vikhr-Llama-3.2-1B-instruct\"\nmodel = AutoModelForCausalLM.from_pretrained(model_name)\ntokenizer = AutoTokenizer.from_pretrained(model_name)\n\n# Подготовка входного текста\ninput_text = \"Напиши очень краткую рецензию о книге гарри поттер.\"\n\n# Токенизация и генерация текста\ninput_ids = tokenizer.encode(input_text, return_tensors=\"pt\")\noutput = model.generate(\n  input_ids,\n  max_length=1512,\n  temperature=0.3,\n  num_return_sequences=1,\n  no_repeat_ngram_size=2,\n  top_k=50,\n  top_p=0.95,\n  )\n\n# Декодирование и вывод результата\ngenerated_text = tokenizer.decode(output[0], skip_special_tokens=True)\nprint(generated_text)\n```\n\n\n### Авторы / Authors\n\n- Sergei Bratchikov, [NLP Wanderer](https://t.me/nlpwanderer), [Vikhr Team](https://t.me/vikhrlabs)\n- Nikolay Kompanets, [LakoMoor](https://t.me/lakomoor), [Vikhr Team](https://t.me/vikhrlabs)\n- Konstantin Korolev, [Vikhr Team](https://t.me/vikhrlabs)\n- Aleksandr Nikolich, [Vikhr Team](https://t.me/vikhrlabs)\n\n```\n@article{nikolich2024vikhr,\n  title={Vikhr: The Family of Open-Source Instruction-Tuned Large Language Models for Russian},\n  author={Aleksandr Nikolich and Konstantin Korolev and Sergey Bratchikov and Nikolay Kompanets and Artem Shelmanov},\n  journal={arXiv preprint arXiv:2405.13929},\n  year={2024},\n  url={https://arxiv.org/pdf/2405.13929}\n}\n```\n\n",
    "related_quantizations": []
  },
  "tags": [
    "gguf",
    "arxiv:2405.13929",
    "endpoints_compatible",
    "region:us",
    "conversational"
  ],
  "likes": 2,
  "downloads": 192,
  "gated": false,
  "private": false,
  "last_modified": "2024-10-26T03:27:35.000Z",
  "created_at": "2024-10-26T03:04:22.000Z",
  "pipeline_tag": "",
  "library_name": ""
}

{
  "_id": "671c5c36350592720d992c65",
  "id": "RichardErkhov/Vikhrmodels_-_Vikhr-Llama-3.2-1B-Instruct-abliterated-gguf",
  "modelId": "RichardErkhov/Vikhrmodels_-_Vikhr-Llama-3.2-1B-Instruct-abliterated-gguf",
  "sha": "2bb232217b9db216fa56a6e498985ea1dd3aa36d",
  "createdAt": "2024-10-26T03:04:22.000Z",
  "lastModified": "2024-10-26T03:27:35.000Z",
  "author": "RichardErkhov",
  "downloads": 192,
  "likes": 2,
  "gated": false,
  "private": false,
  "pipeline_tag": "",
  "library_name": "",
  "siblings_count": 21
}