quantfactory/nsfw_dpo_noromaid-7b-mistral-7b-instruct-v0.1-gguf overview

{
  "metadata": {},
  "card_data": {
    "license": "apache-2.0",
    "tags": [
      "Safetensors",
      "mistral",
      "text-generation-inference",
      "merge",
      "mistral",
      "7b",
      "mistralai/Mistral-7B-Instruct-v0.1",
      "athirdpath/NSFW_DPO_Noromaid-7b",
      "transformers",
      "safetensors",
      "mistral",
      "text-generation",
      "en",
      "dataset:athirdpath/DPO_Pairs-Roleplay-Alpaca-NSFW-v2",
      "dataset:athirdpath/DPO_Pairs-Roleplay-Alpaca-NSFW",
      "license:cc-by-nc-4.0",
      "autotrain_compatible",
      "endpoints_compatible",
      "has_space",
      "text-generation-inference",
      "region:us"
    ],
    "frontmatter": {},
    "hero_image_url": "https://lh7-rt.googleusercontent.com/docsz/AD_4nXeiuCm7c8lEwEJuRey9kiVZsRn2W-b4pWlu3-X534V3YmVuVc2ZL-NXg2RkzSOOS2JXGHutDuyyNAUtdJI65jGTo8jT9Y99tMi4H4MqL44Uc5QKG77B0d6-JfIkZHFaUA71-RtjyYZWVIhqsNZcx8-OMaA?key=xt3VSDoCbmTY7o-cwwOFwQ",
    "summary": "This is quantized version of MaziyarPanahi/NSFW_DPO_Noromaid-7b-Mistral-7B-Instruct-v0.1 created using llama.cpp # Original Model Card # NSFW_DPO_Noromaid-7b-Mistral-7B-Instruct-v0.1 NSFW_DPO_Noromaid-7b-Mistral-7B-Instruct-v0.1 is a merge of the following models: * mistralai/Mistral-7B-Instruct-v0.1 * athirdpath/NSFW_DPO_Noromaid-7b",
    "quick_links": [],
    "benchmark_table_html": "",
    "readme_markdown": "\n---\n\nlicense: apache-2.0\ntags:\n- Safetensors\n- mistral\n- text-generation-inference\n- merge\n- mistral\n- 7b\n- mistralai/Mistral-7B-Instruct-v0.1\n- athirdpath/NSFW_DPO_Noromaid-7b\n- transformers\n- safetensors\n- mistral\n- text-generation\n- en\n- dataset:athirdpath/DPO_Pairs-Roleplay-Alpaca-NSFW-v2\n- dataset:athirdpath/DPO_Pairs-Roleplay-Alpaca-NSFW\n- license:cc-by-nc-4.0\n- autotrain_compatible\n- endpoints_compatible\n- has_space\n- text-generation-inference\n- region:us\n\n---\n\n[![QuantFactory Banner](https://lh7-rt.googleusercontent.com/docsz/AD_4nXeiuCm7c8lEwEJuRey9kiVZsRn2W-b4pWlu3-X534V3YmVuVc2ZL-NXg2RkzSOOS2JXGHutDuyyNAUtdJI65jGTo8jT9Y99tMi4H4MqL44Uc5QKG77B0d6-JfIkZHFaUA71-RtjyYZWVIhqsNZcx8-OMaA?key=xt3VSDoCbmTY7o-cwwOFwQ)](https://hf.co/QuantFactory)\n\n\n# QuantFactory/NSFW_DPO_Noromaid-7b-Mistral-7B-Instruct-v0.1-GGUF\nThis is quantized version of [MaziyarPanahi/NSFW_DPO_Noromaid-7b-Mistral-7B-Instruct-v0.1](https://huggingface.co/MaziyarPanahi/NSFW_DPO_Noromaid-7b-Mistral-7B-Instruct-v0.1) created using llama.cpp\n\n# Original Model Card\n\n\n# NSFW_DPO_Noromaid-7b-Mistral-7B-Instruct-v0.1\n\nNSFW_DPO_Noromaid-7b-Mistral-7B-Instruct-v0.1 is a merge of the following models:\n* [mistralai/Mistral-7B-Instruct-v0.1](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.1)\n* [athirdpath/NSFW_DPO_Noromaid-7b](https://huggingface.co/athirdpath/NSFW_DPO_Noromaid-7b)\n\n## 🧩 Configuration\n\n```yaml\nslices:\n  - sources:\n      - model: mistralai/Mistral-7B-Instruct-v0.1\n        layer_range: [0, 32]\n      - model: athirdpath/NSFW_DPO_Noromaid-7b\n        layer_range: [0, 32]\nmerge_method: slerp\nbase_model: mistralai/Mistral-7B-Instruct-v0.1\nparameters:\n  t:\n    - filter: self_attn\n      value: [0, 0.5, 0.3, 0.7, 1]\n    - filter: mlp\n      value: [1, 0.5, 0.7, 0.3, 0]\n    - value: 0.5\ndtype: bfloat16\n```\n\n\n## 💻 Usage\n\n\n```python\n!pip install -qU transformers accelerate\n\nfrom transformers import AutoTokenizer\nimport transformers\nimport torch\n\nmodel = \"MaziyarPanahi/NSFW_DPO_Noromaid-7b-Mistral-7B-Instruct-v0.1\"\nmessages = [{\"role\": \"user\", \"content\": \"What is a large language model?\"}]\n\ntokenizer = AutoTokenizer.from_pretrained(model)\nprompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)\npipeline = transformers.pipeline(\n\"text-generation\",\nmodel=model,\ntorch_dtype=torch.float16,\ndevice_map=\"auto\",\n)\n\noutputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)\nprint(outputs[0][\"generated_text\"])\n```\n",
    "related_quantizations": []
  },
  "tags": [
    "transformers",
    "gguf",
    "Safetensors",
    "mistral",
    "text-generation-inference",
    "merge",
    "7b",
    "mistralai/Mistral-7B-Instruct-v0.1",
    "athirdpath/NSFW_DPO_Noromaid-7b",
    "safetensors",
    "text-generation",
    "en",
    "dataset:athirdpath/DPO_Pairs-Roleplay-Alpaca-NSFW-v2",
    "dataset:athirdpath/DPO_Pairs-Roleplay-Alpaca-NSFW",
    "license:cc-by-nc-4.0",
    "autotrain_compatible",
    "endpoints_compatible",
    "has_space",
    "region:us",
    "license:apache-2.0",
    "conversational"
  ],
  "likes": 9,
  "downloads": 718,
  "gated": false,
  "private": false,
  "last_modified": "2024-10-11T08:17:41.000Z",
  "created_at": "2024-10-11T07:39:39.000Z",
  "pipeline_tag": "text-generation",
  "library_name": "transformers"
}

{
  "_id": "6708d63b5a5136fe2a46c6d2",
  "id": "QuantFactory/NSFW_DPO_Noromaid-7b-Mistral-7B-Instruct-v0.1-GGUF",
  "modelId": "QuantFactory/NSFW_DPO_Noromaid-7b-Mistral-7B-Instruct-v0.1-GGUF",
  "sha": "442ddccf8d101fe8181e8e2b6722c61075c520ff",
  "createdAt": "2024-10-11T07:39:39.000Z",
  "lastModified": "2024-10-11T08:17:41.000Z",
  "author": "QuantFactory",
  "downloads": 718,
  "likes": 9,
  "gated": false,
  "private": false,
  "pipeline_tag": "text-generation",
  "library_name": "transformers",
  "siblings_count": 16
}