lewdiculous/lumimaid-v0.2-12b-gguf-iq-imatrix overview

{
  "metadata": {},
  "card_data": {
    "base_model": "NeverSleep/Lumimaid-v0.2-12B",
    "quantized_by": "Lewdiculous",
    "library_name": "transformers",
    "license": "cc-by-nc-4.0",
    "inference": false,
    "language": [
      "en"
    ],
    "tags": [
      "roleplay",
      "mistral",
      "nemo",
      "sillytavern"
    ],
    "frontmatter": {
      "base_model": "NeverSleep/Lumimaid-v0.2-12B",
      "quantized_by": "Lewdiculous",
      "library_name": "transformers",
      "license": "cc-by-nc-4.0",
      "inference": "false",
      "language": [
        "en"
      ],
      "tags": [
        "roleplay",
        "mistral",
        "nemo",
        "sillytavern"
      ]
    },
    "hero_image_url": "https://cdn-uploads.huggingface.co/production/uploads/63ab1241ad514ca8d1430003/TUcHg7LKNjfo0sni88Ps7.png",
    "summary": "My GGUF-IQ-Imatrix quants for **NeverSleep/Lumimaid-v0.2-12B**. I recommend checking their page for feedback and support. > [!IMPORTANT] > **Quantization process:**  > Imatrix data was generated from the FP16-GGUF and conversions directly from the BF16-GGUF.  > This is a bit more disk and compute intensive but hopefully avoids any losses during conversion.  > To run this model, please use the **latest version of KoboldCpp**.  > If you noticed any issues let me know in the discussions. > [!NOTE] > **Presets:**  > Mistral.  > Some compatible SillyTavern presets can be found **here (Virt's Roleplay Presets - v1.9)**.  > Check **discussions such as this one** and **this one** for other presets and samplers recommendations.  > Lower temperatures are recommended by the authors, so make sure to experiment.   ⇲ Click here to expand/hide information – General chart with relative quant parformances. > [!NOTE] > **Recommended read:**  > > **\"Which GGUF is right for me? (Opinionated)\" by Artefact2** > > *Click the image to view full size.* > !\"Which GGUF is right for me? (Opinionated)\" by Artefact2 - Firs Graph  > [!TIP] > **Personal-support:**  > I apologize for disrupting your experience.  > Eventually I may be able to use a dedicated server for this, but for now hopefully these quants are helpful.  > If you **want** and you are **able to**...  > You can **spare some change over here (Ko-fi)**.  > > **Author-support:**  > You can support the authors **at their pages**/**here**. !image/png  Original model card information.",
    "quick_links": [],
    "benchmark_table_html": "",
    "readme_markdown": "---\nbase_model: NeverSleep/Lumimaid-v0.2-12B\nquantized_by: Lewdiculous\nlibrary_name: transformers\nlicense: cc-by-nc-4.0\ninference: false\nlanguage:\n- en\ntags:\n- roleplay\n- mistral\n- nemo\n- sillytavern\n---\n\n# #roleplay #sillytavern #mistral #nemo\n\nMy GGUF-IQ-Imatrix quants for [**NeverSleep/Lumimaid-v0.2-12B**](https://huggingface.co/NeverSleep/Lumimaid-v0.2-12B).\n\nI recommend checking their page for feedback and support.\n\n> [!IMPORTANT]\n> **Quantization process:** <br>\n> Imatrix data was generated from the FP16-GGUF and conversions directly from the BF16-GGUF. <br>\n> This is a bit more disk and compute intensive but hopefully avoids any losses during conversion. <br>\n> To run this model, please use the [**latest version of KoboldCpp**](https://github.com/LostRuins/koboldcpp/releases/latest). <br>\n> If you noticed any issues let me know in the discussions.\n\n> [!NOTE]\n> **Presets:** <br>\n> Mistral. <br>\n> Some compatible SillyTavern presets can be found [**here (Virt's Roleplay Presets - v1.9)**](https://huggingface.co/Virt-io/SillyTavern-Presets). <br>\n> Check [**discussions such as this one**](https://huggingface.co/Virt-io/SillyTavern-Presets/discussions/5#664d6fb87c563d4d95151baa) and [**this one**](https://www.reddit.com/r/SillyTavernAI/comments/1dff2tl/my_personal_llama3_stheno_presets/) for other presets and samplers recommendations. <br>\n> Lower temperatures are recommended by the authors, so make sure to experiment. <br>\n\n<details>\n<summary>⇲ Click here to expand/hide information – General chart with relative quant parformances.</summary> \n\n> [!NOTE]\n> **Recommended read:** <br>\n> \n> [**\"Which GGUF is right for me? (Opinionated)\" by Artefact2**](https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9)\n> \n> *Click the image to view full size.*\n> ![\"Which GGUF is right for me? (Opinionated)\" by Artefact2 - Firs Graph](https://cdn-uploads.huggingface.co/production/uploads/65d4cf2693a0a3744a27536c/fScWdHIPix5IzNJ8yswCB.webp)\n\n</details>\n\n> [!TIP]\n> **Personal-support:** <br>\n> I apologize for disrupting your experience. <br>\n> Eventually I may be able to use a dedicated server for this, but for now hopefully these quants are helpful. <br>\n> If you **want** and you are **able to**... <br>\n> You can [**spare some change over here (Ko-fi)**](https://ko-fi.com/Lewdiculous). <br>\n>\n> **Author-support:** <br>\n> You can support the authors [**at their pages**](https://ko-fi.com/undiai)/[**here**](https://ikaridevgit.github.io/).\n\n![image/png](https://cdn-uploads.huggingface.co/production/uploads/65d4cf2693a0a3744a27536c/qEH7KuSGfUGXSHeyWSwS-.png)\n\n<details>\n<summary>Original model card information.</summary>\n\n## **Original card:**\n\n## Lumimaid 0.2\n<img src=\"https://cdn-uploads.huggingface.co/production/uploads/63ab1241ad514ca8d1430003/TUcHg7LKNjfo0sni88Ps7.png\" alt=\"Image\" style=\"display: block; margin-left: auto; margin-right: auto; width: 65%;\">\n<div style=\"text-align: center; font-size: 30px;\">\n    <a href=\"https://huggingface.co/NeverSleep/Lumimaid-v0.2-8B\">[8b]</a> -\n    <a href=\"https://huggingface.co/NeverSleep/Lumimaid-v0.2-12B\">12b</a> -\n    <a href=\"https://huggingface.co/NeverSleep/Lumimaid-v0.2-70B\">70b</a> -\n    <a href=\"https://huggingface.co/NeverSleep/Lumimaid-v0.2-123B\">123b</a>\n</div>\n\n### This model is based on: [Meta-Llama-3.1-8B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3.1-8B-Instruct)\nWandb: https://wandb.ai/undis95/Lumi-Llama-3-1-8B?nw=nwuserundis95\n\nLumimaid 0.1 -> 0.2 is a HUGE step up dataset wise.\n\nAs some people have told us our models are sloppy, Ikari decided to say fuck it and literally nuke all chats out with most slop.\n\nOur dataset stayed the same since day one, we added data over time, cleaned them, and repeat. After not releasing model for a while because we were never satisfied, we think it's time to come back!\n\n\n## Prompt template: Llama-3-Instruct\n\n```\n<|begin_of_text|><|start_header_id|>system<|end_header_id|>\n\n{system_prompt}<|eot_id|><|start_header_id|>user<|end_header_id|>\n\n{input}<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\n{output}<|eot_id|>\n```\n\n## Credits:\n- Undi\n- IkariDev\n\n## Training data we used to make our dataset:\n\n- [Epiculous/Gnosis](https://huggingface.co/Epiculous/Gnosis)\n- [ChaoticNeutrals/Luminous_Opus](https://huggingface.co/datasets/ChaoticNeutrals/Luminous_Opus)\n- [ChaoticNeutrals/Synthetic-Dark-RP](https://huggingface.co/datasets/ChaoticNeutrals/Synthetic-Dark-RP)\n- [ChaoticNeutrals/Synthetic-RP](https://huggingface.co/datasets/ChaoticNeutrals/Synthetic-RP)\n- [Gryphe/Sonnet3.5-SlimOrcaDedupCleaned](https://huggingface.co/datasets/Gryphe/Sonnet3.5-SlimOrcaDedupCleaned)\n- [Gryphe/Opus-WritingPrompts](https://huggingface.co/datasets/Gryphe/Opus-WritingPrompts)\n- [meseca/writing-opus-6k](https://huggingface.co/datasets/meseca/writing-opus-6k)\n- [meseca/opus-instruct-9k](https://huggingface.co/datasets/meseca/opus-instruct-9k)\n- [PJMixers/grimulkan_theory-of-mind-ShareGPT](https://huggingface.co/datasets/PJMixers/grimulkan_theory-of-mind-ShareGPT)\n- [NobodyExistsOnTheInternet/ToxicQAFinal](https://huggingface.co/datasets/NobodyExistsOnTheInternet/ToxicQAFinal)\n- [Undi95/toxic-dpo-v0.1-sharegpt](https://huggingface.co/datasets/Undi95/toxic-dpo-v0.1-sharegpt)\n- [cgato/SlimOrcaDedupCleaned](https://huggingface.co/datasets/cgato/SlimOrcaDedupCleaned)\n- [kalomaze/Opus_Instruct_25k](https://huggingface.co/datasets/kalomaze/Opus_Instruct_25k)\n- [Doctor-Shotgun/no-robots-sharegpt](https://huggingface.co/datasets/Doctor-Shotgun/no-robots-sharegpt)\n- [Norquinal/claude_multiround_chat_30k](https://huggingface.co/datasets/Norquinal/claude_multiround_chat_30k)\n- [nothingiisreal/Claude-3-Opus-Instruct-15K](https://huggingface.co/datasets/nothingiisreal/Claude-3-Opus-Instruct-15K)\n- All the Aesirs dataset, cleaned, unslopped\n- All le luminae dataset, cleaned, unslopped\n- Small part of Airoboros reduced\n\nWe sadly didn't find the sources of the following, DM us if you recognize your set !\n\n- Opus_Instruct-v2-6.5K-Filtered-v2-sharegpt\n- claude_sharegpt_trimmed\n- CapybaraPure_Decontaminated-ShareGPT_reduced\n\n## Datasets credits:\n- Epiculous\n- ChaoticNeutrals\n- Gryphe\n- meseca\n- PJMixers\n- NobodyExistsOnTheInternet\n- cgato\n- kalomaze\n- Doctor-Shotgun\n- Norquinal\n- nothingiisreal\n\n## Others\n\nUndi: If you want to support us, you can [here](https://ko-fi.com/undiai).\n\nIkariDev: Visit my [retro/neocities style website](https://ikaridevgit.github.io/) please kek\n\n</details>",
    "related_quantizations": []
  },
  "tags": [
    "transformers",
    "gguf",
    "roleplay",
    "mistral",
    "nemo",
    "sillytavern",
    "en",
    "base_model:NeverSleep/Lumimaid-v0.2-12B",
    "base_model:quantized:NeverSleep/Lumimaid-v0.2-12B",
    "license:cc-by-nc-4.0",
    "region:us",
    "conversational"
  ],
  "likes": 29,
  "downloads": 906,
  "gated": false,
  "private": false,
  "last_modified": "2024-12-14T10:25:34.000Z",
  "created_at": "2024-07-29T00:32:32.000Z",
  "pipeline_tag": "",
  "library_name": "transformers"
}

{
  "_id": "66a6e320c302ba7970d0702f",
  "id": "Lewdiculous/Lumimaid-v0.2-12B-GGUF-IQ-Imatrix",
  "modelId": "Lewdiculous/Lumimaid-v0.2-12B-GGUF-IQ-Imatrix",
  "sha": "b009c281491450ee7b1dca66ff0b8fadfe637703",
  "createdAt": "2024-07-29T00:32:32.000Z",
  "lastModified": "2024-12-14T10:25:34.000Z",
  "author": "Lewdiculous",
  "downloads": 906,
  "likes": 29,
  "gated": false,
  "private": false,
  "pipeline_tag": "",
  "library_name": "transformers",
  "siblings_count": 19
}