richarderkhov/xxx777xxxasd_-_l3.1-claudemaid-4x8b-gguf overview

{
  "metadata": {},
  "card_data": {
    "frontmatter": {},
    "hero_image_url": "https://cdn-uploads.huggingface.co/production/uploads/64f5e51289c121cb864ba464/O0FlWv4L8ZnehOGETw7qt.png",
    "summary": "Quantization made by Richard Erkhov. Github Discord Request more models L3.1-ClaudeMaid-4x8B - GGUF | Name | Quant method | Size | | ---- | ---- | ---- | | L3.1-ClaudeMaid-4x8B.Q2_K.gguf | Q2_K | 8.66GB | | L3.1-ClaudeMaid-4x8B.IQ3_XS.gguf | IQ3_XS | 9.68GB | | L3.1-ClaudeMaid-4x8B.IQ3_S.gguf | IQ3_S | 10.2GB | | L3.1-ClaudeMaid-4x8B.Q3_K_S.gguf | Q3_K_S | 10.18GB | | L3.1-ClaudeMaid-4x8B.IQ3_M.gguf | IQ3_M | 10.38GB | | L3.1-ClaudeMaid-4x8B.Q3_K.gguf | Q3_K | 11.25GB | | L3.1-ClaudeMaid-4x8B.Q3_K_M.gguf | Q3_K_M | 11.25GB | | L3.1-ClaudeMaid-4x8B.Q3_K_L.gguf | Q3_K_L | 12.15GB | | L3.1-ClaudeMaid-4x8B.IQ4_XS.gguf | IQ4_XS | 12.65GB | | L3.1-ClaudeMaid-4x8B.Q4_0.gguf | Q4_0 | 13.2GB | | L3.1-ClaudeMaid-4x8B.IQ4_NL.gguf | IQ4_NL | 13.33GB | | L3.1-ClaudeMaid-4x8B.Q4_K_S.gguf | Q4_K_S | 13.31GB | | L3.1-ClaudeMaid-4x8B.Q4_K.gguf | Q4_K | 14.12GB | | L3.1-ClaudeMaid-4x8B.Q4_K_M.gguf | Q4_K_M | 14.12GB | | L3.1-ClaudeMaid-4x8B.Q4_1.gguf | Q4_1 | 14.62GB | | L3.1-ClaudeMaid-4x8B.Q5_0.gguf | Q5_0 | 16.04GB | | L3.1-ClaudeMaid-4x8B.Q5_K_S.gguf | Q5_K_S | 16.04GB | | L3.1-ClaudeMaid-4x8B.Q5_K.gguf | Q5_K | 16.52GB | | L3.1-ClaudeMaid-4x8B.Q5_K_M.gguf | Q5_K_M | 16.52GB | | L3.1-ClaudeMaid-4x8B.Q5_1.gguf | Q5_1 | 17.47GB | | L3.1-ClaudeMaid-4x8B.Q6_K.gguf | Q6_K | 19.06GB | | L3.1-ClaudeMaid-4x8B.Q8_0.gguf | Q8_0 | 24.69GB | Original model description: --- language: license: llama3.1 tags: ---  .image-container { position: relative; display: inline-block; } .image-container img { display: block; border-radius: 10px; box-shadow: 0 0 1px rgba(0, 0, 0, 0.3); } .image-container::before { content: \"\"; position: absolute; top: 0px; left: 20px; width: calc(100% - 40px); height: calc(100%); background-image: url(\"https://cdn-uploads.huggingface.co/production/uploads/64f5e51289c121cb864ba464/O0FlWv4L8ZnehOGETw7qt.png\"); background-size: cover; filter: blur(10px); z-index: -1; }      > [!WARNING] > GGUF Experimental RP-oriented MoE ### Llama 3 ClaudeMaid v1.0 4x8B `` base_model: NeverSleep_Lumimaid-v0.2-8B gate_mode: random dtype: bfloat16 experts_per_token: 2 experts: ``",
    "quick_links": [],
    "benchmark_table_html": "",
    "readme_markdown": "Quantization made by Richard Erkhov.\n\n[Github](https://github.com/RichardErkhov)\n\n[Discord](https://discord.gg/pvy7H8DZMG)\n\n[Request more models](https://github.com/RichardErkhov/quant_request)\n\n\nL3.1-ClaudeMaid-4x8B - GGUF\n- Model creator: https://huggingface.co/xxx777xxxASD/\n- Original model: https://huggingface.co/xxx777xxxASD/L3.1-ClaudeMaid-4x8B/\n\n\n| Name | Quant method | Size |\n| ---- | ---- | ---- |\n| [L3.1-ClaudeMaid-4x8B.Q2_K.gguf](https://huggingface.co/RichardErkhov/xxx777xxxASD_-_L3.1-ClaudeMaid-4x8B-gguf/blob/main/L3.1-ClaudeMaid-4x8B.Q2_K.gguf) | Q2_K | 8.66GB |\n| [L3.1-ClaudeMaid-4x8B.IQ3_XS.gguf](https://huggingface.co/RichardErkhov/xxx777xxxASD_-_L3.1-ClaudeMaid-4x8B-gguf/blob/main/L3.1-ClaudeMaid-4x8B.IQ3_XS.gguf) | IQ3_XS | 9.68GB |\n| [L3.1-ClaudeMaid-4x8B.IQ3_S.gguf](https://huggingface.co/RichardErkhov/xxx777xxxASD_-_L3.1-ClaudeMaid-4x8B-gguf/blob/main/L3.1-ClaudeMaid-4x8B.IQ3_S.gguf) | IQ3_S | 10.2GB |\n| [L3.1-ClaudeMaid-4x8B.Q3_K_S.gguf](https://huggingface.co/RichardErkhov/xxx777xxxASD_-_L3.1-ClaudeMaid-4x8B-gguf/blob/main/L3.1-ClaudeMaid-4x8B.Q3_K_S.gguf) | Q3_K_S | 10.18GB |\n| [L3.1-ClaudeMaid-4x8B.IQ3_M.gguf](https://huggingface.co/RichardErkhov/xxx777xxxASD_-_L3.1-ClaudeMaid-4x8B-gguf/blob/main/L3.1-ClaudeMaid-4x8B.IQ3_M.gguf) | IQ3_M | 10.38GB |\n| [L3.1-ClaudeMaid-4x8B.Q3_K.gguf](https://huggingface.co/RichardErkhov/xxx777xxxASD_-_L3.1-ClaudeMaid-4x8B-gguf/blob/main/L3.1-ClaudeMaid-4x8B.Q3_K.gguf) | Q3_K | 11.25GB |\n| [L3.1-ClaudeMaid-4x8B.Q3_K_M.gguf](https://huggingface.co/RichardErkhov/xxx777xxxASD_-_L3.1-ClaudeMaid-4x8B-gguf/blob/main/L3.1-ClaudeMaid-4x8B.Q3_K_M.gguf) | Q3_K_M | 11.25GB |\n| [L3.1-ClaudeMaid-4x8B.Q3_K_L.gguf](https://huggingface.co/RichardErkhov/xxx777xxxASD_-_L3.1-ClaudeMaid-4x8B-gguf/blob/main/L3.1-ClaudeMaid-4x8B.Q3_K_L.gguf) | Q3_K_L | 12.15GB |\n| [L3.1-ClaudeMaid-4x8B.IQ4_XS.gguf](https://huggingface.co/RichardErkhov/xxx777xxxASD_-_L3.1-ClaudeMaid-4x8B-gguf/blob/main/L3.1-ClaudeMaid-4x8B.IQ4_XS.gguf) | IQ4_XS | 12.65GB |\n| [L3.1-ClaudeMaid-4x8B.Q4_0.gguf](https://huggingface.co/RichardErkhov/xxx777xxxASD_-_L3.1-ClaudeMaid-4x8B-gguf/blob/main/L3.1-ClaudeMaid-4x8B.Q4_0.gguf) | Q4_0 | 13.2GB |\n| [L3.1-ClaudeMaid-4x8B.IQ4_NL.gguf](https://huggingface.co/RichardErkhov/xxx777xxxASD_-_L3.1-ClaudeMaid-4x8B-gguf/blob/main/L3.1-ClaudeMaid-4x8B.IQ4_NL.gguf) | IQ4_NL | 13.33GB |\n| [L3.1-ClaudeMaid-4x8B.Q4_K_S.gguf](https://huggingface.co/RichardErkhov/xxx777xxxASD_-_L3.1-ClaudeMaid-4x8B-gguf/blob/main/L3.1-ClaudeMaid-4x8B.Q4_K_S.gguf) | Q4_K_S | 13.31GB |\n| [L3.1-ClaudeMaid-4x8B.Q4_K.gguf](https://huggingface.co/RichardErkhov/xxx777xxxASD_-_L3.1-ClaudeMaid-4x8B-gguf/blob/main/L3.1-ClaudeMaid-4x8B.Q4_K.gguf) | Q4_K | 14.12GB |\n| [L3.1-ClaudeMaid-4x8B.Q4_K_M.gguf](https://huggingface.co/RichardErkhov/xxx777xxxASD_-_L3.1-ClaudeMaid-4x8B-gguf/blob/main/L3.1-ClaudeMaid-4x8B.Q4_K_M.gguf) | Q4_K_M | 14.12GB |\n| [L3.1-ClaudeMaid-4x8B.Q4_1.gguf](https://huggingface.co/RichardErkhov/xxx777xxxASD_-_L3.1-ClaudeMaid-4x8B-gguf/blob/main/L3.1-ClaudeMaid-4x8B.Q4_1.gguf) | Q4_1 | 14.62GB |\n| [L3.1-ClaudeMaid-4x8B.Q5_0.gguf](https://huggingface.co/RichardErkhov/xxx777xxxASD_-_L3.1-ClaudeMaid-4x8B-gguf/blob/main/L3.1-ClaudeMaid-4x8B.Q5_0.gguf) | Q5_0 | 16.04GB |\n| [L3.1-ClaudeMaid-4x8B.Q5_K_S.gguf](https://huggingface.co/RichardErkhov/xxx777xxxASD_-_L3.1-ClaudeMaid-4x8B-gguf/blob/main/L3.1-ClaudeMaid-4x8B.Q5_K_S.gguf) | Q5_K_S | 16.04GB |\n| [L3.1-ClaudeMaid-4x8B.Q5_K.gguf](https://huggingface.co/RichardErkhov/xxx777xxxASD_-_L3.1-ClaudeMaid-4x8B-gguf/blob/main/L3.1-ClaudeMaid-4x8B.Q5_K.gguf) | Q5_K | 16.52GB |\n| [L3.1-ClaudeMaid-4x8B.Q5_K_M.gguf](https://huggingface.co/RichardErkhov/xxx777xxxASD_-_L3.1-ClaudeMaid-4x8B-gguf/blob/main/L3.1-ClaudeMaid-4x8B.Q5_K_M.gguf) | Q5_K_M | 16.52GB |\n| [L3.1-ClaudeMaid-4x8B.Q5_1.gguf](https://huggingface.co/RichardErkhov/xxx777xxxASD_-_L3.1-ClaudeMaid-4x8B-gguf/blob/main/L3.1-ClaudeMaid-4x8B.Q5_1.gguf) | Q5_1 | 17.47GB |\n| [L3.1-ClaudeMaid-4x8B.Q6_K.gguf](https://huggingface.co/RichardErkhov/xxx777xxxASD_-_L3.1-ClaudeMaid-4x8B-gguf/blob/main/L3.1-ClaudeMaid-4x8B.Q6_K.gguf) | Q6_K | 19.06GB |\n| [L3.1-ClaudeMaid-4x8B.Q8_0.gguf](https://huggingface.co/RichardErkhov/xxx777xxxASD_-_L3.1-ClaudeMaid-4x8B-gguf/blob/main/L3.1-ClaudeMaid-4x8B.Q8_0.gguf) | Q8_0 | 24.69GB |\n\n\n\n\nOriginal model description:\n---\nlanguage:\n- en\nlicense: llama3.1\ntags:\n- moe\n---\n\n<style>\n.image-container {\n  position: relative;\n  display: inline-block;\n}\n\n.image-container img {\n  display: block;\n  border-radius: 10px;\n  box-shadow: 0 0 1px rgba(0, 0, 0, 0.3);\n}\n\n.image-container::before {\n  content: \"\";\n  position: absolute;\n  top: 0px;\n  left: 20px;\n  width: calc(100% - 40px);\n  height: calc(100%);\n  background-image: url(\"https://cdn-uploads.huggingface.co/production/uploads/64f5e51289c121cb864ba464/O0FlWv4L8ZnehOGETw7qt.png\");\n  background-size: cover;\n  filter: blur(10px);\n  z-index: -1;\n}\n</style>\n<br>\n\n<div class=\"image-container\">\n  <img src=\"https://cdn-uploads.huggingface.co/production/uploads/64f5e51289c121cb864ba464/O0FlWv4L8ZnehOGETw7qt.png\" style=\"width: 96%; margin: auto;\" >\n</div>\n\n> [!WARNING]\n> [GGUF](https://huggingface.co/collections/xxx777xxxASD/claudemaid-4x8b-66a54c20d608a8c966866a5a)\n\nExperimental RP-oriented MoE\n\n### Llama 3 ClaudeMaid v1.0 4x8B\n```\nbase_model: NeverSleep_Lumimaid-v0.2-8B\ngate_mode: random\ndtype: bfloat16\nexperts_per_token: 2\nexperts:\n  - source_model: aifeifei798_DarkIdol-Llama-3.1-8B-Instruct-1.0-Uncensored\n  - source_model: Nitral-AI_Sekhmet_Bet-L3.1-8B-v0.2\n  - source_model: NeverSleep_Lumimaid-v0.2-8B\n  - source_model: Undi95_Meta-Llama-3.1-8B-Claude-bf16\n```\n\n## Usage\n\nIt seems like 1.71 koboldcpp can't run GGUFs of llama-3.1 MoE models yet, or perhaps im just dumb and messed something up.\nIf anyone has similar problem - run the model directly from llama.cpp, here's simple [open source GUI(Windows)](https://huggingface.co/xxx777xxxASD/LlamaCpp-Server-Ui) you can use if the console is your worst enemy\n![image/png](https://cdn-uploads.huggingface.co/production/uploads/64f5e51289c121cb864ba464/sTM3GUVMucM_AnIk8iG7H.png)\n\nUPDATE 28.07.2024\nTry [this koboldcpp version](https://github.com/Nexesenex/kobold.cpp/releases/tag/v1.71013_b3455%2B9)\n\n## Models used\n\n- [NeverSleep/Lumimaid-v0.2-8B](https://huggingface.co/NeverSleep/Lumimaid-v0.2-8B)\n- [Undi95/Meta-Llama-3.1-8B-Claude](https://huggingface.co/Undi95/Meta-Llama-3.1-8B-Claude)\n- [Nitral-AI/Sekhmet_Bet-L3.1-8B-v0.2](https://huggingface.co/Nitral-AI/Sekhmet_Bet-L3.1-8B-v0.2)\n- [aifeifei798/DarkIdol-Llama-3.1-8B-Instruct-1.0-Uncensored](https://huggingface.co/aifeifei798/DarkIdol-Llama-3.1-8B-Instruct-1.0-Uncensored)\n\n\n## Prompt format: Llama 3\n\n",
    "related_quantizations": []
  },
  "tags": [
    "gguf",
    "endpoints_compatible",
    "region:us",
    "conversational"
  ],
  "likes": 1,
  "downloads": 288,
  "gated": false,
  "private": false,
  "last_modified": "2024-08-09T20:55:20.000Z",
  "created_at": "2024-08-09T14:36:45.000Z",
  "pipeline_tag": "",
  "library_name": ""
}

{
  "_id": "66b6297daf77dd9651dce722",
  "id": "RichardErkhov/xxx777xxxASD_-_L3.1-ClaudeMaid-4x8B-gguf",
  "modelId": "RichardErkhov/xxx777xxxASD_-_L3.1-ClaudeMaid-4x8B-gguf",
  "sha": "2858594454ff5a48e08415bc9f2c46f0c74a0ddb",
  "createdAt": "2024-08-09T14:36:45.000Z",
  "lastModified": "2024-08-09T20:55:20.000Z",
  "author": "RichardErkhov",
  "downloads": 288,
  "likes": 1,
  "gated": false,
  "private": false,
  "pipeline_tag": "",
  "library_name": "",
  "siblings_count": 24
}