richarderkhov/sao10k_-_l3-8b-stheno-v3.2-gguf overview

{
  "metadata": {},
  "card_data": {
    "frontmatter": {},
    "hero_image_url": "https://huggingface.co/Sao10K/L3-8B-Stheno-v3.2/resolve/main/Stheno.png?",
    "summary": "Quantization made by Richard Erkhov. Github Discord Request more models L3-8B-Stheno-v3.2 - GGUF | Name | Quant method | Size | | ---- | ---- | ---- | | L3-8B-Stheno-v3.2.Q2_K.gguf | Q2_K | 2.96GB | | L3-8B-Stheno-v3.2.IQ3_XS.gguf | IQ3_XS | 3.28GB | | L3-8B-Stheno-v3.2.IQ3_S.gguf | IQ3_S | 3.43GB | | L3-8B-Stheno-v3.2.Q3_K_S.gguf | Q3_K_S | 3.41GB | | L3-8B-Stheno-v3.2.IQ3_M.gguf | IQ3_M | 3.52GB | | L3-8B-Stheno-v3.2.Q3_K.gguf | Q3_K | 3.74GB | | L3-8B-Stheno-v3.2.Q3_K_M.gguf | Q3_K_M | 3.74GB | | L3-8B-Stheno-v3.2.Q3_K_L.gguf | Q3_K_L | 4.03GB | | L3-8B-Stheno-v3.2.IQ4_XS.gguf | IQ4_XS | 4.18GB | | L3-8B-Stheno-v3.2.Q4_0.gguf | Q4_0 | 4.34GB | | L3-8B-Stheno-v3.2.IQ4_NL.gguf | IQ4_NL | 4.38GB | | L3-8B-Stheno-v3.2.Q4_K_S.gguf | Q4_K_S | 4.37GB | | L3-8B-Stheno-v3.2.Q4_K.gguf | Q4_K | 4.58GB | | L3-8B-Stheno-v3.2.Q4_K_M.gguf | Q4_K_M | 4.58GB | | L3-8B-Stheno-v3.2.Q4_1.gguf | Q4_1 | 4.78GB | | L3-8B-Stheno-v3.2.Q5_0.gguf | Q5_0 | 5.21GB | | L3-8B-Stheno-v3.2.Q5_K_S.gguf | Q5_K_S | 5.21GB | | L3-8B-Stheno-v3.2.Q5_K.gguf | Q5_K | 5.34GB | | L3-8B-Stheno-v3.2.Q5_K_M.gguf | Q5_K_M | 5.34GB | | L3-8B-Stheno-v3.2.Q5_1.gguf | Q5_1 | 5.65GB | | L3-8B-Stheno-v3.2.Q6_K.gguf | Q6_K | 6.14GB | | L3-8B-Stheno-v3.2.Q8_0.gguf | Q8_0 | 7.95GB | Original model description: --- license: cc-by-nc-4.0 language: datasets: --- *Just message me on discord if you want to host this privately for a service or something. We can talk.* *Train used 1x H100 SXM for like a total of 24 Hours over multiple runs.* Support me here if you're interested: Ko-fi: https://ko-fi.com/sao10k  *wink* Euryale v2? If not, that's fine too. Feedback would be nice. Contact Me in Discord: sao10k // Just ping me in the KoboldAI discord, I'll respond faster. Art by navy_(navy.blue) - Danbooru --- !Stheno Stheno-v3.2-Zeta I have done a test run with multiple variations of the models, merged back to its base at various weights, different training runs too, and this Sixth iteration is the one I like most. Changes compared to v3.1 \\- Included a mix of SFW and NSFW Storywriting Data, thanks to Gryphe \\- Included More Instruct / Assistant-Style Data \\- Further cleaned up Roleplaying Samples from c2 Logs -> A few terrible, really bad samples escaped heavy filtering. Manual pass fixed it. \\- Hyperparameter tinkering for training, resulting in lower loss levels. Testing Notes - Compared to v3.1 \\- Handles SFW / NSFW seperately better. Not as overly excessive with NSFW now. Kinda balanced. \\- Better at Storywriting / Narration. \\- Better at Assistant-type Tasks. \\- Better Multi-Turn Coherency -> Reduced Issues? \\- Slightly less creative? A worthy tradeoff. Still creative. \\- Better prompt / instruction adherence. --- **Recommended Samplers:** `` Temperature - 1.12-1.22 Min-P - 0.075 Top-K - 50 Repetition Penalty - 1.1 ` **Stopping Strings:** ` \\n\\n{{User}} # Or Equivalent, depending on Frontend   ` **Prompting Template - Llama-3-Instruct** ` system {system_prompt}user {input}assistant {output} ` **Basic Roleplay System Prompt** ` You are an expert actor that can fully immerse yourself into any role given. You do not break character for any reason, even if someone tries addressing you as an AI or language model. Currently your role is {{char}}, which is described in detail below. As {{char}}, continue the exchange with {{user}}. `` ---",
    "quick_links": [],
    "benchmark_table_html": "",
    "readme_markdown": "Quantization made by Richard Erkhov.\n\n[Github](https://github.com/RichardErkhov)\n\n[Discord](https://discord.gg/pvy7H8DZMG)\n\n[Request more models](https://github.com/RichardErkhov/quant_request)\n\n\nL3-8B-Stheno-v3.2 - GGUF\n- Model creator: https://huggingface.co/Sao10K/\n- Original model: https://huggingface.co/Sao10K/L3-8B-Stheno-v3.2/\n\n\n| Name | Quant method | Size |\n| ---- | ---- | ---- |\n| [L3-8B-Stheno-v3.2.Q2_K.gguf](https://huggingface.co/RichardErkhov/Sao10K_-_L3-8B-Stheno-v3.2-gguf/blob/main/L3-8B-Stheno-v3.2.Q2_K.gguf) | Q2_K | 2.96GB |\n| [L3-8B-Stheno-v3.2.IQ3_XS.gguf](https://huggingface.co/RichardErkhov/Sao10K_-_L3-8B-Stheno-v3.2-gguf/blob/main/L3-8B-Stheno-v3.2.IQ3_XS.gguf) | IQ3_XS | 3.28GB |\n| [L3-8B-Stheno-v3.2.IQ3_S.gguf](https://huggingface.co/RichardErkhov/Sao10K_-_L3-8B-Stheno-v3.2-gguf/blob/main/L3-8B-Stheno-v3.2.IQ3_S.gguf) | IQ3_S | 3.43GB |\n| [L3-8B-Stheno-v3.2.Q3_K_S.gguf](https://huggingface.co/RichardErkhov/Sao10K_-_L3-8B-Stheno-v3.2-gguf/blob/main/L3-8B-Stheno-v3.2.Q3_K_S.gguf) | Q3_K_S | 3.41GB |\n| [L3-8B-Stheno-v3.2.IQ3_M.gguf](https://huggingface.co/RichardErkhov/Sao10K_-_L3-8B-Stheno-v3.2-gguf/blob/main/L3-8B-Stheno-v3.2.IQ3_M.gguf) | IQ3_M | 3.52GB |\n| [L3-8B-Stheno-v3.2.Q3_K.gguf](https://huggingface.co/RichardErkhov/Sao10K_-_L3-8B-Stheno-v3.2-gguf/blob/main/L3-8B-Stheno-v3.2.Q3_K.gguf) | Q3_K | 3.74GB |\n| [L3-8B-Stheno-v3.2.Q3_K_M.gguf](https://huggingface.co/RichardErkhov/Sao10K_-_L3-8B-Stheno-v3.2-gguf/blob/main/L3-8B-Stheno-v3.2.Q3_K_M.gguf) | Q3_K_M | 3.74GB |\n| [L3-8B-Stheno-v3.2.Q3_K_L.gguf](https://huggingface.co/RichardErkhov/Sao10K_-_L3-8B-Stheno-v3.2-gguf/blob/main/L3-8B-Stheno-v3.2.Q3_K_L.gguf) | Q3_K_L | 4.03GB |\n| [L3-8B-Stheno-v3.2.IQ4_XS.gguf](https://huggingface.co/RichardErkhov/Sao10K_-_L3-8B-Stheno-v3.2-gguf/blob/main/L3-8B-Stheno-v3.2.IQ4_XS.gguf) | IQ4_XS | 4.18GB |\n| [L3-8B-Stheno-v3.2.Q4_0.gguf](https://huggingface.co/RichardErkhov/Sao10K_-_L3-8B-Stheno-v3.2-gguf/blob/main/L3-8B-Stheno-v3.2.Q4_0.gguf) | Q4_0 | 4.34GB |\n| [L3-8B-Stheno-v3.2.IQ4_NL.gguf](https://huggingface.co/RichardErkhov/Sao10K_-_L3-8B-Stheno-v3.2-gguf/blob/main/L3-8B-Stheno-v3.2.IQ4_NL.gguf) | IQ4_NL | 4.38GB |\n| [L3-8B-Stheno-v3.2.Q4_K_S.gguf](https://huggingface.co/RichardErkhov/Sao10K_-_L3-8B-Stheno-v3.2-gguf/blob/main/L3-8B-Stheno-v3.2.Q4_K_S.gguf) | Q4_K_S | 4.37GB |\n| [L3-8B-Stheno-v3.2.Q4_K.gguf](https://huggingface.co/RichardErkhov/Sao10K_-_L3-8B-Stheno-v3.2-gguf/blob/main/L3-8B-Stheno-v3.2.Q4_K.gguf) | Q4_K | 4.58GB |\n| [L3-8B-Stheno-v3.2.Q4_K_M.gguf](https://huggingface.co/RichardErkhov/Sao10K_-_L3-8B-Stheno-v3.2-gguf/blob/main/L3-8B-Stheno-v3.2.Q4_K_M.gguf) | Q4_K_M | 4.58GB |\n| [L3-8B-Stheno-v3.2.Q4_1.gguf](https://huggingface.co/RichardErkhov/Sao10K_-_L3-8B-Stheno-v3.2-gguf/blob/main/L3-8B-Stheno-v3.2.Q4_1.gguf) | Q4_1 | 4.78GB |\n| [L3-8B-Stheno-v3.2.Q5_0.gguf](https://huggingface.co/RichardErkhov/Sao10K_-_L3-8B-Stheno-v3.2-gguf/blob/main/L3-8B-Stheno-v3.2.Q5_0.gguf) | Q5_0 | 5.21GB |\n| [L3-8B-Stheno-v3.2.Q5_K_S.gguf](https://huggingface.co/RichardErkhov/Sao10K_-_L3-8B-Stheno-v3.2-gguf/blob/main/L3-8B-Stheno-v3.2.Q5_K_S.gguf) | Q5_K_S | 5.21GB |\n| [L3-8B-Stheno-v3.2.Q5_K.gguf](https://huggingface.co/RichardErkhov/Sao10K_-_L3-8B-Stheno-v3.2-gguf/blob/main/L3-8B-Stheno-v3.2.Q5_K.gguf) | Q5_K | 5.34GB |\n| [L3-8B-Stheno-v3.2.Q5_K_M.gguf](https://huggingface.co/RichardErkhov/Sao10K_-_L3-8B-Stheno-v3.2-gguf/blob/main/L3-8B-Stheno-v3.2.Q5_K_M.gguf) | Q5_K_M | 5.34GB |\n| [L3-8B-Stheno-v3.2.Q5_1.gguf](https://huggingface.co/RichardErkhov/Sao10K_-_L3-8B-Stheno-v3.2-gguf/blob/main/L3-8B-Stheno-v3.2.Q5_1.gguf) | Q5_1 | 5.65GB |\n| [L3-8B-Stheno-v3.2.Q6_K.gguf](https://huggingface.co/RichardErkhov/Sao10K_-_L3-8B-Stheno-v3.2-gguf/blob/main/L3-8B-Stheno-v3.2.Q6_K.gguf) | Q6_K | 6.14GB |\n| [L3-8B-Stheno-v3.2.Q8_0.gguf](https://huggingface.co/RichardErkhov/Sao10K_-_L3-8B-Stheno-v3.2-gguf/blob/main/L3-8B-Stheno-v3.2.Q8_0.gguf) | Q8_0 | 7.95GB |\n\n\n\n\nOriginal model description:\n---\nlicense: cc-by-nc-4.0\nlanguage:\n- en\ndatasets:\n- Gryphe/Opus-WritingPrompts\n- Sao10K/Claude-3-Opus-Instruct-15K\n- Sao10K/Short-Storygen-v2\n- Sao10K/c2-Logs-Filtered\n---\n\n*Just message me on discord if you want to host this privately for a service or something. We can talk.*\n\n*Train used 1x H100 SXM for like a total of 24 Hours over multiple runs.*\n\nSupport me here if you're interested:\n<br>Ko-fi: https://ko-fi.com/sao10k\n<br> *wink* Euryale v2?\n\nIf not, that's fine too. Feedback would be nice.\n\nContact Me in Discord:\n<br>`sao10k` // `Just ping me in the KoboldAI discord, I'll respond faster.`\n\n`Art by navy_(navy.blue)` - [Danbooru](https://danbooru.donmai.us/posts/3214477)\n\n---\n\n![Stheno](https://huggingface.co/Sao10K/L3-8B-Stheno-v3.2/resolve/main/Stheno.png?)\n\nStheno-v3.2-Zeta\n\nI have done a test run with multiple variations of the models, merged back to its base at various weights, different training runs too, and this Sixth iteration is the one I like most.\n\n\nChanges compared to v3.1\n<br>\\- Included a mix of SFW and NSFW Storywriting Data, thanks to [Gryphe](https://huggingface.co/datasets/Gryphe/Opus-WritingPrompts)\n<br>\\- Included More Instruct / Assistant-Style Data\n<br>\\- Further cleaned up Roleplaying Samples from c2 Logs -> A few terrible, really bad samples escaped heavy filtering. Manual pass fixed it.\n<br>\\- Hyperparameter tinkering for training, resulting in lower loss levels.\n\n\nTesting Notes - Compared to v3.1\n<br>\\- Handles SFW / NSFW seperately better. Not as overly excessive with NSFW now. Kinda balanced.\n<br>\\- Better at Storywriting / Narration.\n<br>\\- Better at Assistant-type Tasks.\n<br>\\- Better Multi-Turn Coherency -> Reduced Issues?\n<br>\\- Slightly less creative? A worthy tradeoff. Still creative.\n<br>\\- Better prompt / instruction adherence.\n\n---\n\n**Recommended Samplers:**\n\n```\nTemperature - 1.12-1.22\nMin-P - 0.075\nTop-K - 50\nRepetition Penalty - 1.1\n```\n\n**Stopping Strings:**\n\n```\n\\n\\n{{User}} # Or Equivalent, depending on Frontend\n<|eot_id|>\n<|end_of_text|>\n```\n\n**Prompting Template - Llama-3-Instruct**\n\n```\n<|begin_of_text|><|start_header_id|>system<|end_header_id|>\n\n{system_prompt}<|eot_id|><|start_header_id|>user<|end_header_id|>\n\n{input}<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\n{output}<|eot_id|>\n```\n\n**Basic Roleplay System Prompt**\n```\nYou are an expert actor that can fully immerse yourself into any role given. You do not break character for any reason, even if someone tries addressing you as an AI or language model.\nCurrently your role is {{char}}, which is described in detail below. As {{char}}, continue the exchange with {{user}}.\n```\n\n---\n\n",
    "related_quantizations": []
  },
  "tags": [
    "gguf",
    "endpoints_compatible",
    "region:us",
    "conversational"
  ],
  "likes": 3,
  "downloads": 579,
  "gated": false,
  "private": false,
  "last_modified": "2024-06-25T08:22:02.000Z",
  "created_at": "2024-06-25T04:05:03.000Z",
  "pipeline_tag": "",
  "library_name": ""
}

{
  "_id": "667a41ef060322abb2b2c14b",
  "id": "RichardErkhov/Sao10K_-_L3-8B-Stheno-v3.2-gguf",
  "modelId": "RichardErkhov/Sao10K_-_L3-8B-Stheno-v3.2-gguf",
  "sha": "08dc467d797897d2a22cb97bba4aa330f5119921",
  "createdAt": "2024-06-25T04:05:03.000Z",
  "lastModified": "2024-06-25T08:22:02.000Z",
  "author": "RichardErkhov",
  "downloads": 579,
  "likes": 3,
  "gated": false,
  "private": false,
  "pipeline_tag": "",
  "library_name": "",
  "siblings_count": 24
}