richarderkhov/bluenipples_-_snowlotus-v2-10.7b-gguf overview

{
  "metadata": {},
  "card_data": {
    "frontmatter": {},
    "hero_image_url": "https://cdn-uploads.huggingface.co/production/uploads/64bb1109aaccfd28b023bcec/gTQtPK46laLIFg0RTAv73.png",
    "summary": "Quantization made by Richard Erkhov. Github Discord Request more models SnowLotus-v2-10.7B - GGUF | Name | Quant method | Size | | ---- | ---- | ---- | | SnowLotus-v2-10.7B.Q2_K.gguf | Q2_K | 3.73GB | | SnowLotus-v2-10.7B.IQ3_XS.gguf | IQ3_XS | 4.14GB | | SnowLotus-v2-10.7B.IQ3_S.gguf | IQ3_S | 4.37GB | | SnowLotus-v2-10.7B.Q3_K_S.gguf | Q3_K_S | 4.34GB | | SnowLotus-v2-10.7B.IQ3_M.gguf | IQ3_M | 4.51GB | | SnowLotus-v2-10.7B.Q3_K.gguf | Q3_K | 4.84GB | | SnowLotus-v2-10.7B.Q3_K_M.gguf | Q3_K_M | 4.84GB | | SnowLotus-v2-10.7B.Q3_K_L.gguf | Q3_K_L | 5.26GB | | SnowLotus-v2-10.7B.IQ4_XS.gguf | IQ4_XS | 5.43GB | | SnowLotus-v2-10.7B.Q4_0.gguf | Q4_0 | 5.66GB | | SnowLotus-v2-10.7B.IQ4_NL.gguf | IQ4_NL | 5.72GB | | SnowLotus-v2-10.7B.Q4_K_S.gguf | Q4_K_S | 5.7GB | | SnowLotus-v2-10.7B.Q4_K.gguf | Q4_K | 6.02GB | | SnowLotus-v2-10.7B.Q4_K_M.gguf | Q4_K_M | 6.02GB | | SnowLotus-v2-10.7B.Q4_1.gguf | Q4_1 | 6.27GB | | SnowLotus-v2-10.7B.Q5_0.gguf | Q5_0 | 6.89GB | | SnowLotus-v2-10.7B.Q5_K_S.gguf | Q5_K_S | 6.89GB | | SnowLotus-v2-10.7B.Q5_K.gguf | Q5_K | 7.08GB | | SnowLotus-v2-10.7B.Q5_K_M.gguf | Q5_K_M | 7.08GB | | SnowLotus-v2-10.7B.Q5_1.gguf | Q5_1 | 7.51GB | | SnowLotus-v2-10.7B.Q6_K.gguf | Q6_K | 8.2GB | | SnowLotus-v2-10.7B.Q8_0.gguf | Q8_0 | 10.62GB | Original model description: --- license: apache-2.0 tags: --- !SnowLotus Logo ### Premise So this is a basic slerp merge between a smart model and a good prose model. Prose and smarts. What we all want in an uncensored RP model right? I feel like Solar has untapped potential, in any case. Sao10K's Frostwind finetune is a key component of the mixture, its smarts are impressive. NyxKrage's Frostmaid experiment, which merges Frostwind with a frankenmerge of Noromaid and a mystery medical model, delivers quite impressive prose. His model creatively incorporates long-range context and instructions too, despite being slightly incoherent due to the fraken merging. So those are the main ingredients. Thanks to Nyx for sorting out the pytorch files btw. GGUF (Small selection of Imatrix and regular k-quants): https://huggingface.co/BlueNipples/DaringLotus-SnowLotus-10.7b-IQ-GGUF EXL2s: https://huggingface.co/zaq-hack/SnowLotus-v2-10.7B-bpw500-h6-exl2 https://huggingface.co/lucyknada/SnowLotus-v2-10.7B-3bpw-exl2 ### Recipe So, the recipe. I added solardoc by Nyx to frostwind at a 0.15 weight, and the gradient SLERP'd Frostwind (+solardoc) into Frostmaid with these params: value: [0.9, 0.4, 0.1, 0, 0] value: [0.05, 0.95] ### Format Notes Solar is desgined for 4k context, but Nyx reports that his merge works to 8k. Given this has a slerp gradient back into that, I'm not sure which applies here. Alpaca instruct formatting. ### Tentative Dozen or So Test Conclusion This model seems to have better prose, less GPT-ish language and no degredation in coherency from the last version whilst retaining coherency from FrostWind (plus medical lora). I'm very pleased with this now, it's exactly what I wanted, basically Nyx's Frostmaid but smarter. Cheers to all the finetuners, mergers and developers without which open source models wouldn't be half of what they are. Resources used: https://huggingface.co/NyxKrage/FrostMaid-10.7B-TESTING-pt https://huggingface.co/Sao10K/Frostwind-10.7B-v1 https://huggingface.co/NyxKrage/Solar-Doc-10.7B-Lora https://github.com/cg123/mergekit/tree/main ### Ayumi Index http://ayumi.m8geil.de/erp4_chatlogs/?S=rma_0#!/index In the Ayumi ERPv4 Chat Log Index, SnowLotus scores a 94.10 in Flesch which means it produces more complex sentences than Daring (quite complex), DaringLotus scores higher in Var and Ad[jv], which means it makes heavier use of adjectives and adverbs (is more descriptive). Noteably Daring is in the top 8 for adjectives in a sentence, highest in it's weight class if you discount the chinese model, and in general both models did very well on this metric (SnowLotus ranks higher here than anything above it in IQ4), showcasing their descriptive ability. SnowLotus beats DaringLotus on IQ4 with a score of 70.94, only bet by SOLAR Instruct and Fimbulvetr in it's weight class (altho also noteably Kunoichi 7b by a slim margin), DaringLotus is a bit lower at 65.37 - not as smart. Interestingly the benchmarking here showed repetition for both models (which I haven't seen), but more with SnowLotus - so it's possible Daring repeats less than SnowLotus? These roughly confirm my impressions of the differences, altho potentially reveal some new details too. I've had a great experience RPing with these models, and seen no repetition myself, but be sure to use MinP or DynaTemp rather than the older samplers and be prepared to regen anything they get stuck on!",
    "quick_links": [],
    "benchmark_table_html": "",
    "readme_markdown": "Quantization made by Richard Erkhov.\n\n[Github](https://github.com/RichardErkhov)\n\n[Discord](https://discord.gg/pvy7H8DZMG)\n\n[Request more models](https://github.com/RichardErkhov/quant_request)\n\n\nSnowLotus-v2-10.7B - GGUF\n- Model creator: https://huggingface.co/BlueNipples/\n- Original model: https://huggingface.co/BlueNipples/SnowLotus-v2-10.7B/\n\n\n| Name | Quant method | Size |\n| ---- | ---- | ---- |\n| [SnowLotus-v2-10.7B.Q2_K.gguf](https://huggingface.co/RichardErkhov/BlueNipples_-_SnowLotus-v2-10.7B-gguf/blob/main/SnowLotus-v2-10.7B.Q2_K.gguf) | Q2_K | 3.73GB |\n| [SnowLotus-v2-10.7B.IQ3_XS.gguf](https://huggingface.co/RichardErkhov/BlueNipples_-_SnowLotus-v2-10.7B-gguf/blob/main/SnowLotus-v2-10.7B.IQ3_XS.gguf) | IQ3_XS | 4.14GB |\n| [SnowLotus-v2-10.7B.IQ3_S.gguf](https://huggingface.co/RichardErkhov/BlueNipples_-_SnowLotus-v2-10.7B-gguf/blob/main/SnowLotus-v2-10.7B.IQ3_S.gguf) | IQ3_S | 4.37GB |\n| [SnowLotus-v2-10.7B.Q3_K_S.gguf](https://huggingface.co/RichardErkhov/BlueNipples_-_SnowLotus-v2-10.7B-gguf/blob/main/SnowLotus-v2-10.7B.Q3_K_S.gguf) | Q3_K_S | 4.34GB |\n| [SnowLotus-v2-10.7B.IQ3_M.gguf](https://huggingface.co/RichardErkhov/BlueNipples_-_SnowLotus-v2-10.7B-gguf/blob/main/SnowLotus-v2-10.7B.IQ3_M.gguf) | IQ3_M | 4.51GB |\n| [SnowLotus-v2-10.7B.Q3_K.gguf](https://huggingface.co/RichardErkhov/BlueNipples_-_SnowLotus-v2-10.7B-gguf/blob/main/SnowLotus-v2-10.7B.Q3_K.gguf) | Q3_K | 4.84GB |\n| [SnowLotus-v2-10.7B.Q3_K_M.gguf](https://huggingface.co/RichardErkhov/BlueNipples_-_SnowLotus-v2-10.7B-gguf/blob/main/SnowLotus-v2-10.7B.Q3_K_M.gguf) | Q3_K_M | 4.84GB |\n| [SnowLotus-v2-10.7B.Q3_K_L.gguf](https://huggingface.co/RichardErkhov/BlueNipples_-_SnowLotus-v2-10.7B-gguf/blob/main/SnowLotus-v2-10.7B.Q3_K_L.gguf) | Q3_K_L | 5.26GB |\n| [SnowLotus-v2-10.7B.IQ4_XS.gguf](https://huggingface.co/RichardErkhov/BlueNipples_-_SnowLotus-v2-10.7B-gguf/blob/main/SnowLotus-v2-10.7B.IQ4_XS.gguf) | IQ4_XS | 5.43GB |\n| [SnowLotus-v2-10.7B.Q4_0.gguf](https://huggingface.co/RichardErkhov/BlueNipples_-_SnowLotus-v2-10.7B-gguf/blob/main/SnowLotus-v2-10.7B.Q4_0.gguf) | Q4_0 | 5.66GB |\n| [SnowLotus-v2-10.7B.IQ4_NL.gguf](https://huggingface.co/RichardErkhov/BlueNipples_-_SnowLotus-v2-10.7B-gguf/blob/main/SnowLotus-v2-10.7B.IQ4_NL.gguf) | IQ4_NL | 5.72GB |\n| [SnowLotus-v2-10.7B.Q4_K_S.gguf](https://huggingface.co/RichardErkhov/BlueNipples_-_SnowLotus-v2-10.7B-gguf/blob/main/SnowLotus-v2-10.7B.Q4_K_S.gguf) | Q4_K_S | 5.7GB |\n| [SnowLotus-v2-10.7B.Q4_K.gguf](https://huggingface.co/RichardErkhov/BlueNipples_-_SnowLotus-v2-10.7B-gguf/blob/main/SnowLotus-v2-10.7B.Q4_K.gguf) | Q4_K | 6.02GB |\n| [SnowLotus-v2-10.7B.Q4_K_M.gguf](https://huggingface.co/RichardErkhov/BlueNipples_-_SnowLotus-v2-10.7B-gguf/blob/main/SnowLotus-v2-10.7B.Q4_K_M.gguf) | Q4_K_M | 6.02GB |\n| [SnowLotus-v2-10.7B.Q4_1.gguf](https://huggingface.co/RichardErkhov/BlueNipples_-_SnowLotus-v2-10.7B-gguf/blob/main/SnowLotus-v2-10.7B.Q4_1.gguf) | Q4_1 | 6.27GB |\n| [SnowLotus-v2-10.7B.Q5_0.gguf](https://huggingface.co/RichardErkhov/BlueNipples_-_SnowLotus-v2-10.7B-gguf/blob/main/SnowLotus-v2-10.7B.Q5_0.gguf) | Q5_0 | 6.89GB |\n| [SnowLotus-v2-10.7B.Q5_K_S.gguf](https://huggingface.co/RichardErkhov/BlueNipples_-_SnowLotus-v2-10.7B-gguf/blob/main/SnowLotus-v2-10.7B.Q5_K_S.gguf) | Q5_K_S | 6.89GB |\n| [SnowLotus-v2-10.7B.Q5_K.gguf](https://huggingface.co/RichardErkhov/BlueNipples_-_SnowLotus-v2-10.7B-gguf/blob/main/SnowLotus-v2-10.7B.Q5_K.gguf) | Q5_K | 7.08GB |\n| [SnowLotus-v2-10.7B.Q5_K_M.gguf](https://huggingface.co/RichardErkhov/BlueNipples_-_SnowLotus-v2-10.7B-gguf/blob/main/SnowLotus-v2-10.7B.Q5_K_M.gguf) | Q5_K_M | 7.08GB |\n| [SnowLotus-v2-10.7B.Q5_1.gguf](https://huggingface.co/RichardErkhov/BlueNipples_-_SnowLotus-v2-10.7B-gguf/blob/main/SnowLotus-v2-10.7B.Q5_1.gguf) | Q5_1 | 7.51GB |\n| [SnowLotus-v2-10.7B.Q6_K.gguf](https://huggingface.co/RichardErkhov/BlueNipples_-_SnowLotus-v2-10.7B-gguf/blob/main/SnowLotus-v2-10.7B.Q6_K.gguf) | Q6_K | 8.2GB |\n| [SnowLotus-v2-10.7B.Q8_0.gguf](https://huggingface.co/RichardErkhov/BlueNipples_-_SnowLotus-v2-10.7B-gguf/blob/main/SnowLotus-v2-10.7B.Q8_0.gguf) | Q8_0 | 10.62GB |\n\n\n\n\nOriginal model description:\n---\nlicense: apache-2.0\ntags:\n- Roleplay\n- Solar\n- Mistral\n- Text Generation\n- merge\n---\n![SnowLotus Logo](https://cdn-uploads.huggingface.co/production/uploads/64bb1109aaccfd28b023bcec/gTQtPK46laLIFg0RTAv73.png)\n\n### Premise\n\nSo this is a basic slerp merge between a smart model and a good prose model. Prose and smarts. What we all want in an uncensored RP model right? I feel like Solar has untapped potential, in any case. \n\nSao10K's Frostwind finetune is a key component of the mixture, its smarts are impressive. NyxKrage's Frostmaid experiment, which merges Frostwind with a frankenmerge of Noromaid and a mystery medical model, delivers quite impressive prose. His model creatively incorporates long-range context and instructions too, despite being slightly incoherent due to the fraken merging. \n\nSo those are the main ingredients. Thanks to Nyx for sorting out the pytorch files btw. \n\nGGUF (Small selection of Imatrix and regular k-quants): https://huggingface.co/BlueNipples/DaringLotus-SnowLotus-10.7b-IQ-GGUF\nEXL2s: https://huggingface.co/zaq-hack/SnowLotus-v2-10.7B-bpw500-h6-exl2\nhttps://huggingface.co/lucyknada/SnowLotus-v2-10.7B-3bpw-exl2\n\n### Recipe\n\nSo, the recipe. I added solardoc by Nyx to frostwind at a 0.15 weight, and the gradient SLERP'd Frostwind (+solardoc) into Frostmaid with these params:\n\n- filter: self_attn\n      value: [0.9, 0.4, 0.1, 0, 0]\n    - filter: mlp\n      value: [0.05, 0.95]\n    - value: 0.45\n\n\n### Format Notes\n\nSolar is desgined for 4k context, but Nyx reports that his merge works to 8k. Given this has a slerp gradient back into that, I'm not sure which applies here. Alpaca instruct formatting.\n\n### Tentative Dozen or So Test Conclusion\n\nThis model seems to have better prose, less GPT-ish language and no degredation in coherency from the last version whilst retaining coherency from FrostWind (plus medical lora). I'm very pleased with this now, it's exactly what I wanted, basically Nyx's Frostmaid but smarter.\n\nCheers to all the finetuners, mergers and developers without which open source models wouldn't be half of what they are. \n\nResources used:\n\nhttps://huggingface.co/NyxKrage/FrostMaid-10.7B-TESTING-pt\n\nhttps://huggingface.co/Sao10K/Frostwind-10.7B-v1\n\nhttps://huggingface.co/NyxKrage/Solar-Doc-10.7B-Lora\n\nhttps://github.com/cg123/mergekit/tree/main\n\n### Ayumi Index\n\nhttp://ayumi.m8geil.de/erp4_chatlogs/?S=rma_0#!/index\n\nIn the Ayumi ERPv4 Chat Log Index, SnowLotus scores a 94.10 in Flesch which means it produces more complex sentences than Daring (quite complex), DaringLotus scores higher in Var and Ad[jv], which means it makes heavier use of adjectives and adverbs (is more descriptive). Noteably Daring is in the top 8 for adjectives in a sentence, highest in it's weight class if you discount the chinese model, and in general both models did very well on this metric (SnowLotus ranks higher here than anything above it in IQ4), showcasing their descriptive ability. \n\nSnowLotus beats DaringLotus on IQ4 with a score of 70.94, only bet by SOLAR Instruct and Fimbulvetr in it's weight class (altho also noteably Kunoichi 7b by a slim margin), DaringLotus is a bit lower at 65.37 - not as smart. \n\nInterestingly the benchmarking here showed repetition for both models (which I haven't seen), but more with SnowLotus - so it's possible Daring repeats less than SnowLotus? These roughly confirm my impressions of the differences, altho potentially reveal some new details too. I've had a great experience RPing with these models, and seen no repetition myself, but be sure to use MinP or DynaTemp rather than the older samplers and be prepared to regen anything they get stuck on!\n\n",
    "related_quantizations": []
  },
  "tags": [
    "gguf",
    "endpoints_compatible",
    "region:us"
  ],
  "likes": 0,
  "downloads": 468,
  "gated": false,
  "private": false,
  "last_modified": "2024-07-30T01:45:07.000Z",
  "created_at": "2024-07-29T21:30:18.000Z",
  "pipeline_tag": "",
  "library_name": ""
}

{
  "_id": "66a809ea809a6e1e87c20fef",
  "id": "RichardErkhov/BlueNipples_-_SnowLotus-v2-10.7B-gguf",
  "modelId": "RichardErkhov/BlueNipples_-_SnowLotus-v2-10.7B-gguf",
  "sha": "3cb5f13e4a47909d2f6aa3f0bcae11ad5f10c201",
  "createdAt": "2024-07-29T21:30:18.000Z",
  "lastModified": "2024-07-30T01:45:07.000Z",
  "author": "RichardErkhov",
  "downloads": 468,
  "likes": 0,
  "gated": false,
  "private": false,
  "pipeline_tag": "",
  "library_name": "",
  "siblings_count": 24
}