nexesquants/teezee_kyllene-yi-34b-v1.1-imat.gguf overview

{
  "metadata": {},
  "card_data": {
    "frontmatter": {},
    "hero_image_url": "https://cdn-uploads.huggingface.co/production/uploads/6451b24dc5d273f95482bfa4/dNJECS98SGzQBSNEM3lsS.jpeg",
    "summary": "Quants with iMatrix for : https://huggingface.co/TeeZee/Kyllene-34B-v1.1 Non-iMatrix quants (more choice in higher bitrate quants) : https://huggingface.co/TeeZee/Kyllene-34B-v1.1-GGUF/tree/main !image/jpeg --- TeeZee's Kyllene 34B v1.1 model is one of the best Yi_34b merge around with those of BruceTheMoose. But it has a little thing which distinguishes it : It uses Gryphe's MergeMonster as a tool to trim out the GPTisms, Yisms, and Llamaisms, and give a more natural output. The clearing of any problematic gptism, llamaism, or yiism which was specified to MergeMonster is noticeable And it's like the model is freed of these sequences which represent some form of \"EOS chains of tokens\" in many models, this in the sense that they conclude many outputs, this ofc in an unwanted way It's quite a step in the right direction which should become the standard practice. That make me wonder about the future, when we'll get Miqu 70b models properly finetuned with the best datatsets AND with the Mistralisms trimmed out as well. --- Available quants : Full offload possible on 48GB VRAM with a huge context size : Q8_0 Full offload possible on 36 GB VRAM with a huge context size : Q5_K_S Full offload possible on 24GB VRAM with a big to huge context size (from 12288 with Q4_K_M, for example) : Q4_K_M, Q4_K_S, Q3_K_M Full offload possible on 16GB VRAM with a decent context size : IQ3_XXS SOTA (which is equivalent to a Q3_K_S with more context!), Q2_K, Q2_K_S Full offload possible on 12GB VRAM with a decent context size : IQ2_XS SOTA. lower quality : IQ2_XXS SOTA Full offload maybe possible on 8GB VRAM with a small context size : IQ1_S revision \"even better\" (b2404) (or v5). All my IQ1_S quant from the 13/03/2024 will be with this new IQ1_S quantization base. --- The merge parameters and logs are in the repo : https://huggingface.co/TeeZee/Kyllene-34B-v1.1/tree/main --- After iMatrixing and quantizing Kyllene, I benched her thoroughly, and she proved herself worthy : Q4_K_S : Q4_K_M : Q5_K_S : ----- IQ1_S V5 : ----- Enjoy these quants!",
    "quick_links": [],
    "benchmark_table_html": "",
    "readme_markdown": "Quants with iMatrix for : https://huggingface.co/TeeZee/Kyllene-34B-v1.1\n\nNon-iMatrix quants (more choice in higher bitrate quants) : https://huggingface.co/TeeZee/Kyllene-34B-v1.1-GGUF/tree/main\n\n![image/jpeg](https://cdn-uploads.huggingface.co/production/uploads/6451b24dc5d273f95482bfa4/dNJECS98SGzQBSNEM3lsS.jpeg)\n\n---\n\nTeeZee's Kyllene 34B v1.1 model is one of the best Yi_34b merge around with those of BruceTheMoose.\n\nBut it has a little thing which distinguishes it :\n\nIt uses Gryphe's MergeMonster as a tool to trim out the GPTisms, Yisms, and Llamaisms, and give a more natural output.\n\nThe clearing of any problematic gptism, llamaism, or yiism which was specified to MergeMonster is noticeable\nAnd it's like the model is freed of these sequences which represent some form of \"EOS chains of tokens\" in many models, this in the sense that they conclude many outputs, this ofc in an unwanted way\nIt's quite a step in the right direction which should become the standard practice.\n\nThat make me wonder about the future, when we'll get Miqu 70b models properly finetuned with the best datatsets AND with the Mistralisms trimmed out as well.\n\n---\n\nAvailable quants :\n\nFull offload possible on 48GB VRAM with a huge context size : Q8_0\n\nFull offload possible on 36 GB VRAM with a huge context size : Q5_K_S\n\nFull offload possible on 24GB VRAM with a big to huge context size (from 12288 with Q4_K_M, for example) : Q4_K_M, Q4_K_S, Q3_K_M\n\nFull offload possible on 16GB VRAM with a decent context size : IQ3_XXS SOTA (which is equivalent to a Q3_K_S with more context!), Q2_K, Q2_K_S\n\nFull offload possible on 12GB VRAM with a decent context size : IQ2_XS SOTA. lower quality : IQ2_XXS SOTA\n\nFull offload maybe possible on 8GB VRAM with a small context size : IQ1_S revision \"even better\" (b2404) (or v5).\nAll my IQ1_S quant from the 13/03/2024 will be with this new IQ1_S quantization base.\n\n---\n\nThe merge parameters and logs are in the repo : https://huggingface.co/TeeZee/Kyllene-34B-v1.1/tree/main\n\n---\n\nAfter iMatrixing and quantizing Kyllene, I benched her thoroughly, and she proved herself worthy :\n\nQ4_K_S :\n- Kyllene-34B-v1.1-b1989-iMat-c32_ch3250-Q4_K_S.gguf,-,Hellaswag,85,,400,2024-01-28 00:00:00,,34b,Yi,200000,,,GGUF,TeeZee,Nexesenex,\n- Kyllene-34B-v1.1-b1989-iMat-c32_ch3250-Q4_K_S.gguf,-,Hellaswag,85.2,,1000,2024-01-28 00:00:00,,34b,Yi,200000,,,GGUF,TeeZee,Nexesenex,\n- Kyllene-34B-v1.1-b1989-iMat-c32_ch3250-Q4_K_S.gguf,-,Hellaswag,84.6,,2000,2024-01-28 00:00:00,,34b,Yi,200000,,,GGUF,TeeZee,Nexesenex,\n- Kyllene-34B-v1.1-b1989-iMat-c32_ch3250-Q4_K_S.gguf,-,Hellaswag_Bin,81,,400,2024-01-28 00:00:00,,34b,Yi,200000,,,GGUF,TeeZee,Nexesenex,\n- Kyllene-34B-v1.1-b1989-iMat-c32_ch3250-Q4_K_S.gguf,-,Hellaswag_Bin,83.5,,1000,2024-01-28 00:00:00,,34b,Yi,200000,,,GGUF,TeeZee,Nexesenex,\n- Kyllene-34B-v1.1-b1989-iMat-c32_ch3250-Q4_K_S.gguf,-,Hellaswag_Bin,82.95,,2000,2024-01-28 00:00:00,,34b,Yi,200000,,,GGUF,TeeZee,Nexesenex,\n- Kyllene-34B-v1.1-b1989-iMat-c32_ch3250-Q4_K_S.gguf,-,Arc-Challenge,61.53846154,,299,2024-01-28 05:40:00,,34b,Yi,200000,,,GGUF,TeeZee,Nexesenex,\n- Kyllene-34B-v1.1-b1989-iMat-c32_ch3250-Q4_K_S.gguf,-,Arc-Easy,80.35087719,,570,2024-01-28 05:40:00,,34b,Yi,200000,,,GGUF,TeeZee,Nexesenex,\n- Kyllene-34B-v1.1-b1989-iMat-c32_ch3250-Q4_K_S.gguf,-,MMLU,43.13099042,,313,2024-01-28 05:40:00,,34b,Yi,200000,,,GGUF,TeeZee,Nexesenex,\n- Kyllene-34B-v1.1-b1989-iMat-c32_ch3250-Q4_K_S.gguf,-,Thruthful-QA,35.00611995,,817,2024-01-28 05:40:00,,34b,Yi,200000,,,GGUF,TeeZee,Nexesenex,\n- Kyllene-34B-v1.1-b1989-iMat-c32_ch3250-Q4_K_S.gguf,-,Winogrande,79.3212,,1267,2024-01-28 05:40:00,,34b,Yi,200000,,,GGUF,TeeZee,Nexesenex,\n- Kyllene-34B-v1.1-b1989-iMat-c32_ch3250-Q4_K_S.gguf,-,wikitext,5.1703,512,512,2024-01-28 00:00:00,,34b,Yi,200000,,,GGUF,TeeZee,Nexesenex,\n\nQ4_K_M :\n- Kyllene-34B-v1.1-b1989-iMat-c32_ch3250-Q4_K_M.gguf,-,Hellaswag,84.75,,400,2024-01-28 00:00:00,,34b,Yi,200000,,,GGUF,TeeZee,Nexesenex,\n- Kyllene-34B-v1.1-b1989-iMat-c32_ch3250-Q4_K_M.gguf,-,Hellaswag,85.6,,1000,2024-01-28 00:00:00,,34b,Yi,200000,,,GGUF,TeeZee,Nexesenex,\n- Kyllene-34B-v1.1-b1989-iMat-c32_ch3250-Q4_K_M.gguf,-,Hellaswag,84.9,,2000,2024-01-28 00:00:00,,34b,Yi,200000,,,GGUF,TeeZee,Nexesenex,\n- Kyllene-34B-v1.1-b1989-iMat-c32_ch3250-Q4_K_M.gguf,-,Hellaswag_Bin,81,,400,2024-01-28 00:00:00,,34b,Yi,200000,,,GGUF,TeeZee,Nexesenex,\n- Kyllene-34B-v1.1-b1989-iMat-c32_ch3250-Q4_K_M.gguf,-,Hellaswag_Bin,83.4,,1000,2024-01-28 00:00:00,,34b,Yi,200000,,,GGUF,TeeZee,Nexesenex,\n- Kyllene-34B-v1.1-b1989-iMat-c32_ch3250-Q4_K_M.gguf,-,Hellaswag_Bin,82.9,,2000,2024-01-28 00:00:00,,34b,Yi,200000,,,GGUF,TeeZee,Nexesenex,\n- Kyllene-34B-v1.1-b1989-iMat-c32_ch3250-Q4_K_M.gguf,-,Arc-Challenge,60.53511706,,299,2024-01-28 05:40:00,,34b,Yi,200000,,,GGUF,TeeZee,Nexesenex,\n- Kyllene-34B-v1.1-b1989-iMat-c32_ch3250-Q4_K_M.gguf,-,Arc-Easy,80.52631579,,570,2024-01-28 05:40:00,,34b,Yi,200000,,,GGUF,TeeZee,Nexesenex,\n- Kyllene-34B-v1.1-b1989-iMat-c32_ch3250-Q4_K_M.gguf,-,MMLU,42.49201278,,313,2024-01-28 05:40:00,,34b,Yi,200000,,,GGUF,TeeZee,Nexesenex,\n- Kyllene-34B-v1.1-b1989-iMat-c32_ch3250-Q4_K_M.gguf,-,Thruthful-QA,34.39412485,,817,2024-01-28 05:40:00,,34b,Yi,200000,,,GGUF,TeeZee,Nexesenex,\n- Kyllene-34B-v1.1-b1989-iMat-c32_ch3250-Q4_K_M.gguf,-,Winogrande,79.4791,,1267,2024-01-28 05:40:00,,34b,Yi,200000,,,GGUF,TeeZee,Nexesenex,\n- Kyllene-34B-v1.1-b1989-iMat-c32_ch3250-Q4_K_M.gguf,-,wikitext,5.1679,512,512,2024-01-28 00:00:00,,34b,Yi,200000,,,GGUF,TeeZee,Nexesenex,\n- Kyllene-34B-v1.1-b1989-iMat-c32_ch3250-Q4_K_M.gguf,-,wikitext,4.3623,4096,4096,2024-01-28 00:00:00,,34b,Yi,200000,,,GGUF,TeeZee,Nexesenex,\n- Kyllene-34B-v1.1-b1989-iMat-c32_ch3250-Q4_K_M.gguf,-,wikitext,4.4061,8192,8192,2024-01-28 00:00:00,,34b,Yi,200000,,,GGUF,TeeZee,Nexesenex,\n\nQ5_K_S :\n- Kyllene-34B-v1.1-b1989-iMat-c32_ch3250-Q5_K_S.gguf,-,Hellaswag,85.25,,400,2024-01-28 00:00:00,,34b,Yi,200000,,,GGUF,TeeZee,Nexesenex,\n- Kyllene-34B-v1.1-b1989-iMat-c32_ch3250-Q5_K_S.gguf,-,Hellaswag,85.6,,1000,2024-01-28 00:00:00,,34b,Yi,200000,,,GGUF,TeeZee,Nexesenex,\n- Kyllene-34B-v1.1-b1989-iMat-c32_ch3250-Q5_K_S.gguf,-,Hellaswag,84.95,,2000,2024-01-28 00:00:00,,34b,Yi,200000,,,GGUF,TeeZee,Nexesenex,\n- Kyllene-34B-v1.1-b1989-iMat-c32_ch3250-Q5_K_S.gguf,-,Hellaswag_Bin,81.25,,400,2024-01-28 00:00:00,,34b,Yi,200000,,,GGUF,TeeZee,Nexesenex,\n- Kyllene-34B-v1.1-b1989-iMat-c32_ch3250-Q5_K_S.gguf,-,Hellaswag_Bin,83.3,,1000,2024-01-28 00:00:00,,34b,Yi,200000,,,GGUF,TeeZee,Nexesenex,\n- Kyllene-34B-v1.1-b1989-iMat-c32_ch3250-Q5_K_S.gguf,-,Hellaswag_Bin,83,,2000,2024-01-28 00:00:00,,34b,Yi,200000,,,GGUF,TeeZee,Nexesenex,\n- Kyllene-34B-v1.1-b1989-iMat-c32_ch3250-Q5_K_S.gguf,-,Arc-Challenge,60.20066890,,299,2024-01-28 05:40:00,,34b,Yi,200000,,,GGUF,TeeZee,Nexesenex,\n- Kyllene-34B-v1.1-b1989-iMat-c32_ch3250-Q5_K_S.gguf,-,Arc-Easy,81.05263158,,570,2024-01-28 05:40:00,,34b,Yi,200000,,,GGUF,TeeZee,Nexesenex,\n- Kyllene-34B-v1.1-b1989-iMat-c32_ch3250-Q5_K_S.gguf,-,MMLU,42.17252396,,313,2024-01-28 05:40:00,,34b,Yi,200000,,,GGUF,TeeZee,Nexesenex,\n- Kyllene-34B-v1.1-b1989-iMat-c32_ch3250-Q5_K_S.gguf,-,Thruthful-QA,36.96450428,,817,2024-01-28 05:40:00,,34b,Yi,200000,,,GGUF,TeeZee,Nexesenex,\n- Kyllene-34B-v1.1-b1989-iMat-c32_ch3250-Q5_K_S.gguf,-,Winogrande,79.5580,,1267,2024-01-28 05:40:00,,34b,Yi,200000,,,GGUF,TeeZee,Nexesenex,\n- Kyllene-34B-v1.1-b1989-iMat-c32_ch3250-Q5_K_S.gguf,-,wikitext,5.1806,512,512,2024-01-28 00:00:00,,34b,Yi,200000,,,GGUF,TeeZee,Nexesenex,\n\n-----\n\nIQ1_S V5 :\n\n- TeeZee_Kyllene-34B-v1.1-b2409-iMat-c32_ch3250-IQ1_S_v5.gguf,-,Hellaswag,70.3,,1000,2024-03-12 00:00:00,,34b,Yi,2000000,,,GGUF,TeeZee,Nexesenex,\n- TeeZee_Kyllene-34B-v1.1-b2409-iMat-c32_ch3250-IQ1_S_v5.gguf,-,Arc-Challenge,40.46822742,299,2024-03-12 00:00:00,,34b,Yi,2000000,,,GGUF,TeeZee,Nexesenex,\n- TeeZee_Kyllene-34B-v1.1-b2409-iMat-c32_ch3250-IQ1_S_v5.gguf,-,Arc-Easy,62.28070175,,570,2024-03-12 00:00:00,,34b,Yi,2000000,,,GGUF,TeeZee,Nexesenex,\n- TeeZee_Kyllene-34B-v1.1-b2409-iMat-c32_ch3250-IQ1_S_v5.gguf,-,MMLU,32.90734824,,313,2024-03-12 00:00:00,,34b,Yi,2000000,,,GGUF,TeeZee,Nexesenex,\n- TeeZee_Kyllene-34B-v1.1-b2409-iMat-c32_ch3250-IQ1_S_v5.gguf,-,Thruthful-QA,29.37576499,,817,2024-03-12 00:00:00,,34b,Yi,2000000,,,GGUF,TeeZee,Nexesenex,\n- TeeZee_Kyllene-34B-v1.1-b2409-iMat-c32_ch3250-IQ1_S_v5.gguf,-,Winogrande,68.7451,,1267,2024-03-12 00:00:00,,34b,Yi,2000000,,,GGUF,TeeZee,Nexesenex,\n- TeeZee_Kyllene-34B-v1.1-b2409-iMat-c32_ch3250-IQ1_S_v5.gguf,-,wikitext,9.8761,512,512,2024-03-12 00:00:00,,34b,Yi,2000000,,,GGUF,TeeZee,Nexesenex,\n- TeeZee_Kyllene-34B-v1.1-b2409-iMat-c32_ch3250-IQ1_S_v5.gguf,-,wikitext,7.8954,4096,4096,2024-03-12 00:00:00,,34b,Yi,2000000,,,GGUF,TeeZee,Nexesenex,\n\n-----\n\nEnjoy these quants!",
    "related_quantizations": []
  },
  "tags": [
    "gguf",
    "endpoints_compatible",
    "region:us",
    "conversational"
  ],
  "likes": 27,
  "downloads": 507,
  "gated": false,
  "private": false,
  "last_modified": "2024-10-12T23:32:50.000Z",
  "created_at": "2024-01-28T08:01:02.000Z",
  "pipeline_tag": "",
  "library_name": ""
}

{
  "_id": "65b609be336df60785ebfec0",
  "id": "NexesQuants/TeeZee_Kyllene-Yi-34B-v1.1-iMat.GGUF",
  "modelId": "NexesQuants/TeeZee_Kyllene-Yi-34B-v1.1-iMat.GGUF",
  "sha": "4d40b9a38b076c428f1af2cbbb778744c4db051c",
  "createdAt": "2024-01-28T08:01:02.000Z",
  "lastModified": "2024-10-12T23:32:50.000Z",
  "author": "NexesQuants",
  "downloads": 507,
  "likes": 27,
  "gated": false,
  "private": false,
  "pipeline_tag": "",
  "library_name": "",
  "siblings_count": 29
}