lewdiculous/erosumika-7b-v3-0.2-gguf-iq-imatrix overview

{
  "metadata": {},
  "card_data": {
    "language": [
      "en"
    ],
    "pipeline_tag": "text-generation",
    "tags": [
      "text-generation-inference",
      "instruct",
      "conversational",
      "roleplay",
      "mistral",
      "merge",
      "sillytavern"
    ],
    "license": "cc-by-4.0",
    "frontmatter": {
      "language": [
        "en"
      ],
      "pipeline_tag": "text-generation",
      "tags": [
        "text-generation-inference",
        "instruct",
        "conversational",
        "roleplay",
        "mistral",
        "merge",
        "sillytavern"
      ],
      "license": "cc-by-4.0"
    },
    "hero_image_url": "https://cdn-uploads.huggingface.co/production/uploads/6512681f4151fb1fa719e033/8YBKfcegQZliRlQNm0oir.gif",
    "summary": "Erosumika-7B-v3-0.2 ~Mistral 0.2 Edition~",
    "quick_links": [],
    "benchmark_table_html": "",
    "readme_markdown": "---\nlanguage:\n- en\npipeline_tag: text-generation\ntags:\n- text-generation-inference\n- instruct\n- conversational\n- roleplay\n- mistral\n- merge\n- sillytavern\nlicense: cc-by-4.0\n---\n\nThis repo contains GGUF-IQ-Imatrix quantized model files for [Erosumika-7B-v3-0.2](https://huggingface.co/localfultonextractor/Erosumika-7B-v3-0.2).\n\n**Recommended starting [SillyTavern presets here](https://huggingface.co/Lewdiculous/Model-Requests/tree/main/data/presets/lewdicu-3.0.2-mistral-0.2).**\n\nQuants:\n\n         \"Q4_K_M\", \"Q4_K_S\", \"IQ4_XS\", \"Q5_K_M\", \"Q5_K_S\",\n         \"Q6_K\", \"Q8_0\", \"IQ3_M\", \"IQ3_S\", \"IQ3_XXS\"\n\n![image/png](https://cdn-uploads.huggingface.co/production/uploads/65d4cf2693a0a3744a27536c/L2l7DXKyLTD6MuL16JPn1.png)\n\n**What does \"Imatrix\" mean?**\n\nIt stands for **Importance Matrix**, a technique used to improve the quality of quantized models.\nThe **Imatrix** is calculated based on calibration data, and it helps determine the importance of different model activations during the quantization process.\nThe idea is to preserve the most important information during quantization, which can help reduce the loss of model performance, especially when the calibration data is diverse.\n[[1]](https://github.com/ggerganov/llama.cpp/discussions/5006) [[2]](https://github.com/ggerganov/llama.cpp/discussions/5263#discussioncomment-8395384)\n\nFor imatrix data generation, kalomaze's `groups_merged.txt` with added roleplay chats was used, you can find it [here](https://huggingface.co/Lewdiculous/Datura_7B-GGUF-Imatrix/blob/main/imatrix-with-rp-format-data.txt). This was just to add a bit more diversity to the data.\n\n**Steps:**\n\n```\nBase⇢ GGUF(F16)⇢ Imatrix-Data(F16)⇢ GGUF(Imatrix-Quants)\n```\n*Using the latest llama.cpp at the time.*\n\n# Original model information:\n\n<h1 style=\"text-align: center\">Erosumika-7B-v3-0.2</h1>\n<h2 style=\"text-align: center\">~Mistral 0.2 Edition~</h1>\n\n<div style=\"display: flex; justify-content: center;\">\n    <img src=\"https://cdn-uploads.huggingface.co/production/uploads/6512681f4151fb1fa719e033/8YBKfcegQZliRlQNm0oir.gif\" alt=\"Header GIF\">\n</div>\n\n## Model Details\nThe Mistral 0.2 version of Erosumika-7B-v3, a DARE TIES merge between Nitral's [Kunocchini-7b](https://huggingface.co/Nitral-AI/Kunocchini-7b), Endevor's [InfinityRP-v1-7B](https://huggingface.co/Endevor/InfinityRP-v1-7B) and my [FlatErosAlpha](https://huggingface.co/localfultonextractor/FlatErosAlpha), a flattened(in order to keep the vocab size 32000) version of tavtav's [eros-7B-ALPHA](https://huggingface.co/tavtav/eros-7B-ALPHA). Alpaca and ChatML work best. Slightly smarter and better prompt comprehension than Mistral 0.1 Erosumika-7B-v3. 32k context should work.\n\n[GGUF quants](https://huggingface.co/localfultonextractor/Erosumika-7B-v3-0.2-GGUF)\n\n\n## Limitations and biases\nThe intended use-case for this model is fictional writing for entertainment purposes. Any other sort of usage is out of scope. \nIt may produce socially unacceptable or undesirable text, even if the prompt itself does not include anything explicitly offensive. Outputs might often be factually wrong or misleading.\n\n\n```yaml\nmerge_method: task_arithmetic\nbase_model: alpindale/Mistral-7B-v0.2-hf\nmodels:\n  - model: localfultonextractor/Erosumika-7B-v3\n    parameters:\n      weight: 1.0\ndtype: float16\n```",
    "related_quantizations": []
  },
  "tags": [
    "gguf",
    "text-generation-inference",
    "instruct",
    "conversational",
    "roleplay",
    "mistral",
    "merge",
    "sillytavern",
    "text-generation",
    "en",
    "license:cc-by-4.0",
    "endpoints_compatible",
    "region:us"
  ],
  "likes": 31,
  "downloads": 323,
  "gated": false,
  "private": false,
  "last_modified": "2024-03-27T10:47:03.000Z",
  "created_at": "2024-03-26T22:25:29.000Z",
  "pipeline_tag": "text-generation",
  "library_name": ""
}

{
  "_id": "66034b59d4c6a6264df11d6e",
  "id": "Lewdiculous/Erosumika-7B-v3-0.2-GGUF-IQ-Imatrix",
  "modelId": "Lewdiculous/Erosumika-7B-v3-0.2-GGUF-IQ-Imatrix",
  "sha": "ac6d8bcaa051a39b18f0f2dc41cbaee7bf718270",
  "createdAt": "2024-03-26T22:25:29.000Z",
  "lastModified": "2024-03-27T10:47:03.000Z",
  "author": "Lewdiculous",
  "downloads": 323,
  "likes": 31,
  "gated": false,
  "private": false,
  "pipeline_tag": "text-generation",
  "library_name": "",
  "siblings_count": 18
}