inferenceillusionist/llama-3-70b-instruct-storywriter-imat-gguf overview

{
  "metadata": {},
  "card_data": {
    "tags": [
      "merge",
      "gguf",
      "llama3",
      "iMat"
    ],
    "frontmatter": {
      "tags": [
        "merge",
        "gguf",
        "llama3",
        "iMat"
      ]
    },
    "hero_image_url": "https://i.imgur.com/P68dXux.png",
    "summary": "Special request. Quantized from fp32 with love. If you can't fit IQ quants in your VRAM, try using the K quants in this repo instead. * The .imatrix file in this repo was created using the Q8_0 quantization of Llama-3-70B-Instruct-Storywriter-iMat-GGUF. * Calculated in 88 chunks with n_ctx=512 using groups_merged.txt For a brief rundown of iMatrix quant performance please see this PR All quants are verified working prior to uploading to repo for your safety and convenience.  Tip: Pick a file size under your GPU's VRAM while still allowing some room for context for best speed. You may need to pad this further depending on if you are running image gen or TTS as well. BFloat16 model card can be found here",
    "quick_links": [],
    "benchmark_table_html": "",
    "readme_markdown": "---\ntags:\n- merge\n- gguf\n- llama3\n- iMat\n---\n<img src=\"https://i.imgur.com/P68dXux.png\" width=\"400\"/>\n\n# Llama-3-70B-Instruct-Storywriter-iMat-GGUF\n\n\n<b>Special request.</b> Quantized from fp32 with love. If you can't fit IQ quants in your VRAM, try using the K quants in this repo instead.\n* The [.imatrix](https://huggingface.co/InferenceIllusionist/Llama-3-70B-Instruct-Storywriter-iMat-GGUF/resolve/main/Llama-3-70B-Instruct-Storywriter.imatrix?download=true) file in this repo was created using the Q8_0 quantization of Llama-3-70B-Instruct-Storywriter-iMat-GGUF.\n* Calculated in 88 chunks with n_ctx=512 using groups_merged.txt\n\nFor a brief rundown of iMatrix quant performance please see this [PR](https://github.com/ggerganov/llama.cpp/pull/5747)\n\n<i>All quants are verified working prior to uploading to repo for your safety and convenience. </i>\n\n\n<b>Tip:</b> Pick a file size under your GPU's VRAM while still allowing some room for context for best speed. You may need to pad this further depending on if you are running image gen or TTS as well.\n\nBFloat16 model card can be found [here](https://huggingface.co/tdrussell/Llama-3-70B-Instruct-Storywriter)",
    "related_quantizations": []
  },
  "tags": [
    "gguf",
    "merge",
    "llama3",
    "iMat",
    "endpoints_compatible",
    "region:us",
    "imatrix",
    "conversational"
  ],
  "likes": 0,
  "downloads": 241,
  "gated": false,
  "private": false,
  "last_modified": "2024-05-11T19:25:28.000Z",
  "created_at": "2024-05-01T08:42:19.000Z",
  "pipeline_tag": "",
  "library_name": ""
}

{
  "_id": "6632006b1fd3ec937aff5a80",
  "id": "InferenceIllusionist/Llama-3-70B-Instruct-Storywriter-iMat-GGUF",
  "modelId": "InferenceIllusionist/Llama-3-70B-Instruct-Storywriter-iMat-GGUF",
  "sha": "1b17f5064cd2d36ec09a5f6f1bf88ff75e4640cc",
  "createdAt": "2024-05-01T08:42:19.000Z",
  "lastModified": "2024-05-11T19:25:28.000Z",
  "author": "InferenceIllusionist",
  "downloads": 241,
  "likes": 0,
  "gated": false,
  "private": false,
  "pipeline_tag": "",
  "library_name": "",
  "siblings_count": 23
}