dumb-dev/flan-t5-xxl-gguf overview

{
  "metadata": {},
  "card_data": {
    "license": "apache-2.0",
    "language": [
      "de",
      "fr",
      "en",
      "ro"
    ],
    "base_model": [
      "google/flan-t5-xxl"
    ],
    "library_name": "llama.cpp",
    "tags": [
      "llama.cpp"
    ],
    "frontmatter": {
      "license": "apache-2.0",
      "language": [
        "de",
        "fr",
        "en",
        "ro"
      ],
      "base_model": [
        "google/flan-t5-xxl"
      ],
      "library_name": "llama.cpp",
      "tags": [
        "llama.cpp"
      ]
    },
    "hero_image_url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/model_doc/flan2_architecture.jpg",
    "summary": "## This is a quantized version of google/flan-t5-xxl !Google Original Model Architecture",
    "quick_links": [],
    "benchmark_table_html": "",
    "readme_markdown": "---\nlicense: apache-2.0\nlanguage:\n- de\n- fr\n- en\n- ro\nbase_model:\n- google/flan-t5-xxl\n\nlibrary_name: llama.cpp\n\ntags:\n- llama.cpp\n\n---\n\n# flan-t5-xxl-gguf \n## This is a quantized version of [google/flan-t5-xxl](https://huggingface.co/google/flan-t5-xxl/)\n![Google Original Model Architecture](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/model_doc/flan2_architecture.jpg)\n\n\n\n\n\n## Usage/Examples\n\n```sh\n./llama-cli -m /path/to/file.gguf --prompt \"your prompt\" --n-gpu-layers nn\n```\nnn --> numbers of layers to offload to gpu\n\n## Quants\n\nBITs     |       TYPE    |\n--------|-------------  |\nQ2      |       Q2_K     |\nQ3      | Q3_K, Q3_K_L, Q3_K_M, Q3_K_S  | \nQ4      | Q4_0, Q4_1, Q4_K, Q4_K_M, Q4_K_S  |\nQ5      | Q5_0, Q5_1, Q5_K, Q5_K_M, Q5_K_S  |\nQ6      | Q6_K          |\nQ8      | Q8_0       |\n\n#### Additional:\nBITs     | TYPE/float    |\n--------|-------------  |\n16     |       f16   |\n32      | f32  | \n\n\n\n## Disclaimer\n I don't claim any rights on this model. All rights go to google.\n## Acknowledgements\n\n - [Original model](https://huggingface.co/google/flan-t5-xxl/)\n - [Original README](https://huggingface.co/google/flan-t5-xxl/blob/main/README.md)\n - [Original license](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md)\n\n",
    "related_quantizations": []
  },
  "tags": [
    "llama.cpp",
    "gguf",
    "de",
    "fr",
    "en",
    "ro",
    "base_model:google/flan-t5-xxl",
    "base_model:quantized:google/flan-t5-xxl",
    "license:apache-2.0",
    "endpoints_compatible",
    "region:us"
  ],
  "likes": 18,
  "downloads": 636,
  "gated": false,
  "private": false,
  "last_modified": "2024-10-29T23:21:30.000Z",
  "created_at": "2024-10-28T09:01:29.000Z",
  "pipeline_tag": "",
  "library_name": "llama.cpp"
}

{
  "_id": "671f52e9748faf685e1d0070",
  "id": "dumb-dev/flan-t5-xxl-gguf",
  "modelId": "dumb-dev/flan-t5-xxl-gguf",
  "sha": "f795b14c218fa149b78955bdae0bde710ddf82cf",
  "createdAt": "2024-10-28T09:01:29.000Z",
  "lastModified": "2024-10-29T23:21:30.000Z",
  "author": "dumb-dev",
  "downloads": 636,
  "likes": 18,
  "gated": false,
  "private": false,
  "pipeline_tag": "",
  "library_name": "llama.cpp",
  "siblings_count": 21
}