cstr/aya-expanse-8b-q4_k_m-gguf overview

{
  "metadata": {},
  "card_data": {
    "inference": false,
    "library_name": "transformers",
    "language": [
      "en",
      "fr",
      "de",
      "es",
      "it",
      "pt",
      "ja",
      "ko",
      "zh",
      "ar",
      "el",
      "fa",
      "pl",
      "id",
      "cs",
      "he",
      "hi",
      "nl",
      "ro",
      "ru",
      "tr",
      "uk",
      "vi"
    ],
    "license": "cc-by-nc-4.0",
    "extra_gated_prompt": "By submitting this form, you agree to the [License Agreement](https://cohere.com/c4ai-cc-by-nc-license)  and acknowledge that the information you provide will be collected, used, and shared in accordance with Cohere’s [Privacy Policy]( https://cohere.com/privacy). You’ll receive email updates about C4AI and Cohere research, events, products and services. You can unsubscribe at any time.",
    "extra_gated_fields": {
      "Name": "text",
      "Affiliation": "text",
      "Country": "country",
      "I agree to use this model for non-commercial use ONLY": "checkbox"
    },
    "base_model": "CohereForAI/aya-expanse-8b",
    "tags": [
      "llama-cpp",
      "gguf-my-repo"
    ],
    "frontmatter": {
      "inference": "false",
      "library_name": "transformers",
      "language": [
        "en",
        "fr",
        "de",
        "es",
        "it",
        "pt",
        "ja",
        "ko",
        "zh",
        "ar",
        "el",
        "fa",
        "pl",
        "id",
        "cs",
        "he",
        "hi",
        "nl",
        "ro",
        "ru",
        "tr",
        "uk",
        "vi"
      ],
      "license": "cc-by-nc-4.0",
      "extra_gated_prompt": "By submitting this form, you agree to the [License Agreement](https://cohere.com/c4ai-cc-by-nc-license)  and",
      "extra_gated_fields": [],
      "base_model": "CohereForAI/aya-expanse-8b",
      "tags": [
        "llama-cpp",
        "gguf-my-repo"
      ]
    },
    "hero_image_url": "",
    "summary": "This model was converted to GGUF format from CohereForAI/aya-expanse-8b using llama.cpp via the ggml.ai's GGUF-my-repo space. Refer to the original model card for more details on the model.",
    "quick_links": [],
    "benchmark_table_html": "",
    "readme_markdown": "---\ninference: false\nlibrary_name: transformers\nlanguage:\n- en\n- fr\n- de\n- es\n- it\n- pt\n- ja\n- ko\n- zh\n- ar\n- el\n- fa\n- pl\n- id\n- cs\n- he\n- hi\n- nl\n- ro\n- ru\n- tr\n- uk\n- vi\nlicense: cc-by-nc-4.0\nextra_gated_prompt: By submitting this form, you agree to the [License Agreement](https://cohere.com/c4ai-cc-by-nc-license)  and\n  acknowledge that the information you provide will be collected, used, and shared\n  in accordance with Cohere’s [Privacy Policy]( https://cohere.com/privacy). You’ll\n  receive email updates about C4AI and Cohere research, events, products and services.\n  You can unsubscribe at any time.\nextra_gated_fields:\n  Name: text\n  Affiliation: text\n  Country: country\n  I agree to use this model for non-commercial use ONLY: checkbox\nbase_model: CohereForAI/aya-expanse-8b\ntags:\n- llama-cpp\n- gguf-my-repo\n---\n\n# cstr/aya-expanse-8b-Q4_K_M-GGUF\nThis model was converted to GGUF format from [`CohereForAI/aya-expanse-8b`](https://huggingface.co/CohereForAI/aya-expanse-8b) using llama.cpp via the ggml.ai's [GGUF-my-repo](https://huggingface.co/spaces/ggml-org/gguf-my-repo) space.\nRefer to the [original model card](https://huggingface.co/CohereForAI/aya-expanse-8b) for more details on the model.\n\n## Use with llama.cpp\nInstall llama.cpp through brew (works on Mac and Linux)\n\n```bash\nbrew install llama.cpp\n\n```\nInvoke the llama.cpp server or the CLI.\n\n### CLI:\n```bash\nllama-cli --hf-repo cstr/aya-expanse-8b-Q4_K_M-GGUF --hf-file aya-expanse-8b-q4_k_m.gguf -p \"The meaning to life and the universe is\"\n```\n\n### Server:\n```bash\nllama-server --hf-repo cstr/aya-expanse-8b-Q4_K_M-GGUF --hf-file aya-expanse-8b-q4_k_m.gguf -c 2048\n```\n\nNote: You can also use this checkpoint directly through the [usage steps](https://github.com/ggerganov/llama.cpp?tab=readme-ov-file#usage) listed in the Llama.cpp repo as well.\n\nStep 1: Clone llama.cpp from GitHub.\n```\ngit clone https://github.com/ggerganov/llama.cpp\n```\n\nStep 2: Move into the llama.cpp folder and build it with `LLAMA_CURL=1` flag along with other hardware-specific flags (for ex: LLAMA_CUDA=1 for Nvidia GPUs on Linux).\n```\ncd llama.cpp && LLAMA_CURL=1 make\n```\n\nStep 3: Run inference through the main binary.\n```\n./llama-cli --hf-repo cstr/aya-expanse-8b-Q4_K_M-GGUF --hf-file aya-expanse-8b-q4_k_m.gguf -p \"The meaning to life and the universe is\"\n```\nor \n```\n./llama-server --hf-repo cstr/aya-expanse-8b-Q4_K_M-GGUF --hf-file aya-expanse-8b-q4_k_m.gguf -c 2048\n```\n\n### ollama\n\nuse a modelfile like:\n\n```\nFROM aya-expanse-8b-q4_k_m.gguf\nTEMPLATE \"{{ if .System }}<|START_OF_TURN_TOKEN|><|SYSTEM_TOKEN|>{{ .System }}<|END_OF_TURN_TOKEN|>\n{{ end }}{{ if .Prompt }}<|START_OF_TURN_TOKEN|><|USER_TOKEN|>{{ .Prompt }}<|END_OF_TURN_TOKEN|>\n{{ end }}<|START_OF_TURN_TOKEN|><|CHATBOT_TOKEN|>\"\n\nPARAMETER num_ctx 8192\nPARAMETER stop \"<|END_OF_TURN_TOKEN|>\"\nPARAMETER stop \"<|START_OF_TURN_TOKEN|>\"\nPARAMETER stop \"|END_OF_TURN_TOKEN\"\nPARAMETER stop \"|START_OF_TURN_TOKEN\"\n\nSYSTEM \"You are Aya, a brilliant, sophisticated, multilingual AI-assistant trained to assist human users by providing thorough responses. You are able to interact and respond to questions in 23 languages and you are powered by a multilingual model built by Cohere For AI.\"\n```\n",
    "related_quantizations": []
  },
  "tags": [
    "transformers",
    "gguf",
    "llama-cpp",
    "gguf-my-repo",
    "en",
    "fr",
    "de",
    "es",
    "it",
    "pt",
    "ja",
    "ko",
    "zh",
    "ar",
    "el",
    "fa",
    "pl",
    "id",
    "cs",
    "he",
    "hi",
    "nl",
    "ro",
    "ru",
    "tr",
    "uk",
    "vi",
    "base_model:CohereLabs/aya-expanse-8b",
    "base_model:quantized:CohereLabs/aya-expanse-8b",
    "license:cc-by-nc-4.0",
    "region:us",
    "conversational"
  ],
  "likes": 0,
  "downloads": 82,
  "gated": false,
  "private": false,
  "last_modified": "2024-10-24T14:49:36.000Z",
  "created_at": "2024-10-24T14:22:33.000Z",
  "pipeline_tag": "",
  "library_name": "transformers"
}

{
  "_id": "671a58290173eeef8e19961c",
  "id": "cstr/aya-expanse-8b-Q4_K_M-GGUF",
  "modelId": "cstr/aya-expanse-8b-Q4_K_M-GGUF",
  "sha": "e58f5070d36bc9fd669e36c66a04a79f8f719d45",
  "createdAt": "2024-10-24T14:22:33.000Z",
  "lastModified": "2024-10-24T14:49:36.000Z",
  "author": "cstr",
  "downloads": 82,
  "likes": 0,
  "gated": false,
  "private": false,
  "pipeline_tag": "",
  "library_name": "transformers",
  "siblings_count": 3
}