lmstudio-community/llama3-chatqa-1.5-8b-gguf overview

{
  "metadata": {},
  "card_data": {
    "license": "llama3",
    "language": [
      "en"
    ],
    "pipeline_tag": "text-generation",
    "tags": [
      "nvidia",
      "chatqa-1.5",
      "chatqa",
      "llama-3",
      "pytorch"
    ],
    "base_model": "nvidia/Llama3-ChatQA-1.5-8B",
    "quantized_by": "bartowski",
    "lm_studio": {
      "param_count": "8b",
      "use_case": "RAG",
      "release_date": "28-04-2024",
      "model_creator": "nvidia",
      "prompt_template": "chatQA",
      "system_prompt": "This is a chat between a user and an artificial intelligence assistant. The assistant gives helpful, detailed, and polite answers to the user's questions based on the context. The assistant should also indicate when the answer cannot be found in the context.",
      "base_model": "llama",
      "original_repo": "nvidia/Llama3-ChatQA-1.5-8B"
    },
    "frontmatter": {
      "license": "llama3",
      "language": [
        "en"
      ],
      "pipeline_tag": "text-generation",
      "tags": [
        "nvidia",
        "chatqa-1.5",
        "chatqa",
        "llama-3",
        "pytorch"
      ],
      "base_model": "nvidia/Llama3-ChatQA-1.5-8B",
      "quantized_by": "bartowski",
      "lm_studio": []
    },
    "hero_image_url": "",
    "summary": "",
    "quick_links": [],
    "benchmark_table_html": "",
    "readme_markdown": "---\nlicense: llama3\nlanguage:\n- en\npipeline_tag: text-generation\ntags:\n- nvidia\n- chatqa-1.5\n- chatqa\n- llama-3\n- pytorch\nbase_model: nvidia/Llama3-ChatQA-1.5-8B\nquantized_by: bartowski\nlm_studio:\n  param_count: 8b\n  use_case: RAG\n  release_date: 28-04-2024\n  model_creator: nvidia\n  prompt_template: chatQA\n  system_prompt: \"This is a chat between a user and an artificial intelligence assistant. The assistant gives helpful, detailed, and polite answers to the user's questions based on the context. The assistant should also indicate when the answer cannot be found in the context.\"\n  base_model: llama\n  original_repo: nvidia/Llama3-ChatQA-1.5-8B\n---\n\n## 💫 Community Model> Llama 3 ChatQA 1.5 8B by NVIDIA\n\n*👾 [LM Studio](https://lmstudio.ai) Community models highlights program. Highlighting new & noteworthy models by the community. Join the conversation on [Discord](https://discord.gg/aPQfnNkxGC)*.\n\n**Model creator:** [nvidia](https://huggingface.co/nvidia)<br>\n**Original model**: [Llama3-ChatQA-1.5-8B](https://huggingface.co/nvidia/Llama3-ChatQA-1.5-8B)<br>\n**GGUF quantization:** provided by [bartowski](https://huggingface.co/bartowski) based on `llama.cpp` release [b2777](https://github.com/ggerganov/llama.cpp/releases/tag/b2777)<br>\n\n## Model Summary:\nChatQA 1.5 is a series of models trained to excel at RAG (retrieval augmented generation) tasks.<br>\nThis model may work for general uses, but it primarily meant for use as a context sumarizer or context extraction.<br>\nUsing the context provided after the system message, the model is able to provide contextual and accurate answers to queries.<br>\n\n## Prompt Template:\n\nFor now, you'll need to make your own template. Choose the `LM Studio Blank Preset` in your LM Studio.\n\nThen, set the system prompt to whatever you'd like (check the recommended one below), and set the following values:<br>\n`System Message Prefix`: 'System: '<br>\n`User Message Prefix`: '\\n\\nUser: '<br>\n`User Message Suffix`: '\\n\\nAssistant: <|begin_of_text|>'<br>\n\nIf you want to provide context, place that in the system message suffix like so:\n\n`System Message Suffix`: '\\n\\n{context}'<br>\n\nUnder the hood, the model will see a prompt that's formatted like:\n\n```\nSystem: This is a chat between a user and an artificial intelligence assistant. The assistant gives helpful, detailed, and polite answers to the user's questions based on the context. The assistant should also indicate when the answer cannot be found in the context.\n\nThis is some context\n\nUser: {Question}\n\nAssistant: \n```\n\nnVidia also seems to recommend starting your query with \"Please give a full and complete answer for the question.\"\n\n## Technical Details\n\nLlama3-ChatQA-1.5 excels at conversational question answering (QA) and retrieval-augmented generation (RAG). Llama3-ChatQA-1.5 is developed using an improved training recipe from ChatQA (1.0), and it is built on top of Llama-3 base model.<br>\nSpecifically, more conversational QA data was used to enhance its tabular and arithmetic calculation capability.\n\n## Special thanks\n\n🙏 Special thanks to [Georgi Gerganov](https://github.com/ggerganov) and the whole team working on [llama.cpp](https://github.com/ggerganov/llama.cpp/) for making all of this possible.\n\n🙏 Special thanks to [Kalomaze](https://github.com/kalomaze) for his dataset (linked [here](https://github.com/ggerganov/llama.cpp/discussions/5263)) that was used for calculating the imatrix for the IQ1_M and IQ2_XS quants, which makes them usable even at their tiny size!\n\n## Disclaimers\n\nLM Studio is not the creator, originator, or owner of any Model featured in the Community Model Program. Each Community Model is created and provided by third parties. LM Studio does not endorse, support, represent or guarantee the completeness, truthfulness, accuracy, or reliability of any Community Model.  You understand that Community Models can produce content that might be offensive, harmful, inaccurate or otherwise inappropriate, or deceptive. Each Community Model is the sole responsibility of the person or entity who originated such Model. LM Studio may not monitor or control the Community Models and cannot, and does not, take responsibility for any such Model. LM Studio disclaims all warranties or guarantees about the accuracy, reliability or benefits of the Community Models.  LM Studio further disclaims any warranty that the Community Model will meet your requirements, be secure, uninterrupted or available at any time or location, or error-free, viruses-free, or that any errors will be corrected, or otherwise. You will be solely responsible for any damage resulting from your use of or access to the Community Models, your downloading of any Community Model, or use of any other Community Model provided by or through LM Studio.",
    "related_quantizations": []
  },
  "tags": [
    "gguf",
    "nvidia",
    "chatqa-1.5",
    "chatqa",
    "llama-3",
    "pytorch",
    "text-generation",
    "en",
    "base_model:nvidia/Llama3-ChatQA-1.5-8B",
    "base_model:quantized:nvidia/Llama3-ChatQA-1.5-8B",
    "license:llama3",
    "endpoints_compatible",
    "region:us",
    "imatrix",
    "conversational"
  ],
  "likes": 6,
  "downloads": 616,
  "gated": false,
  "private": false,
  "last_modified": "2024-05-04T22:58:34.000Z",
  "created_at": "2024-05-04T22:55:31.000Z",
  "pipeline_tag": "text-generation",
  "library_name": ""
}

{
  "_id": "6636bce34b8613df5ccba096",
  "id": "lmstudio-community/Llama3-ChatQA-1.5-8B-GGUF",
  "modelId": "lmstudio-community/Llama3-ChatQA-1.5-8B-GGUF",
  "sha": "141c500a9f659d2123651a5d409ef99d2034e4a8",
  "createdAt": "2024-05-04T22:55:31.000Z",
  "lastModified": "2024-05-04T22:58:34.000Z",
  "author": "lmstudio-community",
  "downloads": 616,
  "likes": 6,
  "gated": false,
  "private": false,
  "pipeline_tag": "text-generation",
  "library_name": "",
  "siblings_count": 8
}