richarderkhov/raj-maharajwala_-_open-insurance-llm-llama3-8b-gguf overview

{
  "metadata": {},
  "card_data": {
    "frontmatter": {},
    "hero_image_url": "",
    "summary": "This model is a domain-specific language model based on Nvidia Llama 3 ChatQA, fine-tuned for insurance-related queries and conversations. It leverages the architecture of Llama 3 and is specifically trained to handle insurance domain tasks.",
    "quick_links": [],
    "benchmark_table_html": "",
    "readme_markdown": "Quantization made by Richard Erkhov.\n\n[Github](https://github.com/RichardErkhov)\n\n[Discord](https://discord.gg/pvy7H8DZMG)\n\n[Request more models](https://github.com/RichardErkhov/quant_request)\n\n\nOpen-Insurance-LLM-Llama3-8B - GGUF\n- Model creator: https://huggingface.co/Raj-Maharajwala/\n- Original model: https://huggingface.co/Raj-Maharajwala/Open-Insurance-LLM-Llama3-8B/\n\n\n| Name | Quant method | Size |\n| ---- | ---- | ---- |\n| [Open-Insurance-LLM-Llama3-8B.Q2_K.gguf](https://huggingface.co/RichardErkhov/Raj-Maharajwala_-_Open-Insurance-LLM-Llama3-8B-gguf/blob/main/Open-Insurance-LLM-Llama3-8B.Q2_K.gguf) | Q2_K | 2.96GB |\n| [Open-Insurance-LLM-Llama3-8B.IQ3_XS.gguf](https://huggingface.co/RichardErkhov/Raj-Maharajwala_-_Open-Insurance-LLM-Llama3-8B-gguf/blob/main/Open-Insurance-LLM-Llama3-8B.IQ3_XS.gguf) | IQ3_XS | 3.28GB |\n| [Open-Insurance-LLM-Llama3-8B.IQ3_S.gguf](https://huggingface.co/RichardErkhov/Raj-Maharajwala_-_Open-Insurance-LLM-Llama3-8B-gguf/blob/main/Open-Insurance-LLM-Llama3-8B.IQ3_S.gguf) | IQ3_S | 3.43GB |\n| [Open-Insurance-LLM-Llama3-8B.Q3_K_S.gguf](https://huggingface.co/RichardErkhov/Raj-Maharajwala_-_Open-Insurance-LLM-Llama3-8B-gguf/blob/main/Open-Insurance-LLM-Llama3-8B.Q3_K_S.gguf) | Q3_K_S | 3.41GB |\n| [Open-Insurance-LLM-Llama3-8B.IQ3_M.gguf](https://huggingface.co/RichardErkhov/Raj-Maharajwala_-_Open-Insurance-LLM-Llama3-8B-gguf/blob/main/Open-Insurance-LLM-Llama3-8B.IQ3_M.gguf) | IQ3_M | 3.52GB |\n| [Open-Insurance-LLM-Llama3-8B.Q3_K.gguf](https://huggingface.co/RichardErkhov/Raj-Maharajwala_-_Open-Insurance-LLM-Llama3-8B-gguf/blob/main/Open-Insurance-LLM-Llama3-8B.Q3_K.gguf) | Q3_K | 3.74GB |\n| [Open-Insurance-LLM-Llama3-8B.Q3_K_M.gguf](https://huggingface.co/RichardErkhov/Raj-Maharajwala_-_Open-Insurance-LLM-Llama3-8B-gguf/blob/main/Open-Insurance-LLM-Llama3-8B.Q3_K_M.gguf) | Q3_K_M | 3.74GB |\n| [Open-Insurance-LLM-Llama3-8B.Q3_K_L.gguf](https://huggingface.co/RichardErkhov/Raj-Maharajwala_-_Open-Insurance-LLM-Llama3-8B-gguf/blob/main/Open-Insurance-LLM-Llama3-8B.Q3_K_L.gguf) | Q3_K_L | 4.03GB |\n| [Open-Insurance-LLM-Llama3-8B.IQ4_XS.gguf](https://huggingface.co/RichardErkhov/Raj-Maharajwala_-_Open-Insurance-LLM-Llama3-8B-gguf/blob/main/Open-Insurance-LLM-Llama3-8B.IQ4_XS.gguf) | IQ4_XS | 4.18GB |\n| [Open-Insurance-LLM-Llama3-8B.Q4_0.gguf](https://huggingface.co/RichardErkhov/Raj-Maharajwala_-_Open-Insurance-LLM-Llama3-8B-gguf/blob/main/Open-Insurance-LLM-Llama3-8B.Q4_0.gguf) | Q4_0 | 4.34GB |\n| [Open-Insurance-LLM-Llama3-8B.IQ4_NL.gguf](https://huggingface.co/RichardErkhov/Raj-Maharajwala_-_Open-Insurance-LLM-Llama3-8B-gguf/blob/main/Open-Insurance-LLM-Llama3-8B.IQ4_NL.gguf) | IQ4_NL | 4.38GB |\n| [Open-Insurance-LLM-Llama3-8B.Q4_K_S.gguf](https://huggingface.co/RichardErkhov/Raj-Maharajwala_-_Open-Insurance-LLM-Llama3-8B-gguf/blob/main/Open-Insurance-LLM-Llama3-8B.Q4_K_S.gguf) | Q4_K_S | 4.37GB |\n| [Open-Insurance-LLM-Llama3-8B.Q4_K.gguf](https://huggingface.co/RichardErkhov/Raj-Maharajwala_-_Open-Insurance-LLM-Llama3-8B-gguf/blob/main/Open-Insurance-LLM-Llama3-8B.Q4_K.gguf) | Q4_K | 4.58GB |\n| [Open-Insurance-LLM-Llama3-8B.Q4_K_M.gguf](https://huggingface.co/RichardErkhov/Raj-Maharajwala_-_Open-Insurance-LLM-Llama3-8B-gguf/blob/main/Open-Insurance-LLM-Llama3-8B.Q4_K_M.gguf) | Q4_K_M | 4.58GB |\n| [Open-Insurance-LLM-Llama3-8B.Q4_1.gguf](https://huggingface.co/RichardErkhov/Raj-Maharajwala_-_Open-Insurance-LLM-Llama3-8B-gguf/blob/main/Open-Insurance-LLM-Llama3-8B.Q4_1.gguf) | Q4_1 | 4.78GB |\n| [Open-Insurance-LLM-Llama3-8B.Q5_0.gguf](https://huggingface.co/RichardErkhov/Raj-Maharajwala_-_Open-Insurance-LLM-Llama3-8B-gguf/blob/main/Open-Insurance-LLM-Llama3-8B.Q5_0.gguf) | Q5_0 | 5.21GB |\n| [Open-Insurance-LLM-Llama3-8B.Q5_K_S.gguf](https://huggingface.co/RichardErkhov/Raj-Maharajwala_-_Open-Insurance-LLM-Llama3-8B-gguf/blob/main/Open-Insurance-LLM-Llama3-8B.Q5_K_S.gguf) | Q5_K_S | 5.21GB |\n| [Open-Insurance-LLM-Llama3-8B.Q5_K.gguf](https://huggingface.co/RichardErkhov/Raj-Maharajwala_-_Open-Insurance-LLM-Llama3-8B-gguf/blob/main/Open-Insurance-LLM-Llama3-8B.Q5_K.gguf) | Q5_K | 5.34GB |\n| [Open-Insurance-LLM-Llama3-8B.Q5_K_M.gguf](https://huggingface.co/RichardErkhov/Raj-Maharajwala_-_Open-Insurance-LLM-Llama3-8B-gguf/blob/main/Open-Insurance-LLM-Llama3-8B.Q5_K_M.gguf) | Q5_K_M | 5.34GB |\n| [Open-Insurance-LLM-Llama3-8B.Q5_1.gguf](https://huggingface.co/RichardErkhov/Raj-Maharajwala_-_Open-Insurance-LLM-Llama3-8B-gguf/blob/main/Open-Insurance-LLM-Llama3-8B.Q5_1.gguf) | Q5_1 | 5.65GB |\n| [Open-Insurance-LLM-Llama3-8B.Q6_K.gguf](https://huggingface.co/RichardErkhov/Raj-Maharajwala_-_Open-Insurance-LLM-Llama3-8B-gguf/blob/main/Open-Insurance-LLM-Llama3-8B.Q6_K.gguf) | Q6_K | 6.14GB |\n| [Open-Insurance-LLM-Llama3-8B.Q8_0.gguf](https://huggingface.co/RichardErkhov/Raj-Maharajwala_-_Open-Insurance-LLM-Llama3-8B-gguf/blob/main/Open-Insurance-LLM-Llama3-8B.Q8_0.gguf) | Q8_0 | 7.95GB |\n\n\n\n\nOriginal model description:\n---\nlanguage:\n- en\nlibrary_name: transformers\npipeline_tag: text-generation\ntags:\n- Text Generation\n- Transformers\n- llama\n- llama-3\n- 8B\n- nvidia\n- facebook\n- meta\n- LLM\n- insurance\n- research\n- pytorch\n- instruct\n- chatqa-1.5\n- chatqa\n- finetune\n- gpt4\n- conversational\n- text-generation-inference\ndatasets:\n- InsuranceQA\n\nbase_model: \"nvidia/Llama3-ChatQA-1.5-8B\"\nfinetuned: \"Raj-Maharajwala/Open-Insurance-LLM-Llama3-8B\"\nquantized: \"Raj-Maharajwala/Open-Insurance-LLM-Llama3-8B-GGUF\"\nlicense: llama3\n---\n\n# Open-Insurance-LLM-Llama3-8B\n\nThis model is a domain-specific language model based on Nvidia Llama 3 ChatQA, fine-tuned for insurance-related queries and conversations. It leverages the architecture of Llama 3 and is specifically trained to handle insurance domain tasks.\n\n## Model Details\n\n- **Model Type:** Instruction-tuned Language Model\n- **Base Model:** nvidia/Llama3-ChatQA-1.5-8B\n- **Finetuned Model:** Raj-Maharajwala/Open-Insurance-LLM-Llama3-8B\n- **Quantized Model:** Raj-Maharajwala/Open-Insurance-LLM-Llama3-8B-GGUF\n- **Model Architecture:** Llama\n- **Parameters:** 8.05 billion\n- **Developer:** Raj Maharajwala\n- **License:** llama3\n- **Language:** English\n\n### Quantized Model \n\nRaj-Maharajwala/Open-Insurance-LLM-Llama3-8B-GGUF: https://huggingface.co/Raj-Maharajwala/Open-Insurance-LLM-Llama3-8B-GGUF\n\n## Training Data\n\nThe model has been fine-tuned on the InsuranceQA dataset using LoRA (8 bit), which contains insurance-specific question-answer pairs and domain knowledge.\ntrainable params: 20.97M || all params: 8.05B || trainable %: 0.26%\n```bash\nLoraConfig(\n  r=8,\n  lora_alpha=32,\n  lora_dropout=0.05,\n  bias=\"none\",\n  task_type=\"CAUSAL_LM\",\n  target_modules=['up_proj', 'down_proj', 'gate_proj', 'k_proj', 'q_proj', 'v_proj', 'o_proj']\n)\n```\n\n## Model Architecture\n\nThe model uses the Llama 3 architecture with the following key components:\n- 8B parameter configuration\n- Enhanced attention mechanisms from Llama 3\n- ChatQA 1.5 instruction-tuning framework\n- Insurance domain-specific adaptations\n\n## Files in Repository\n\n- **Model Files:**\n  - `model-00001-of-00004.safetensors` (4.98 GB)\n  - `model-00002-of-00004.safetensors` (5 GB)\n  - `model-00003-of-00004.safetensors` (4.92 GB)\n  - `model-00004-of-00004.safetensors` (1.17 GB)\n  - `model.safetensors.index.json` (24 kB)\n\n- **Tokenizer Files:**\n  - `tokenizer.json` (17.2 MB)\n  - `tokenizer_config.json` (51.3 kB)\n  - `special_tokens_map.json` (335 Bytes)\n\n- **Configuration Files:**\n  - `config.json` (738 Bytes)\n  - `generation_config.json` (143 Bytes)\n\n## Use Cases\n\nThis model is specifically designed for:\n- Insurance policy understanding and explanation\n- Claims processing assistance\n- Coverage analysis\n- Insurance terminology clarification\n- Policy comparison and recommendations\n- Risk assessment queries\n- Insurance compliance questions\n\n## Limitations\n\n- The model's knowledge is limited to its training data cutoff\n- Should not be used as a replacement for professional insurance advice\n- May occasionally generate plausible-sounding but incorrect information\n\n## Bias and Ethics\n\nThis model should be used with awareness that:\n- It may reflect biases present in insurance industry training data\n- Output should be verified by insurance professionals for critical decisions\n- It should not be used as the sole basis for insurance decisions\n- The model's responses should be treated as informational, not as legal or professional advice\n\n## Citation and Attribution\n\nIf you use this model in your research or applications, please cite:\n```\n@misc{maharajwala2024openinsurance,\n  author = {Raj Maharajwala},\n  title = {Open-Insurance-LLM-Llama3-8B},\n  year = {2024},\n  publisher = {HuggingFace},\n  url = {https://huggingface.co/Raj-Maharajwala/Open-Insurance-LLM-Llama3-8B}\n}\n```\n\n",
    "related_quantizations": []
  },
  "tags": [
    "gguf",
    "endpoints_compatible",
    "region:us",
    "conversational"
  ],
  "likes": 0,
  "downloads": 88,
  "gated": false,
  "private": false,
  "last_modified": "2025-05-19T20:01:59.000Z",
  "created_at": "2025-05-19T11:46:45.000Z",
  "pipeline_tag": "",
  "library_name": ""
}

{
  "_id": "682b1a25f879efa43ed66fd2",
  "id": "RichardErkhov/Raj-Maharajwala_-_Open-Insurance-LLM-Llama3-8B-gguf",
  "modelId": "RichardErkhov/Raj-Maharajwala_-_Open-Insurance-LLM-Llama3-8B-gguf",
  "sha": "4617d60bc34656c4dff4409a826aaed07f990b47",
  "createdAt": "2025-05-19T11:46:45.000Z",
  "lastModified": "2025-05-19T20:01:59.000Z",
  "author": "RichardErkhov",
  "downloads": 88,
  "likes": 0,
  "gated": false,
  "private": false,
  "pipeline_tag": "",
  "library_name": "",
  "siblings_count": 24
}