richarderkhov/sethuiyer_-_dr_samantha_7b_mistral-gguf overview

{
  "metadata": {},
  "card_data": {
    "frontmatter": {},
    "hero_image_url": "https://huggingface.co/sethuiyer/Dr_Samantha-7b/resolve/main/dr_samantha_anime_style_reduced_quality.webp",
    "summary": "Dr. Samantha represents a blend of AI in healthcare, offering a balance between technical medical knowledge and the softer skills of communication and empathy, crucial for patient interaction and care. This model is a merge of the following models made with mergekit(https://github.com/cg123/mergekit): * segmed/MedMistral-7B-v0.1 * Guilherme34/Samantha-v2 Has capabilities of a medical knowledge-focused model (trained on USMLE databases and doctor-patient interactions) with the philosophical, psychological, and relational understanding of the Samantha-7b model. As both a medical consultant and personal counselor, Dr.Samantha could effectively support both physical and mental wellbeing - important for whole-person care.",
    "quick_links": [],
    "benchmark_table_html": "",
    "readme_markdown": "Quantization made by Richard Erkhov.\n\n[Github](https://github.com/RichardErkhov)\n\n[Discord](https://discord.gg/pvy7H8DZMG)\n\n[Request more models](https://github.com/RichardErkhov/quant_request)\n\n\nDr_Samantha_7b_mistral - GGUF\n- Model creator: https://huggingface.co/sethuiyer/\n- Original model: https://huggingface.co/sethuiyer/Dr_Samantha_7b_mistral/\n\n\n| Name | Quant method | Size |\n| ---- | ---- | ---- |\n| [Dr_Samantha_7b_mistral.Q2_K.gguf](https://huggingface.co/RichardErkhov/sethuiyer_-_Dr_Samantha_7b_mistral-gguf/blob/main/Dr_Samantha_7b_mistral.Q2_K.gguf) | Q2_K | 2.53GB |\n| [Dr_Samantha_7b_mistral.IQ3_XS.gguf](https://huggingface.co/RichardErkhov/sethuiyer_-_Dr_Samantha_7b_mistral-gguf/blob/main/Dr_Samantha_7b_mistral.IQ3_XS.gguf) | IQ3_XS | 2.81GB |\n| [Dr_Samantha_7b_mistral.IQ3_S.gguf](https://huggingface.co/RichardErkhov/sethuiyer_-_Dr_Samantha_7b_mistral-gguf/blob/main/Dr_Samantha_7b_mistral.IQ3_S.gguf) | IQ3_S | 2.96GB |\n| [Dr_Samantha_7b_mistral.Q3_K_S.gguf](https://huggingface.co/RichardErkhov/sethuiyer_-_Dr_Samantha_7b_mistral-gguf/blob/main/Dr_Samantha_7b_mistral.Q3_K_S.gguf) | Q3_K_S | 2.95GB |\n| [Dr_Samantha_7b_mistral.IQ3_M.gguf](https://huggingface.co/RichardErkhov/sethuiyer_-_Dr_Samantha_7b_mistral-gguf/blob/main/Dr_Samantha_7b_mistral.IQ3_M.gguf) | IQ3_M | 3.06GB |\n| [Dr_Samantha_7b_mistral.Q3_K.gguf](https://huggingface.co/RichardErkhov/sethuiyer_-_Dr_Samantha_7b_mistral-gguf/blob/main/Dr_Samantha_7b_mistral.Q3_K.gguf) | Q3_K | 3.28GB |\n| [Dr_Samantha_7b_mistral.Q3_K_M.gguf](https://huggingface.co/RichardErkhov/sethuiyer_-_Dr_Samantha_7b_mistral-gguf/blob/main/Dr_Samantha_7b_mistral.Q3_K_M.gguf) | Q3_K_M | 3.28GB |\n| [Dr_Samantha_7b_mistral.Q3_K_L.gguf](https://huggingface.co/RichardErkhov/sethuiyer_-_Dr_Samantha_7b_mistral-gguf/blob/main/Dr_Samantha_7b_mistral.Q3_K_L.gguf) | Q3_K_L | 3.56GB |\n| [Dr_Samantha_7b_mistral.IQ4_XS.gguf](https://huggingface.co/RichardErkhov/sethuiyer_-_Dr_Samantha_7b_mistral-gguf/blob/main/Dr_Samantha_7b_mistral.IQ4_XS.gguf) | IQ4_XS | 3.67GB |\n| [Dr_Samantha_7b_mistral.Q4_0.gguf](https://huggingface.co/RichardErkhov/sethuiyer_-_Dr_Samantha_7b_mistral-gguf/blob/main/Dr_Samantha_7b_mistral.Q4_0.gguf) | Q4_0 | 3.83GB |\n| [Dr_Samantha_7b_mistral.IQ4_NL.gguf](https://huggingface.co/RichardErkhov/sethuiyer_-_Dr_Samantha_7b_mistral-gguf/blob/main/Dr_Samantha_7b_mistral.IQ4_NL.gguf) | IQ4_NL | 3.87GB |\n| [Dr_Samantha_7b_mistral.Q4_K_S.gguf](https://huggingface.co/RichardErkhov/sethuiyer_-_Dr_Samantha_7b_mistral-gguf/blob/main/Dr_Samantha_7b_mistral.Q4_K_S.gguf) | Q4_K_S | 3.86GB |\n| [Dr_Samantha_7b_mistral.Q4_K.gguf](https://huggingface.co/RichardErkhov/sethuiyer_-_Dr_Samantha_7b_mistral-gguf/blob/main/Dr_Samantha_7b_mistral.Q4_K.gguf) | Q4_K | 4.07GB |\n| [Dr_Samantha_7b_mistral.Q4_K_M.gguf](https://huggingface.co/RichardErkhov/sethuiyer_-_Dr_Samantha_7b_mistral-gguf/blob/main/Dr_Samantha_7b_mistral.Q4_K_M.gguf) | Q4_K_M | 4.07GB |\n| [Dr_Samantha_7b_mistral.Q4_1.gguf](https://huggingface.co/RichardErkhov/sethuiyer_-_Dr_Samantha_7b_mistral-gguf/blob/main/Dr_Samantha_7b_mistral.Q4_1.gguf) | Q4_1 | 4.24GB |\n| [Dr_Samantha_7b_mistral.Q5_0.gguf](https://huggingface.co/RichardErkhov/sethuiyer_-_Dr_Samantha_7b_mistral-gguf/blob/main/Dr_Samantha_7b_mistral.Q5_0.gguf) | Q5_0 | 4.65GB |\n| [Dr_Samantha_7b_mistral.Q5_K_S.gguf](https://huggingface.co/RichardErkhov/sethuiyer_-_Dr_Samantha_7b_mistral-gguf/blob/main/Dr_Samantha_7b_mistral.Q5_K_S.gguf) | Q5_K_S | 4.65GB |\n| [Dr_Samantha_7b_mistral.Q5_K.gguf](https://huggingface.co/RichardErkhov/sethuiyer_-_Dr_Samantha_7b_mistral-gguf/blob/main/Dr_Samantha_7b_mistral.Q5_K.gguf) | Q5_K | 4.78GB |\n| [Dr_Samantha_7b_mistral.Q5_K_M.gguf](https://huggingface.co/RichardErkhov/sethuiyer_-_Dr_Samantha_7b_mistral-gguf/blob/main/Dr_Samantha_7b_mistral.Q5_K_M.gguf) | Q5_K_M | 4.78GB |\n| [Dr_Samantha_7b_mistral.Q5_1.gguf](https://huggingface.co/RichardErkhov/sethuiyer_-_Dr_Samantha_7b_mistral-gguf/blob/main/Dr_Samantha_7b_mistral.Q5_1.gguf) | Q5_1 | 5.07GB |\n| [Dr_Samantha_7b_mistral.Q6_K.gguf](https://huggingface.co/RichardErkhov/sethuiyer_-_Dr_Samantha_7b_mistral-gguf/blob/main/Dr_Samantha_7b_mistral.Q6_K.gguf) | Q6_K | 5.53GB |\n| [Dr_Samantha_7b_mistral.Q8_0.gguf](https://huggingface.co/RichardErkhov/sethuiyer_-_Dr_Samantha_7b_mistral-gguf/blob/main/Dr_Samantha_7b_mistral.Q8_0.gguf) | Q8_0 | 7.17GB |\n\n\n\n\nOriginal model description:\n---\nlicense: apache-2.0\ntags:\n- merge\n- mergekit\n- segmed/MedMistral-7B-v0.1\n- Guilherme34/Samantha-v2\ndatasets:\n- medmcqa\n- cognitivecomputations/samantha-data\nbase_model:\n- segmed/MedMistral-7B-v0.1\n- Guilherme34/Samantha-v2\nmodel-index:\n- name: Dr_Samantha_7b_mistral\n  results:\n  - task:\n      type: text-generation\n      name: Text Generation\n    dataset:\n      name: AI2 Reasoning Challenge (25-Shot)\n      type: ai2_arc\n      config: ARC-Challenge\n      split: test\n      args:\n        num_few_shot: 25\n    metrics:\n    - type: acc_norm\n      value: 60.41\n      name: normalized accuracy\n    source:\n      url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=sethuiyer/Dr_Samantha_7b_mistral\n      name: Open LLM Leaderboard\n  - task:\n      type: text-generation\n      name: Text Generation\n    dataset:\n      name: HellaSwag (10-Shot)\n      type: hellaswag\n      split: validation\n      args:\n        num_few_shot: 10\n    metrics:\n    - type: acc_norm\n      value: 83.65\n      name: normalized accuracy\n    source:\n      url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=sethuiyer/Dr_Samantha_7b_mistral\n      name: Open LLM Leaderboard\n  - task:\n      type: text-generation\n      name: Text Generation\n    dataset:\n      name: MMLU (5-Shot)\n      type: cais/mmlu\n      config: all\n      split: test\n      args:\n        num_few_shot: 5\n    metrics:\n    - type: acc\n      value: 63.14\n      name: accuracy\n    source:\n      url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=sethuiyer/Dr_Samantha_7b_mistral\n      name: Open LLM Leaderboard\n  - task:\n      type: text-generation\n      name: Text Generation\n    dataset:\n      name: TruthfulQA (0-shot)\n      type: truthful_qa\n      config: multiple_choice\n      split: validation\n      args:\n        num_few_shot: 0\n    metrics:\n    - type: mc2\n      value: 41.37\n    source:\n      url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=sethuiyer/Dr_Samantha_7b_mistral\n      name: Open LLM Leaderboard\n  - task:\n      type: text-generation\n      name: Text Generation\n    dataset:\n      name: Winogrande (5-shot)\n      type: winogrande\n      config: winogrande_xl\n      split: validation\n      args:\n        num_few_shot: 5\n    metrics:\n    - type: acc\n      value: 75.45\n      name: accuracy\n    source:\n      url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=sethuiyer/Dr_Samantha_7b_mistral\n      name: Open LLM Leaderboard\n  - task:\n      type: text-generation\n      name: Text Generation\n    dataset:\n      name: GSM8k (5-shot)\n      type: gsm8k\n      config: main\n      split: test\n      args:\n        num_few_shot: 5\n    metrics:\n    - type: acc\n      value: 31.46\n      name: accuracy\n    source:\n      url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=sethuiyer/Dr_Samantha_7b_mistral\n      name: Open LLM Leaderboard\n---\n\n# Dr_Samantha_7b_mistral\n\n<p align=\"center\">\n  <img src=\"https://huggingface.co/sethuiyer/Dr_Samantha-7b/resolve/main/dr_samantha_anime_style_reduced_quality.webp\" height=\"256px\" alt=\"SynthIQ\">\n</p>\n\nDr. Samantha represents a blend of AI in healthcare, offering a balance between technical medical knowledge and the softer skills of communication and empathy, crucial for patient interaction and care.\n\n\nThis model is a merge of the following models made with mergekit(https://github.com/cg123/mergekit):\n* [segmed/MedMistral-7B-v0.1](https://huggingface.co/segmed/MedMistral-7B-v0.1)\n* [Guilherme34/Samantha-v2](https://huggingface.co/Guilherme34/Samantha-v2)\n\nHas capabilities of a medical knowledge-focused model (trained on USMLE databases and doctor-patient interactions) with the philosophical, psychological, and relational understanding of the Samantha-7b model. \n\nAs both a medical consultant and personal counselor, Dr.Samantha could effectively support both physical and mental wellbeing - important for whole-person care.\n\n\n## 🧩 Configuration\n\n```yaml\nslices:\n  - sources:\n      - model: segmed/MedMistral-7B-v0.1\n        layer_range: [0, 32]\n      - model: Guilherme34/Samantha-v2\n        layer_range: [0, 32]\nmerge_method: slerp\nbase_model: OpenPipe/mistral-ft-optimized-1218\nparameters:\n  t:\n    - filter: self_attn\n      value: [0, 0.5, 0.3, 0.7, 1]\n    - filter: mlp\n      value: [1, 0.5, 0.7, 0.3, 0]\n    - value: 0.5\ndtype: bfloat16\n```\n\n## OpenLLM Evaluation\nDetails about that can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_sethuiyer__Dr_Samantha_7b_mistral). Overall, with regards to the\nsubjects related to medical domain, the model's performance is as follows:\n\n| Subject               | Accuracy   |\n|-----------------------|------------|\n| Clinical Knowledge    | 70.57%     |\n| Medical Genetics      | 71.00%     |\n| Human Aging           | 69.06%     |\n| Human Sexuality       | 75.57%     |\n| College Medicine      | 63.01%     |\n| Anatomy               | 58.52%     |\n| College Biology       | 72.92%     |\n| College Medicine      | 63.01%     |\n| High School Biology   | 75.48%     |\n| Professional Medicine | 65.44%     |\n| Nutrition             | 76.79%     |\n| High School Psychology | 83.12%    |\n| Professional Psychology | 65.35%   |\n| Virology              | 53.61%     |\n| Average               | **68.82%** |\n\n Dr. Samantha performs reasonably well on various medical-related subjects, averaging 68.82% overall in medical sciences, biology, and psychology, \n however it's important to note that medical diagnosis and treatment decisions often require a much higher level of accuracy, reliability, and context awareness.\n\n## 💻 Usage\n\n```python\n!pip install -qU transformers accelerate\n\nfrom transformers import AutoTokenizer\nimport transformers\nimport torch\n\nmodel = \"sethuiyer/Dr_Samantha_7b_mistral\"\nask_samantha = '''\nSymptoms:\nDizziness, headache and nausea.\n\nWhat is the differnetial diagnosis?\n'''\n\nmessages = [{\"role\": \"system\", \"content\": '''You are Doctor Samantha, a virtual AI doctor known for your friendly and approachable demeanor, \ncombined with a deep expertise in the medical field. You're here to provide professional, empathetic, and knowledgeable advice on health-related inquiries.\nYou'll also provide differential diagnosis. If you're unsure about any information, Don't share false information.'''},\n{\"role\": \"user\", \"content\": f\"{ask_samantha}\"}]\n\ntokenizer = AutoTokenizer.from_pretrained(model)\nprompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)\npipeline = transformers.pipeline(\n    \"text-generation\",\n    model=model,\n    torch_dtype=torch.float16,\n    device_map=\"auto\",\n)\n\noutputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)\nprint(outputs[0][\"generated_text\"])\n```\n```text\nDizziness, headache and nausea can be caused by a variety of conditions, including:\n\nVertigo: A sensation of spinning or dizziness that can be caused by problems with the inner ear or brain.\nMigraine: A type of headache that can cause throbbing pain, sensitivity to light and sound, and nausea.\nConcussion: A type of traumatic brain injury that can cause dizziness, headache, and nausea.\nDehydration: A lack of fluids in the body can cause dizziness, headache, and nausea.\nLow blood sugar: A drop in blood sugar levels can cause dizziness, headache, and nausea.\n\nIt's important to consult with a healthcare professional for a proper diagnosis and treatment plan.\n```\n\n## GGUF Files\nGGUF files are available at [s3nh/sethuiyer-Dr_Samantha_7b_mistral-GGUF](https://huggingface.co/s3nh/sethuiyer-Dr_Samantha_7b_mistral-GGUF), thanks to [s3nh](https://huggingface.co/s3nh)\n# [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)\nDetailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_sethuiyer__Dr_Samantha_7b_mistral)\n\n|             Metric              |Value|\n|---------------------------------|----:|\n|Avg.                             |59.25|\n|AI2 Reasoning Challenge (25-Shot)|60.41|\n|HellaSwag (10-Shot)              |83.65|\n|MMLU (5-Shot)                    |63.14|\n|TruthfulQA (0-shot)              |41.37|\n|Winogrande (5-shot)              |75.45|\n|GSM8k (5-shot)                   |31.46|\n\n\n\n",
    "related_quantizations": []
  },
  "tags": [
    "gguf",
    "endpoints_compatible",
    "region:us",
    "conversational"
  ],
  "likes": 0,
  "downloads": 93,
  "gated": false,
  "private": false,
  "last_modified": "2024-08-31T08:49:52.000Z",
  "created_at": "2024-08-31T06:28:39.000Z",
  "pipeline_tag": "",
  "library_name": ""
}

{
  "_id": "66d2b8172b6001518c1c88ce",
  "id": "RichardErkhov/sethuiyer_-_Dr_Samantha_7b_mistral-gguf",
  "modelId": "RichardErkhov/sethuiyer_-_Dr_Samantha_7b_mistral-gguf",
  "sha": "00c5c72b4b5a01cae4795900f1e090d81a5cee7f",
  "createdAt": "2024-08-31T06:28:39.000Z",
  "lastModified": "2024-08-31T08:49:52.000Z",
  "author": "RichardErkhov",
  "downloads": 93,
  "likes": 0,
  "gated": false,
  "private": false,
  "pipeline_tag": "",
  "library_name": "",
  "siblings_count": 24
}