richarderkhov/maziyarpanahi_-_thetop-5x7b-instruct-s3-v0.1-gguf overview

{
  "metadata": {},
  "card_data": {
    "frontmatter": {},
    "hero_image_url": "",
    "summary": "Detailed results can be found here |             Metric              |Value| |---------------------------------|----:| |Avg.                             |74.03| |AI2 Reasoning Challenge (25-Shot)|70.90| |HellaSwag (10-Shot)              |88.00| |MMLU (5-Shot)                    |65.13| |TruthfulQA (0-shot)              |64.47| |Winogrande (5-shot)              |83.66| |GSM8k (5-shot)                   |72.02|",
    "quick_links": [],
    "benchmark_table_html": "",
    "readme_markdown": "Quantization made by Richard Erkhov.\n\n[Github](https://github.com/RichardErkhov)\n\n[Discord](https://discord.gg/pvy7H8DZMG)\n\n[Request more models](https://github.com/RichardErkhov/quant_request)\n\n\nTheTop-5x7B-Instruct-S3-v0.1 - GGUF\n- Model creator: https://huggingface.co/MaziyarPanahi/\n- Original model: https://huggingface.co/MaziyarPanahi/TheTop-5x7B-Instruct-S3-v0.1/\n\n\n| Name | Quant method | Size |\n| ---- | ---- | ---- |\n| [TheTop-5x7B-Instruct-S3-v0.1.Q2_K.gguf](https://huggingface.co/RichardErkhov/MaziyarPanahi_-_TheTop-5x7B-Instruct-S3-v0.1-gguf/blob/main/TheTop-5x7B-Instruct-S3-v0.1.Q2_K.gguf) | Q2_K | 2.53GB |\n| [TheTop-5x7B-Instruct-S3-v0.1.IQ3_XS.gguf](https://huggingface.co/RichardErkhov/MaziyarPanahi_-_TheTop-5x7B-Instruct-S3-v0.1-gguf/blob/main/TheTop-5x7B-Instruct-S3-v0.1.IQ3_XS.gguf) | IQ3_XS | 2.81GB |\n| [TheTop-5x7B-Instruct-S3-v0.1.IQ3_S.gguf](https://huggingface.co/RichardErkhov/MaziyarPanahi_-_TheTop-5x7B-Instruct-S3-v0.1-gguf/blob/main/TheTop-5x7B-Instruct-S3-v0.1.IQ3_S.gguf) | IQ3_S | 2.96GB |\n| [TheTop-5x7B-Instruct-S3-v0.1.Q3_K_S.gguf](https://huggingface.co/RichardErkhov/MaziyarPanahi_-_TheTop-5x7B-Instruct-S3-v0.1-gguf/blob/main/TheTop-5x7B-Instruct-S3-v0.1.Q3_K_S.gguf) | Q3_K_S | 2.95GB |\n| [TheTop-5x7B-Instruct-S3-v0.1.IQ3_M.gguf](https://huggingface.co/RichardErkhov/MaziyarPanahi_-_TheTop-5x7B-Instruct-S3-v0.1-gguf/blob/main/TheTop-5x7B-Instruct-S3-v0.1.IQ3_M.gguf) | IQ3_M | 3.06GB |\n| [TheTop-5x7B-Instruct-S3-v0.1.Q3_K.gguf](https://huggingface.co/RichardErkhov/MaziyarPanahi_-_TheTop-5x7B-Instruct-S3-v0.1-gguf/blob/main/TheTop-5x7B-Instruct-S3-v0.1.Q3_K.gguf) | Q3_K | 3.28GB |\n| [TheTop-5x7B-Instruct-S3-v0.1.Q3_K_M.gguf](https://huggingface.co/RichardErkhov/MaziyarPanahi_-_TheTop-5x7B-Instruct-S3-v0.1-gguf/blob/main/TheTop-5x7B-Instruct-S3-v0.1.Q3_K_M.gguf) | Q3_K_M | 3.28GB |\n| [TheTop-5x7B-Instruct-S3-v0.1.Q3_K_L.gguf](https://huggingface.co/RichardErkhov/MaziyarPanahi_-_TheTop-5x7B-Instruct-S3-v0.1-gguf/blob/main/TheTop-5x7B-Instruct-S3-v0.1.Q3_K_L.gguf) | Q3_K_L | 3.56GB |\n| [TheTop-5x7B-Instruct-S3-v0.1.IQ4_XS.gguf](https://huggingface.co/RichardErkhov/MaziyarPanahi_-_TheTop-5x7B-Instruct-S3-v0.1-gguf/blob/main/TheTop-5x7B-Instruct-S3-v0.1.IQ4_XS.gguf) | IQ4_XS | 3.67GB |\n| [TheTop-5x7B-Instruct-S3-v0.1.Q4_0.gguf](https://huggingface.co/RichardErkhov/MaziyarPanahi_-_TheTop-5x7B-Instruct-S3-v0.1-gguf/blob/main/TheTop-5x7B-Instruct-S3-v0.1.Q4_0.gguf) | Q4_0 | 3.83GB |\n| [TheTop-5x7B-Instruct-S3-v0.1.IQ4_NL.gguf](https://huggingface.co/RichardErkhov/MaziyarPanahi_-_TheTop-5x7B-Instruct-S3-v0.1-gguf/blob/main/TheTop-5x7B-Instruct-S3-v0.1.IQ4_NL.gguf) | IQ4_NL | 3.87GB |\n| [TheTop-5x7B-Instruct-S3-v0.1.Q4_K_S.gguf](https://huggingface.co/RichardErkhov/MaziyarPanahi_-_TheTop-5x7B-Instruct-S3-v0.1-gguf/blob/main/TheTop-5x7B-Instruct-S3-v0.1.Q4_K_S.gguf) | Q4_K_S | 3.86GB |\n| [TheTop-5x7B-Instruct-S3-v0.1.Q4_K.gguf](https://huggingface.co/RichardErkhov/MaziyarPanahi_-_TheTop-5x7B-Instruct-S3-v0.1-gguf/blob/main/TheTop-5x7B-Instruct-S3-v0.1.Q4_K.gguf) | Q4_K | 4.07GB |\n| [TheTop-5x7B-Instruct-S3-v0.1.Q4_K_M.gguf](https://huggingface.co/RichardErkhov/MaziyarPanahi_-_TheTop-5x7B-Instruct-S3-v0.1-gguf/blob/main/TheTop-5x7B-Instruct-S3-v0.1.Q4_K_M.gguf) | Q4_K_M | 4.07GB |\n| [TheTop-5x7B-Instruct-S3-v0.1.Q4_1.gguf](https://huggingface.co/RichardErkhov/MaziyarPanahi_-_TheTop-5x7B-Instruct-S3-v0.1-gguf/blob/main/TheTop-5x7B-Instruct-S3-v0.1.Q4_1.gguf) | Q4_1 | 4.24GB |\n| [TheTop-5x7B-Instruct-S3-v0.1.Q5_0.gguf](https://huggingface.co/RichardErkhov/MaziyarPanahi_-_TheTop-5x7B-Instruct-S3-v0.1-gguf/blob/main/TheTop-5x7B-Instruct-S3-v0.1.Q5_0.gguf) | Q5_0 | 4.65GB |\n| [TheTop-5x7B-Instruct-S3-v0.1.Q5_K_S.gguf](https://huggingface.co/RichardErkhov/MaziyarPanahi_-_TheTop-5x7B-Instruct-S3-v0.1-gguf/blob/main/TheTop-5x7B-Instruct-S3-v0.1.Q5_K_S.gguf) | Q5_K_S | 4.65GB |\n| [TheTop-5x7B-Instruct-S3-v0.1.Q5_K.gguf](https://huggingface.co/RichardErkhov/MaziyarPanahi_-_TheTop-5x7B-Instruct-S3-v0.1-gguf/blob/main/TheTop-5x7B-Instruct-S3-v0.1.Q5_K.gguf) | Q5_K | 4.78GB |\n| [TheTop-5x7B-Instruct-S3-v0.1.Q5_K_M.gguf](https://huggingface.co/RichardErkhov/MaziyarPanahi_-_TheTop-5x7B-Instruct-S3-v0.1-gguf/blob/main/TheTop-5x7B-Instruct-S3-v0.1.Q5_K_M.gguf) | Q5_K_M | 4.78GB |\n| [TheTop-5x7B-Instruct-S3-v0.1.Q5_1.gguf](https://huggingface.co/RichardErkhov/MaziyarPanahi_-_TheTop-5x7B-Instruct-S3-v0.1-gguf/blob/main/TheTop-5x7B-Instruct-S3-v0.1.Q5_1.gguf) | Q5_1 | 5.07GB |\n| [TheTop-5x7B-Instruct-S3-v0.1.Q6_K.gguf](https://huggingface.co/RichardErkhov/MaziyarPanahi_-_TheTop-5x7B-Instruct-S3-v0.1-gguf/blob/main/TheTop-5x7B-Instruct-S3-v0.1.Q6_K.gguf) | Q6_K | 5.53GB |\n| [TheTop-5x7B-Instruct-S3-v0.1.Q8_0.gguf](https://huggingface.co/RichardErkhov/MaziyarPanahi_-_TheTop-5x7B-Instruct-S3-v0.1-gguf/blob/main/TheTop-5x7B-Instruct-S3-v0.1.Q8_0.gguf) | Q8_0 | 7.17GB |\n\n\n\n\nOriginal model description:\n---\nlicense: apache-2.0\nlibrary_name: transformers\ntags:\n- merge\npipeline_tag: text-generation\nmodel-index:\n- name: TheTop-5x7B-Instruct-S3-v0.1\n  results:\n  - task:\n      type: text-generation\n      name: Text Generation\n    dataset:\n      name: AI2 Reasoning Challenge (25-Shot)\n      type: ai2_arc\n      config: ARC-Challenge\n      split: test\n      args:\n        num_few_shot: 25\n    metrics:\n    - type: acc_norm\n      value: 70.9\n      name: normalized accuracy\n    source:\n      url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=MaziyarPanahi/TheTop-5x7B-Instruct-S3-v0.1\n      name: Open LLM Leaderboard\n  - task:\n      type: text-generation\n      name: Text Generation\n    dataset:\n      name: HellaSwag (10-Shot)\n      type: hellaswag\n      split: validation\n      args:\n        num_few_shot: 10\n    metrics:\n    - type: acc_norm\n      value: 88.0\n      name: normalized accuracy\n    source:\n      url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=MaziyarPanahi/TheTop-5x7B-Instruct-S3-v0.1\n      name: Open LLM Leaderboard\n  - task:\n      type: text-generation\n      name: Text Generation\n    dataset:\n      name: MMLU (5-Shot)\n      type: cais/mmlu\n      config: all\n      split: test\n      args:\n        num_few_shot: 5\n    metrics:\n    - type: acc\n      value: 65.13\n      name: accuracy\n    source:\n      url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=MaziyarPanahi/TheTop-5x7B-Instruct-S3-v0.1\n      name: Open LLM Leaderboard\n  - task:\n      type: text-generation\n      name: Text Generation\n    dataset:\n      name: TruthfulQA (0-shot)\n      type: truthful_qa\n      config: multiple_choice\n      split: validation\n      args:\n        num_few_shot: 0\n    metrics:\n    - type: mc2\n      value: 64.47\n    source:\n      url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=MaziyarPanahi/TheTop-5x7B-Instruct-S3-v0.1\n      name: Open LLM Leaderboard\n  - task:\n      type: text-generation\n      name: Text Generation\n    dataset:\n      name: Winogrande (5-shot)\n      type: winogrande\n      config: winogrande_xl\n      split: validation\n      args:\n        num_few_shot: 5\n    metrics:\n    - type: acc\n      value: 83.66\n      name: accuracy\n    source:\n      url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=MaziyarPanahi/TheTop-5x7B-Instruct-S3-v0.1\n      name: Open LLM Leaderboard\n  - task:\n      type: text-generation\n      name: Text Generation\n    dataset:\n      name: GSM8k (5-shot)\n      type: gsm8k\n      config: main\n      split: test\n      args:\n        num_few_shot: 5\n    metrics:\n    - type: acc\n      value: 72.02\n      name: accuracy\n    source:\n      url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=MaziyarPanahi/TheTop-5x7B-Instruct-S3-v0.1\n      name: Open LLM Leaderboard\n---\n\nMerge of top 7B models and the SLERP of other 7B models\n\n> mergekit is a toolkit for merging pre-trained language models. mergekit uses an out-of-core approach to perform unreasonably elaborate merges in resource-constrained situations. Merges can be run entirely on CPU or accelerated with as little as 8 GB of VRAM. Many merging algorithms are supported, with more coming as they catch my attention.\n>\n> ## Eval\n> ```python\n> {\n    \"all\": {\n        \"acc\": 0.6571641282160704,\n        \"acc_stderr\": 0.031918970852064334,\n        \"acc_norm\": 0.6561506230894164,\n        \"acc_norm_stderr\": 0.03258982989656136,\n        \"mc1\": 0.4834761321909425,\n        \"mc1_stderr\": 0.017493940190057723,\n        \"mc2\": 0.6447306680251751,\n        \"mc2_stderr\": 0.015519245883344577\n    },\n    \"harness|arc:challenge|25\": {\n        \"acc\": 0.689419795221843,\n        \"acc_stderr\": 0.01352229209805306,\n        \"acc_norm\": 0.7090443686006825,\n        \"acc_norm_stderr\": 0.013273077865907595\n    },\n    \"harness|hellaswag|10\": {\n        \"acc\": 0.7168890659231228,\n        \"acc_stderr\": 0.004495891440519419,\n        \"acc_norm\": 0.8800039832702649,\n        \"acc_norm_stderr\": 0.0032429275808698544\n    },\n    \"harness|hendrycksTest-abstract_algebra|5\": {\n        \"acc\": 0.33,\n        \"acc_stderr\": 0.047258156262526045,\n        \"acc_norm\": 0.33,\n        \"acc_norm_stderr\": 0.047258156262526045\n    },\n    \"harness|hendrycksTest-anatomy|5\": {\n        \"acc\": 0.6370370370370371,\n        \"acc_stderr\": 0.04153948404742398,\n        \"acc_norm\": 0.6370370370370371,\n        \"acc_norm_stderr\": 0.04153948404742398\n    },\n    \"harness|hendrycksTest-astronomy|5\": {\n        \"acc\": 0.7105263157894737,\n        \"acc_stderr\": 0.03690677986137283,\n        \"acc_norm\": 0.7105263157894737,\n        \"acc_norm_stderr\": 0.03690677986137283\n    },\n    \"harness|hendrycksTest-business_ethics|5\": {\n        \"acc\": 0.65,\n        \"acc_stderr\": 0.0479372485441102,\n        \"acc_norm\": 0.65,\n        \"acc_norm_stderr\": 0.0479372485441102\n    },\n    \"harness|hendrycksTest-clinical_knowledge|5\": {\n        \"acc\": 0.6981132075471698,\n        \"acc_stderr\": 0.02825420034443866,\n        \"acc_norm\": 0.6981132075471698,\n        \"acc_norm_stderr\": 0.02825420034443866\n    },\n    \"harness|hendrycksTest-college_biology|5\": {\n        \"acc\": 0.7638888888888888,\n        \"acc_stderr\": 0.03551446610810826,\n        \"acc_norm\": 0.7638888888888888,\n        \"acc_norm_stderr\": 0.03551446610810826\n    },\n    \"harness|hendrycksTest-college_chemistry|5\": {\n        \"acc\": 0.48,\n        \"acc_stderr\": 0.050211673156867795,\n        \"acc_norm\": 0.48,\n        \"acc_norm_stderr\": 0.050211673156867795\n    },\n    \"harness|hendrycksTest-college_computer_science|5\": {\n        \"acc\": 0.56,\n        \"acc_stderr\": 0.049888765156985884,\n        \"acc_norm\": 0.56,\n        \"acc_norm_stderr\": 0.049888765156985884\n    },\n    \"harness|hendrycksTest-college_mathematics|5\": {\n        \"acc\": 0.27,\n        \"acc_stderr\": 0.0446196043338474,\n        \"acc_norm\": 0.27,\n        \"acc_norm_stderr\": 0.0446196043338474\n    },\n    \"harness|hendrycksTest-college_medicine|5\": {\n        \"acc\": 0.6589595375722543,\n        \"acc_stderr\": 0.03614665424180826,\n        \"acc_norm\": 0.6589595375722543,\n        \"acc_norm_stderr\": 0.03614665424180826\n    },\n    \"harness|hendrycksTest-college_physics|5\": {\n        \"acc\": 0.4117647058823529,\n        \"acc_stderr\": 0.048971049527263666,\n        \"acc_norm\": 0.4117647058823529,\n        \"acc_norm_stderr\": 0.048971049527263666\n    },\n    \"harness|hendrycksTest-computer_security|5\": {\n        \"acc\": 0.75,\n        \"acc_stderr\": 0.04351941398892446,\n        \"acc_norm\": 0.75,\n        \"acc_norm_stderr\": 0.04351941398892446\n    },\n    \"harness|hendrycksTest-conceptual_physics|5\": {\n        \"acc\": 0.5787234042553191,\n        \"acc_stderr\": 0.03227834510146268,\n        \"acc_norm\": 0.5787234042553191,\n        \"acc_norm_stderr\": 0.03227834510146268\n    },\n    \"harness|hendrycksTest-econometrics|5\": {\n        \"acc\": 0.5175438596491229,\n        \"acc_stderr\": 0.04700708033551038,\n        \"acc_norm\": 0.5175438596491229,\n        \"acc_norm_stderr\": 0.04700708033551038\n    },\n    \"harness|hendrycksTest-electrical_engineering|5\": {\n        \"acc\": 0.5655172413793104,\n        \"acc_stderr\": 0.04130740879555497,\n        \"acc_norm\": 0.5655172413793104,\n        \"acc_norm_stderr\": 0.04130740879555497\n    },\n    \"harness|hendrycksTest-elementary_mathematics|5\": {\n        \"acc\": 0.4312169312169312,\n        \"acc_stderr\": 0.02550648169813821,\n        \"acc_norm\": 0.4312169312169312,\n        \"acc_norm_stderr\": 0.02550648169813821\n    },\n    \"harness|hendrycksTest-formal_logic|5\": {\n        \"acc\": 0.48412698412698413,\n        \"acc_stderr\": 0.04469881854072606,\n        \"acc_norm\": 0.48412698412698413,\n        \"acc_norm_stderr\": 0.04469881854072606\n    },\n    \"harness|hendrycksTest-global_facts|5\": {\n        \"acc\": 0.33,\n        \"acc_stderr\": 0.04725815626252604,\n        \"acc_norm\": 0.33,\n        \"acc_norm_stderr\": 0.04725815626252604\n    },\n    \"harness|hendrycksTest-high_school_biology|5\": {\n        \"acc\": 0.7838709677419354,\n        \"acc_stderr\": 0.02341529343356853,\n        \"acc_norm\": 0.7838709677419354,\n        \"acc_norm_stderr\": 0.02341529343356853\n    },\n    \"harness|hendrycksTest-high_school_chemistry|5\": {\n        \"acc\": 0.4975369458128079,\n        \"acc_stderr\": 0.03517945038691063,\n        \"acc_norm\": 0.4975369458128079,\n        \"acc_norm_stderr\": 0.03517945038691063\n    },\n    \"harness|hendrycksTest-high_school_computer_science|5\": {\n        \"acc\": 0.67,\n        \"acc_stderr\": 0.04725815626252607,\n        \"acc_norm\": 0.67,\n        \"acc_norm_stderr\": 0.04725815626252607\n    },\n    \"harness|hendrycksTest-high_school_european_history|5\": {\n        \"acc\": 0.7878787878787878,\n        \"acc_stderr\": 0.031922715695483,\n        \"acc_norm\": 0.7878787878787878,\n        \"acc_norm_stderr\": 0.031922715695483\n    },\n    \"harness|hendrycksTest-high_school_geography|5\": {\n        \"acc\": 0.7929292929292929,\n        \"acc_stderr\": 0.028869778460267045,\n        \"acc_norm\": 0.7929292929292929,\n        \"acc_norm_stderr\": 0.028869778460267045\n    },\n    \"harness|hendrycksTest-high_school_government_and_politics|5\": {\n        \"acc\": 0.9015544041450777,\n        \"acc_stderr\": 0.021500249576033456,\n        \"acc_norm\": 0.9015544041450777,\n        \"acc_norm_stderr\": 0.021500249576033456\n    },\n    \"harness|hendrycksTest-high_school_macroeconomics|5\": {\n        \"acc\": 0.6666666666666666,\n        \"acc_stderr\": 0.023901157979402534,\n        \"acc_norm\": 0.6666666666666666,\n        \"acc_norm_stderr\": 0.023901157979402534\n    },\n    \"harness|hendrycksTest-high_school_mathematics|5\": {\n        \"acc\": 0.34814814814814815,\n        \"acc_stderr\": 0.029045600290616255,\n        \"acc_norm\": 0.34814814814814815,\n        \"acc_norm_stderr\": 0.029045600290616255\n    },\n    \"harness|hendrycksTest-high_school_microeconomics|5\": {\n        \"acc\": 0.680672268907563,\n        \"acc_stderr\": 0.030283995525884396,\n        \"acc_norm\": 0.680672268907563,\n        \"acc_norm_stderr\": 0.030283995525884396\n    },\n    \"harness|hendrycksTest-high_school_physics|5\": {\n        \"acc\": 0.33112582781456956,\n        \"acc_stderr\": 0.038425817186598696,\n        \"acc_norm\": 0.33112582781456956,\n        \"acc_norm_stderr\": 0.038425817186598696\n    },\n    \"harness|hendrycksTest-high_school_psychology|5\": {\n        \"acc\": 0.8385321100917431,\n        \"acc_stderr\": 0.015776239256163224,\n        \"acc_norm\": 0.8385321100917431,\n        \"acc_norm_stderr\": 0.015776239256163224\n    },\n    \"harness|hendrycksTest-high_school_statistics|5\": {\n        \"acc\": 0.5138888888888888,\n        \"acc_stderr\": 0.03408655867977749,\n        \"acc_norm\": 0.5138888888888888,\n        \"acc_norm_stderr\": 0.03408655867977749\n    },\n    \"harness|hendrycksTest-high_school_us_history|5\": {\n        \"acc\": 0.8578431372549019,\n        \"acc_stderr\": 0.024509803921568603,\n        \"acc_norm\": 0.8578431372549019,\n        \"acc_norm_stderr\": 0.024509803921568603\n    },\n    \"harness|hendrycksTest-high_school_world_history|5\": {\n        \"acc\": 0.8143459915611815,\n        \"acc_stderr\": 0.025310495376944856,\n        \"acc_norm\": 0.8143459915611815,\n        \"acc_norm_stderr\": 0.025310495376944856\n    },\n    \"harness|hendrycksTest-human_aging|5\": {\n        \"acc\": 0.6860986547085202,\n        \"acc_stderr\": 0.031146796482972465,\n        \"acc_norm\": 0.6860986547085202,\n        \"acc_norm_stderr\": 0.031146796482972465\n    },\n    \"harness|hendrycksTest-human_sexuality|5\": {\n        \"acc\": 0.7862595419847328,\n        \"acc_stderr\": 0.0359546161177469,\n        \"acc_norm\": 0.7862595419847328,\n        \"acc_norm_stderr\": 0.0359546161177469\n    },\n    \"harness|hendrycksTest-international_law|5\": {\n        \"acc\": 0.8099173553719008,\n        \"acc_stderr\": 0.03581796951709282,\n        \"acc_norm\": 0.8099173553719008,\n        \"acc_norm_stderr\": 0.03581796951709282\n    },\n    \"harness|hendrycksTest-jurisprudence|5\": {\n        \"acc\": 0.7962962962962963,\n        \"acc_stderr\": 0.03893542518824847,\n        \"acc_norm\": 0.7962962962962963,\n        \"acc_norm_stderr\": 0.03893542518824847\n    },\n    \"harness|hendrycksTest-logical_fallacies|5\": {\n        \"acc\": 0.7730061349693251,\n        \"acc_stderr\": 0.03291099578615769,\n        \"acc_norm\": 0.7730061349693251,\n        \"acc_norm_stderr\": 0.03291099578615769\n    },\n    \"harness|hendrycksTest-machine_learning|5\": {\n        \"acc\": 0.5,\n        \"acc_stderr\": 0.04745789978762494,\n        \"acc_norm\": 0.5,\n        \"acc_norm_stderr\": 0.04745789978762494\n    },\n    \"harness|hendrycksTest-management|5\": {\n        \"acc\": 0.7961165048543689,\n        \"acc_stderr\": 0.03989139859531771,\n        \"acc_norm\": 0.7961165048543689,\n        \"acc_norm_stderr\": 0.03989139859531771\n    },\n    \"harness|hendrycksTest-marketing|5\": {\n        \"acc\": 0.8760683760683761,\n        \"acc_stderr\": 0.02158649400128137,\n        \"acc_norm\": 0.8760683760683761,\n        \"acc_norm_stderr\": 0.02158649400128137\n    },\n    \"harness|hendrycksTest-medical_genetics|5\": {\n        \"acc\": 0.73,\n        \"acc_stderr\": 0.0446196043338474,\n        \"acc_norm\": 0.73,\n        \"acc_norm_stderr\": 0.0446196043338474\n    },\n    \"harness|hendrycksTest-miscellaneous|5\": {\n        \"acc\": 0.8288633461047255,\n        \"acc_stderr\": 0.013468201614066307,\n        \"acc_norm\": 0.8288633461047255,\n        \"acc_norm_stderr\": 0.013468201614066307\n    },\n    \"harness|hendrycksTest-moral_disputes|5\": {\n        \"acc\": 0.7514450867052023,\n        \"acc_stderr\": 0.023267528432100174,\n        \"acc_norm\": 0.7514450867052023,\n        \"acc_norm_stderr\": 0.023267528432100174\n    },\n    \"harness|hendrycksTest-moral_scenarios|5\": {\n        \"acc\": 0.4480446927374302,\n        \"acc_stderr\": 0.016631976628930595,\n        \"acc_norm\": 0.4480446927374302,\n        \"acc_norm_stderr\": 0.016631976628930595\n    },\n    \"harness|hendrycksTest-nutrition|5\": {\n        \"acc\": 0.7320261437908496,\n        \"acc_stderr\": 0.025360603796242553,\n        \"acc_norm\": 0.7320261437908496,\n        \"acc_norm_stderr\": 0.025360603796242553\n    },\n    \"harness|hendrycksTest-philosophy|5\": {\n        \"acc\": 0.707395498392283,\n        \"acc_stderr\": 0.02583989833487798,\n        \"acc_norm\": 0.707395498392283,\n        \"acc_norm_stderr\": 0.02583989833487798\n    },\n    \"harness|hendrycksTest-prehistory|5\": {\n        \"acc\": 0.7530864197530864,\n        \"acc_stderr\": 0.023993501709042107,\n        \"acc_norm\": 0.7530864197530864,\n        \"acc_norm_stderr\": 0.023993501709042107\n    },\n    \"harness|hendrycksTest-professional_accounting|5\": {\n        \"acc\": 0.4787234042553192,\n        \"acc_stderr\": 0.029800481645628693,\n        \"acc_norm\": 0.4787234042553192,\n        \"acc_norm_stderr\": 0.029800481645628693\n    },\n    \"harness|hendrycksTest-professional_law|5\": {\n        \"acc\": 0.4791395045632334,\n        \"acc_stderr\": 0.012759117066518015,\n        \"acc_norm\": 0.4791395045632334,\n        \"acc_norm_stderr\": 0.012759117066518015\n    },\n    \"harness|hendrycksTest-professional_medicine|5\": {\n        \"acc\": 0.7058823529411765,\n        \"acc_stderr\": 0.02767846864214472,\n        \"acc_norm\": 0.7058823529411765,\n        \"acc_norm_stderr\": 0.02767846864214472\n    },\n    \"harness|hendrycksTest-professional_psychology|5\": {\n        \"acc\": 0.6862745098039216,\n        \"acc_stderr\": 0.018771683893528176,\n        \"acc_norm\": 0.6862745098039216,\n        \"acc_norm_stderr\": 0.018771683893528176\n    },\n    \"harness|hendrycksTest-public_relations|5\": {\n        \"acc\": 0.6818181818181818,\n        \"acc_stderr\": 0.04461272175910509,\n        \"acc_norm\": 0.6818181818181818,\n        \"acc_norm_stderr\": 0.04461272175910509\n    },\n    \"harness|hendrycksTest-security_studies|5\": {\n        \"acc\": 0.7346938775510204,\n        \"acc_stderr\": 0.028263889943784603,\n        \"acc_norm\": 0.7346938775510204,\n        \"acc_norm_stderr\": 0.028263889943784603\n    },\n    \"harness|hendrycksTest-sociology|5\": {\n        \"acc\": 0.835820895522388,\n        \"acc_stderr\": 0.026193923544454115,\n        \"acc_norm\": 0.835820895522388,\n        \"acc_norm_stderr\": 0.026193923544454115\n    },\n    \"harness|hendrycksTest-us_foreign_policy|5\": {\n        \"acc\": 0.85,\n        \"acc_stderr\": 0.03588702812826371,\n        \"acc_norm\": 0.85,\n        \"acc_norm_stderr\": 0.03588702812826371\n    },\n    \"harness|hendrycksTest-virology|5\": {\n        \"acc\": 0.5481927710843374,\n        \"acc_stderr\": 0.03874371556587953,\n        \"acc_norm\": 0.5481927710843374,\n        \"acc_norm_stderr\": 0.03874371556587953\n    },\n    \"harness|hendrycksTest-world_religions|5\": {\n        \"acc\": 0.8362573099415205,\n        \"acc_stderr\": 0.028380919596145866,\n        \"acc_norm\": 0.8362573099415205,\n        \"acc_norm_stderr\": 0.028380919596145866\n    },\n    \"harness|truthfulqa:mc|0\": {\n        \"mc1\": 0.4834761321909425,\n        \"mc1_stderr\": 0.017493940190057723,\n        \"mc2\": 0.6447306680251751,\n        \"mc2_stderr\": 0.015519245883344577\n    },\n    \"harness|winogrande|5\": {\n        \"acc\": 0.8366219415943172,\n        \"acc_stderr\": 0.010390695970273764\n    },\n    \"harness|gsm8k|5\": {\n        \"acc\": 0.7202426080363912,\n        \"acc_stderr\": 0.012364384016735319\n    }\n}\n\n```\n# [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)\nDetailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_MaziyarPanahi__TheTop-5x7B-Instruct-S3-v0.1)\n\n|             Metric              |Value|\n|---------------------------------|----:|\n|Avg.                             |74.03|\n|AI2 Reasoning Challenge (25-Shot)|70.90|\n|HellaSwag (10-Shot)              |88.00|\n|MMLU (5-Shot)                    |65.13|\n|TruthfulQA (0-shot)              |64.47|\n|Winogrande (5-shot)              |83.66|\n|GSM8k (5-shot)                   |72.02|\n\n\n\n",
    "related_quantizations": []
  },
  "tags": [
    "gguf",
    "endpoints_compatible",
    "region:us"
  ],
  "likes": 0,
  "downloads": 93,
  "gated": false,
  "private": false,
  "last_modified": "2024-09-22T16:20:43.000Z",
  "created_at": "2024-09-22T13:05:09.000Z",
  "pipeline_tag": "",
  "library_name": ""
}

{
  "_id": "66f01605e32a9f85b6bf50d1",
  "id": "RichardErkhov/MaziyarPanahi_-_TheTop-5x7B-Instruct-S3-v0.1-gguf",
  "modelId": "RichardErkhov/MaziyarPanahi_-_TheTop-5x7B-Instruct-S3-v0.1-gguf",
  "sha": "d550cb47228179b50b4af1f4ad213daea076855a",
  "createdAt": "2024-09-22T13:05:09.000Z",
  "lastModified": "2024-09-22T16:20:43.000Z",
  "author": "RichardErkhov",
  "downloads": 93,
  "likes": 0,
  "gated": false,
  "private": false,
  "pipeline_tag": "",
  "library_name": "",
  "siblings_count": 24
}