richarderkhov/syed-hasan-8503_-_openhermes-gemma-2b-it-gguf overview

{
  "metadata": {},
  "card_data": {
    "frontmatter": {},
    "hero_image_url": "https://raw.githubusercontent.com/OpenAccess-AI-Collective/axolotl/main/image/axolotl-badge-web.png",
    "summary": "!image/jpeg openhermes-gemma-2b-it is a variant of the Gemma 2B language model, which has been further fine-tuned on the OpenHermes-2.5 preference dataset using QLoRA. This fine-tuning process enhances the model's ability to understand and generate responses that align with user preferences in conversational settings. * google/gemma-2b-it * mlabonne/chatml-OpenHermes2.5-dpo-binarized-alpha",
    "quick_links": [],
    "benchmark_table_html": "",
    "readme_markdown": "Quantization made by Richard Erkhov.\n\n[Github](https://github.com/RichardErkhov)\n\n[Discord](https://discord.gg/pvy7H8DZMG)\n\n[Request more models](https://github.com/RichardErkhov/quant_request)\n\n\nopenhermes-gemma-2b-it - GGUF\n- Model creator: https://huggingface.co/Syed-Hasan-8503/\n- Original model: https://huggingface.co/Syed-Hasan-8503/openhermes-gemma-2b-it/\n\n\n| Name | Quant method | Size |\n| ---- | ---- | ---- |\n| [openhermes-gemma-2b-it.Q2_K.gguf](https://huggingface.co/RichardErkhov/Syed-Hasan-8503_-_openhermes-gemma-2b-it-gguf/blob/main/openhermes-gemma-2b-it.Q2_K.gguf) | Q2_K | 1.08GB |\n| [openhermes-gemma-2b-it.IQ3_XS.gguf](https://huggingface.co/RichardErkhov/Syed-Hasan-8503_-_openhermes-gemma-2b-it-gguf/blob/main/openhermes-gemma-2b-it.IQ3_XS.gguf) | IQ3_XS | 1.16GB |\n| [openhermes-gemma-2b-it.IQ3_S.gguf](https://huggingface.co/RichardErkhov/Syed-Hasan-8503_-_openhermes-gemma-2b-it-gguf/blob/main/openhermes-gemma-2b-it.IQ3_S.gguf) | IQ3_S | 1.2GB |\n| [openhermes-gemma-2b-it.Q3_K_S.gguf](https://huggingface.co/RichardErkhov/Syed-Hasan-8503_-_openhermes-gemma-2b-it-gguf/blob/main/openhermes-gemma-2b-it.Q3_K_S.gguf) | Q3_K_S | 1.2GB |\n| [openhermes-gemma-2b-it.IQ3_M.gguf](https://huggingface.co/RichardErkhov/Syed-Hasan-8503_-_openhermes-gemma-2b-it-gguf/blob/main/openhermes-gemma-2b-it.IQ3_M.gguf) | IQ3_M | 1.22GB |\n| [openhermes-gemma-2b-it.Q3_K.gguf](https://huggingface.co/RichardErkhov/Syed-Hasan-8503_-_openhermes-gemma-2b-it-gguf/blob/main/openhermes-gemma-2b-it.Q3_K.gguf) | Q3_K | 1.29GB |\n| [openhermes-gemma-2b-it.Q3_K_M.gguf](https://huggingface.co/RichardErkhov/Syed-Hasan-8503_-_openhermes-gemma-2b-it-gguf/blob/main/openhermes-gemma-2b-it.Q3_K_M.gguf) | Q3_K_M | 1.29GB |\n| [openhermes-gemma-2b-it.Q3_K_L.gguf](https://huggingface.co/RichardErkhov/Syed-Hasan-8503_-_openhermes-gemma-2b-it-gguf/blob/main/openhermes-gemma-2b-it.Q3_K_L.gguf) | Q3_K_L | 1.36GB |\n| [openhermes-gemma-2b-it.IQ4_XS.gguf](https://huggingface.co/RichardErkhov/Syed-Hasan-8503_-_openhermes-gemma-2b-it-gguf/blob/main/openhermes-gemma-2b-it.IQ4_XS.gguf) | IQ4_XS | 1.4GB |\n| [openhermes-gemma-2b-it.Q4_0.gguf](https://huggingface.co/RichardErkhov/Syed-Hasan-8503_-_openhermes-gemma-2b-it-gguf/blob/main/openhermes-gemma-2b-it.Q4_0.gguf) | Q4_0 | 1.44GB |\n| [openhermes-gemma-2b-it.IQ4_NL.gguf](https://huggingface.co/RichardErkhov/Syed-Hasan-8503_-_openhermes-gemma-2b-it-gguf/blob/main/openhermes-gemma-2b-it.IQ4_NL.gguf) | IQ4_NL | 1.45GB |\n| [openhermes-gemma-2b-it.Q4_K_S.gguf](https://huggingface.co/RichardErkhov/Syed-Hasan-8503_-_openhermes-gemma-2b-it-gguf/blob/main/openhermes-gemma-2b-it.Q4_K_S.gguf) | Q4_K_S | 1.45GB |\n| [openhermes-gemma-2b-it.Q4_K.gguf](https://huggingface.co/RichardErkhov/Syed-Hasan-8503_-_openhermes-gemma-2b-it-gguf/blob/main/openhermes-gemma-2b-it.Q4_K.gguf) | Q4_K | 1.52GB |\n| [openhermes-gemma-2b-it.Q4_K_M.gguf](https://huggingface.co/RichardErkhov/Syed-Hasan-8503_-_openhermes-gemma-2b-it-gguf/blob/main/openhermes-gemma-2b-it.Q4_K_M.gguf) | Q4_K_M | 1.52GB |\n| [openhermes-gemma-2b-it.Q4_1.gguf](https://huggingface.co/RichardErkhov/Syed-Hasan-8503_-_openhermes-gemma-2b-it-gguf/blob/main/openhermes-gemma-2b-it.Q4_1.gguf) | Q4_1 | 1.56GB |\n| [openhermes-gemma-2b-it.Q5_0.gguf](https://huggingface.co/RichardErkhov/Syed-Hasan-8503_-_openhermes-gemma-2b-it-gguf/blob/main/openhermes-gemma-2b-it.Q5_0.gguf) | Q5_0 | 1.68GB |\n| [openhermes-gemma-2b-it.Q5_K_S.gguf](https://huggingface.co/RichardErkhov/Syed-Hasan-8503_-_openhermes-gemma-2b-it-gguf/blob/main/openhermes-gemma-2b-it.Q5_K_S.gguf) | Q5_K_S | 1.68GB |\n| [openhermes-gemma-2b-it.Q5_K.gguf](https://huggingface.co/RichardErkhov/Syed-Hasan-8503_-_openhermes-gemma-2b-it-gguf/blob/main/openhermes-gemma-2b-it.Q5_K.gguf) | Q5_K | 1.71GB |\n| [openhermes-gemma-2b-it.Q5_K_M.gguf](https://huggingface.co/RichardErkhov/Syed-Hasan-8503_-_openhermes-gemma-2b-it-gguf/blob/main/openhermes-gemma-2b-it.Q5_K_M.gguf) | Q5_K_M | 1.71GB |\n| [openhermes-gemma-2b-it.Q5_1.gguf](https://huggingface.co/RichardErkhov/Syed-Hasan-8503_-_openhermes-gemma-2b-it-gguf/blob/main/openhermes-gemma-2b-it.Q5_1.gguf) | Q5_1 | 1.79GB |\n| [openhermes-gemma-2b-it.Q6_K.gguf](https://huggingface.co/RichardErkhov/Syed-Hasan-8503_-_openhermes-gemma-2b-it-gguf/blob/main/openhermes-gemma-2b-it.Q6_K.gguf) | Q6_K | 1.92GB |\n| [openhermes-gemma-2b-it.Q8_0.gguf](https://huggingface.co/RichardErkhov/Syed-Hasan-8503_-_openhermes-gemma-2b-it-gguf/blob/main/openhermes-gemma-2b-it.Q8_0.gguf) | Q8_0 | 2.49GB |\n\n\n\n\nOriginal model description:\n---\nlicense: cc-by-nc-4.0\nbase_model: google/gemma-2b-it\ntags:\n- generated_from_trainer\n- axolotl\n- gemma\n- instruct\n- finetune\n- chatml\n- gpt4\n- synthetic data\n- distillation\nmodel-index:\n- name: openhermes-gemma-2b-it\n  results: []\ndatasets:\n- mlabonne/chatml-OpenHermes2.5-dpo-binarized-alpha\nlanguage:\n- en\nlibrary_name: transformers\npipeline_tag: text-generation\n---\n<!-- This model card has been generated automatically according to the information the Trainer had access to. You\nshould probably proofread and complete it, then remove this comment. -->\n\n# openhermes-gemma-2b-it\n\n![image/jpeg](https://cdn-uploads.huggingface.co/production/uploads/64e09e72e43b9464c835735f/lvnDbi9iJIqb1DRlNNE-c.jpeg)\n\nopenhermes-gemma-2b-it is a variant of the Gemma 2B language model, which has been further fine-tuned on the OpenHermes-2.5 preference dataset \nusing QLoRA. This fine-tuning process enhances the model's ability to understand and generate responses that align \nwith user preferences in conversational settings.\n\n\n* [google/gemma-2b-it](https://huggingface.co/google/gemma-2b-it)\n* [mlabonne/chatml-OpenHermes2.5-dpo-binarized-alpha](https://huggingface.co/datasets/mlabonne/chatml-OpenHermes2.5-dpo-binarized-alpha)\n\n</details><br>\n\n## Usage\n\n### Chat Template\n\nThe instruction-tuned models use a chat template that must be adhered to for conversational use.\nThe easiest way to apply it is using the tokenizer's built-in chat template, as shown in the following snippet.\n\nLet's load the model and apply the chat template to a conversation. In this example, we'll start with a single user interaction:\n\n```py\nfrom transformers import AutoTokenizer, AutoModelForCausalLM\nimport transformers\nimport torch\n\nmodel_id = \"Syed-Hasan-8503/openhermes-gemma-2b-it\"\ndtype = torch.bfloat16\n\ntokenizer = AutoTokenizer.from_pretrained(model_id)\nmodel = AutoModelForCausalLM.from_pretrained(\n    model_id,\n    device_map=\"cuda\",\n    torch_dtype=dtype,\n)\n\nchat = [{ \"role\": \"user\", \"content\": \"What is Machine Learning?\" }]\nprompt = tokenizer.apply_chat_template(chat, tokenize=False, add_generation_prompt=True)\n```\n\nAfter the prompt is ready, generation can be performed like this:\n\n```py\ninputs = tokenizer.encode(prompt, add_special_tokens=True, return_tensors=\"pt\")\noutputs = model.generate(input_ids=inputs.to(model.device), max_new_tokens=250)\nprint(tokenizer.decode(outputs[0]))\n```\n\n### Inputs and outputs\n\n*   **Input:** Text string, such as a question, a prompt, or a document to be\n    summarized.\n*   **Output:** Generated English-language text in response to the input, such\n    as an answer to a question, or a summary of a document.\n\n## Evaluation data\n\n🏆 Evaluation\n\n### AGIEVAL\n\n| Task                                      | Version | Metric | Value |   | StdErr |\n|-------------------------------------------|---------|--------|-------|---|---------|\n| agieval\\_aqua\\_rat                        | 0       | acc    | 24.02 | _ | 2.69    |\n| agieval\\_aqua\\_rat                        | 0       | acc\\_norm | 24.02 | _ | 2.69    |\n| agieval\\_logiqa\\_en                      | 0       | acc    | 23.20 | _ | 1.66    |\n| agieval\\_logiqa\\_en                      | 0       | acc\\_norm | 24.42 | _ | 1.69    |\n| agieval\\_lsat\\_ar                        | 0       | acc    | 18.26 | _ | 2.55    |\n| agieval\\_lsat\\_ar                        | 0       | acc\\_norm | 18.70 | _ | 2.58    |\n| agieval\\_lsat\\_lr                        | 0       | acc    | 22.35 | _ | 1.85    |\n| agieval\\_lsat\\_lr                        | 0       | acc\\_norm | 23.53 | _ | 1.88    |\n| agieval\\_lsat\\_rc                        | 0       | acc    | 20.82 | _ | 2.48    |\n| agieval\\_lsat\\_rc                        | 0       | acc\\_norm | 20.07 | _ | 2.45    |\n| agieval\\_sat\\_en                         | 0       | acc    | 32.52 | _ | 3.27    |\n| agieval\\_sat\\_en                         | 0       | acc\\_norm | 32.52 | _ | 3.27    |\n| agieval\\_sat\\_en\\_without\\_passage       | 0       | acc    | 25.73 | _ | 3.05    |\n| agieval\\_sat\\_en\\_without\\_passage       | 0       | acc\\_norm | 24.27 | _ | 2.99    |\n| agieval\\_sat\\_math                        | 0       | acc    | 25.00 | _ | 2.93    |\n| agieval\\_sat\\_math                        | 0       | acc\\_norm | 20.91 | _ | 2.75    |\n\nAverage: 23.8\n\n### GPT4ALL\n\n| Task                 | Version | Metric | Value |   | StdErr |\n|----------------------|---------|--------|-------|---|---------|\n| arc\\_challenge       | 0       | acc    | 21.77 | _ | 1.21    |\n| arc\\_challenge       | 0       | acc\\_norm | 24.15 | _ | 1.25    |\n| arc\\_easy            | 0       | acc    | 37.37 | _ | 0.99    |\n| arc\\_easy            | 0       | acc\\_norm | 36.95 | _ | 0.99    |\n| boolq               | 1       | acc    | 65.60 | _ | 0.83    |\n| hellaswag           | 0       | acc    | 34.54 | _ | 0.47    |\n| hellaswag           | 0       | acc\\_norm | 40.54 | _ | 0.49    |\n| openbookqa          | 0       | acc    | 15.00 | _ | 1.59    |\n| openbookqa          | 0       | acc\\_norm | 27.40 | _ | 2.00    |\n| piqa                | 0       | acc    | 60.88 | _ | 1.14    |\n| piqa                | 0       | acc\\_norm | 60.55 | _ | 1.14    |\n| winogrande          | 0       | acc    | 50.91 | _ | 1.41    |\n\nAverage: 39.9\n\n### BIGBENCH\n\n| Task                              | Version | Metric | Value  | Std Err |\n|-----------------------------------|---------|--------|--------|---------|\n| bigbench\\_causal\\_judgement        | 0       | MCG    | 50     | 2.26   |\n| bigbench\\_date\\_understanding       | 0       | MCG    | 49.14  | 2.18   |\n| bigbench\\_disambiguation\\_qa        | 0       | MCG    | 49.31  | 2.74   |\n| bigbench\\_geometric\\_shapes         | 0       | MCG    | 14.18  | 1.37   |\n| bigbench\\_logical\\_deduction\\_5objs | 0       | MCG    | 49.41  | 2.73   |\n| bigbench\\_logical\\_deduction\\_7objs | 0       | MCG    | 41.48  | 2.46   |\n| bigbench\\_logical\\_deduction\\_3objs | 0       | MCG    | 69.33  | 2.75   |\n| bigbench\\_movie\\_recommendation     | 0       | MCG    | 51.71  | 2.25   |\n| bigbench\\_navigate                 | 0       | MCG    | 50     | 1.58   |\n| bigbench\\_reasoning\\_colored\\_obj   | 0       | MCG    | 51.92  | 0.99   |\n| bigbench\\_ruin\\_names               | 0       | MCG    | 48.14  | 2.01   |\n| bigbench\\_salient\\_trans\\_err\\_detec | 0       | MCG    | 39.92  | 1.2    |\n| bigbench\\_snarks                   | 0       | MCG    | 64.14  | 3.71   |\n| bigbench\\_sports\\_understanding     | 0       | MCG    | 55.31  | 1.59   |\n| bigbench\\_temporal\\_sequences       | 0       | MCG    | 46.92  | 1.4    |\n| bigbench\\_tsk\\_shuff\\_objs\\_5       | 0       | MCG    | 25.04  | 1.01   |\n| bigbench\\_tsk\\_shuff\\_objs\\_7       | 0       | MCG    | 15.04  | 0.72   |\n| bigbench\\_tsk\\_shuff\\_objs\\_3       | 0       | MCG    | 55.33  | 2.75   |\n\n\nAverage: 44.7\n\n### TRUTHFULQA\n\n| Task                             | Version | Metric | Value | Std Err |\n|----------------------------------|---------|--------|--------|----------|\n| truthfulqa\\_mc                   | 1       | mc1    | 30.11  | 1.61    |\n| truthfulqa\\_mc                   | 1       | mc2    | 47.69  | 1.61    |\n\nAverage:38.9\n\n### Training hyperparameters\n\nThe following hyperparameters were used during training:\n- learning_rate: 5e-07\n- train_batch_size: 1\n- eval_batch_size: 8\n- seed: 42\n- gradient_accumulation_steps: 8\n- total_train_batch_size: 8\n- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08\n- lr_scheduler_type: cosine\n- lr_scheduler_warmup_steps: 100\n- training_steps: 1000\n\n\n### 📝 Axolotl Configuration\n\n```yaml\nbase_model: google/gemma-2b-it\nmodel_type: GemmaForCausalLM\ntokenizer_type: GemmaTokenizer\ntrust_remote_code: true\n\nload_in_8bit: false\nload_in_4bit: true\nstrict: false\n\nrl: dpo\nchat_template: chatml\ndatasets:\n  - path: mlabonne/chatml-OpenHermes2.5-dpo-binarized-alpha\n    split: train\n    type: chatml.intel\ndataset_prepared_path:\nval_set_size: 0.01\noutput_dir: ./out\n\nadapter: qlora\nlora_model_dir:\n\nsequence_len: 1800\nsample_packing: false\npad_to_sequence_len: false\n\nlora_r: 16\nlora_alpha: 16\nlora_dropout: 0.05\nlora_target_linear: true\nlora_fan_in_fan_out:\nlora_target_modules:\n\nwandb_project: axolotl-gemma-dpo\nwandb_entity:\nwandb_watch:\nwandb_name:\nwandb_log_model:\n\ngradient_accumulation_steps: 8\nmicro_batch_size: 1\nnum_epochs: 1\noptimizer: paged_adamw_32bit\nlr_scheduler: cosine\nlearning_rate: 5e-7\n\ntrain_on_inputs: false\ngroup_by_length: false\nbf16: true\nfp16: false\ntf32: true\n\ngradient_checkpointing: true\nearly_stopping_patience:\nresume_from_checkpoint:\nlocal_rank:\nlogging_steps: 1\nxformers_attention:\nflash_attention: false\n\nwarmup_steps: 100\nevals_per_epoch: 1\neval_table_size:\neval_table_max_new_tokens: 128\nsave_steps: 1000\nmax_steps: 1000\ndebug:\ndeepspeed:\nweight_decay: 0.0\nfsdp:\nfsdp_config:\nspecial_tokens:\n```\n\n\n### Framework versions\n\n- Transformers 4.39.0.dev0\n- Pytorch 2.1.2+cu118\n- Datasets 2.17.0\n- Tokenizers 0.15.0\n- axolotl: 0.4.0\n\n[<img src=\"https://raw.githubusercontent.com/OpenAccess-AI-Collective/axolotl/main/image/axolotl-badge-web.png\" alt=\"Built with Axolotl\" width=\"200\" height=\"32\"/>](https://github.com/OpenAccess-AI-Collective/axolotl)\n\n",
    "related_quantizations": []
  },
  "tags": [
    "gguf",
    "endpoints_compatible",
    "region:us",
    "conversational"
  ],
  "likes": 0,
  "downloads": 102,
  "gated": false,
  "private": false,
  "last_modified": "2024-10-14T23:05:04.000Z",
  "created_at": "2024-10-14T21:52:29.000Z",
  "pipeline_tag": "",
  "library_name": ""
}

{
  "_id": "670d929db8a445cbf42039dc",
  "id": "RichardErkhov/Syed-Hasan-8503_-_openhermes-gemma-2b-it-gguf",
  "modelId": "RichardErkhov/Syed-Hasan-8503_-_openhermes-gemma-2b-it-gguf",
  "sha": "feda80a90efb4058b19167c402c84be1f7da934d",
  "createdAt": "2024-10-14T21:52:29.000Z",
  "lastModified": "2024-10-14T23:05:04.000Z",
  "author": "RichardErkhov",
  "downloads": 102,
  "likes": 0,
  "gated": false,
  "private": false,
  "pipeline_tag": "",
  "library_name": "",
  "siblings_count": 24
}