richarderkhov/aalf_-_gemma-2-27b-it-simpo-37k-gguf overview

{
  "metadata": {},
  "card_data": {
    "frontmatter": {},
    "hero_image_url": "",
    "summary": "",
    "quick_links": [],
    "benchmark_table_html": "",
    "readme_markdown": "Quantization made by Richard Erkhov.\n\n[Github](https://github.com/RichardErkhov)\n\n[Discord](https://discord.gg/pvy7H8DZMG)\n\n[Request more models](https://github.com/RichardErkhov/quant_request)\n\n\ngemma-2-27b-it-SimPO-37K - GGUF\n- Model creator: https://huggingface.co/AALF/\n- Original model: https://huggingface.co/AALF/gemma-2-27b-it-SimPO-37K/\n\n\n| Name | Quant method | Size |\n| ---- | ---- | ---- |\n| [gemma-2-27b-it-SimPO-37K.Q2_K.gguf](https://huggingface.co/RichardErkhov/AALF_-_gemma-2-27b-it-SimPO-37K-gguf/blob/main/gemma-2-27b-it-SimPO-37K.Q2_K.gguf) | Q2_K | 9.73GB |\n| [gemma-2-27b-it-SimPO-37K.IQ3_XS.gguf](https://huggingface.co/RichardErkhov/AALF_-_gemma-2-27b-it-SimPO-37K-gguf/blob/main/gemma-2-27b-it-SimPO-37K.IQ3_XS.gguf) | IQ3_XS | 10.76GB |\n| [gemma-2-27b-it-SimPO-37K.IQ3_S.gguf](https://huggingface.co/RichardErkhov/AALF_-_gemma-2-27b-it-SimPO-37K-gguf/blob/main/gemma-2-27b-it-SimPO-37K.IQ3_S.gguf) | IQ3_S | 11.33GB |\n| [gemma-2-27b-it-SimPO-37K.Q3_K_S.gguf](https://huggingface.co/RichardErkhov/AALF_-_gemma-2-27b-it-SimPO-37K-gguf/blob/main/gemma-2-27b-it-SimPO-37K.Q3_K_S.gguf) | Q3_K_S | 11.33GB |\n| [gemma-2-27b-it-SimPO-37K.IQ3_M.gguf](https://huggingface.co/RichardErkhov/AALF_-_gemma-2-27b-it-SimPO-37K-gguf/blob/main/gemma-2-27b-it-SimPO-37K.IQ3_M.gguf) | IQ3_M | 11.6GB |\n| [gemma-2-27b-it-SimPO-37K.Q3_K.gguf](https://huggingface.co/RichardErkhov/AALF_-_gemma-2-27b-it-SimPO-37K-gguf/blob/main/gemma-2-27b-it-SimPO-37K.Q3_K.gguf) | Q3_K | 12.5GB |\n| [gemma-2-27b-it-SimPO-37K.Q3_K_M.gguf](https://huggingface.co/RichardErkhov/AALF_-_gemma-2-27b-it-SimPO-37K-gguf/blob/main/gemma-2-27b-it-SimPO-37K.Q3_K_M.gguf) | Q3_K_M | 12.5GB |\n| [gemma-2-27b-it-SimPO-37K.Q3_K_L.gguf](https://huggingface.co/RichardErkhov/AALF_-_gemma-2-27b-it-SimPO-37K-gguf/blob/main/gemma-2-27b-it-SimPO-37K.Q3_K_L.gguf) | Q3_K_L | 13.52GB |\n| [gemma-2-27b-it-SimPO-37K.IQ4_XS.gguf](https://huggingface.co/RichardErkhov/AALF_-_gemma-2-27b-it-SimPO-37K-gguf/blob/main/gemma-2-27b-it-SimPO-37K.IQ4_XS.gguf) | IQ4_XS | 13.92GB |\n| [gemma-2-27b-it-SimPO-37K.Q4_0.gguf](https://huggingface.co/RichardErkhov/AALF_-_gemma-2-27b-it-SimPO-37K-gguf/blob/main/gemma-2-27b-it-SimPO-37K.Q4_0.gguf) | Q4_0 | 14.56GB |\n| [gemma-2-27b-it-SimPO-37K.IQ4_NL.gguf](https://huggingface.co/RichardErkhov/AALF_-_gemma-2-27b-it-SimPO-37K-gguf/blob/main/gemma-2-27b-it-SimPO-37K.IQ4_NL.gguf) | IQ4_NL | 14.65GB |\n| [gemma-2-27b-it-SimPO-37K.Q4_K_S.gguf](https://huggingface.co/RichardErkhov/AALF_-_gemma-2-27b-it-SimPO-37K-gguf/blob/main/gemma-2-27b-it-SimPO-37K.Q4_K_S.gguf) | Q4_K_S | 14.66GB |\n| [gemma-2-27b-it-SimPO-37K.Q4_K.gguf](https://huggingface.co/RichardErkhov/AALF_-_gemma-2-27b-it-SimPO-37K-gguf/blob/main/gemma-2-27b-it-SimPO-37K.Q4_K.gguf) | Q4_K | 15.5GB |\n| [gemma-2-27b-it-SimPO-37K.Q4_K_M.gguf](https://huggingface.co/RichardErkhov/AALF_-_gemma-2-27b-it-SimPO-37K-gguf/blob/main/gemma-2-27b-it-SimPO-37K.Q4_K_M.gguf) | Q4_K_M | 15.5GB |\n| [gemma-2-27b-it-SimPO-37K.Q4_1.gguf](https://huggingface.co/RichardErkhov/AALF_-_gemma-2-27b-it-SimPO-37K-gguf/blob/main/gemma-2-27b-it-SimPO-37K.Q4_1.gguf) | Q4_1 | 16.07GB |\n| [gemma-2-27b-it-SimPO-37K.Q5_0.gguf](https://huggingface.co/RichardErkhov/AALF_-_gemma-2-27b-it-SimPO-37K-gguf/blob/main/gemma-2-27b-it-SimPO-37K.Q5_0.gguf) | Q5_0 | 17.59GB |\n| [gemma-2-27b-it-SimPO-37K.Q5_K_S.gguf](https://huggingface.co/RichardErkhov/AALF_-_gemma-2-27b-it-SimPO-37K-gguf/blob/main/gemma-2-27b-it-SimPO-37K.Q5_K_S.gguf) | Q5_K_S | 17.59GB |\n| [gemma-2-27b-it-SimPO-37K.Q5_K.gguf](https://huggingface.co/RichardErkhov/AALF_-_gemma-2-27b-it-SimPO-37K-gguf/blob/main/gemma-2-27b-it-SimPO-37K.Q5_K.gguf) | Q5_K | 18.08GB |\n| [gemma-2-27b-it-SimPO-37K.Q5_K_M.gguf](https://huggingface.co/RichardErkhov/AALF_-_gemma-2-27b-it-SimPO-37K-gguf/blob/main/gemma-2-27b-it-SimPO-37K.Q5_K_M.gguf) | Q5_K_M | 18.08GB |\n| [gemma-2-27b-it-SimPO-37K.Q5_1.gguf](https://huggingface.co/RichardErkhov/AALF_-_gemma-2-27b-it-SimPO-37K-gguf/blob/main/gemma-2-27b-it-SimPO-37K.Q5_1.gguf) | Q5_1 | 19.1GB |\n| [gemma-2-27b-it-SimPO-37K.Q6_K.gguf](https://huggingface.co/RichardErkhov/AALF_-_gemma-2-27b-it-SimPO-37K-gguf/blob/main/gemma-2-27b-it-SimPO-37K.Q6_K.gguf) | Q6_K | 20.81GB |\n| [gemma-2-27b-it-SimPO-37K.Q8_0.gguf](https://huggingface.co/RichardErkhov/AALF_-_gemma-2-27b-it-SimPO-37K-gguf/blob/main/gemma-2-27b-it-SimPO-37K.Q8_0.gguf) | Q8_0 | 26.95GB |\n\n\n\n\nOriginal model description:\n---\nlicense: gemma\nlibrary_name: transformers\npipeline_tag: text-generation\nbase_model: google/gemma-2-27b-it\ntags:\n- alignment-handbook\n- generated_from_trainer\n---\n\n# gemma-2-27b-it-SimPO-37K Model Card\n\n## Implementation Details\nWe first followed the [SimPO](https://github.com/princeton-nlp/SimPO) framework to apply [On-Policy Preference Data Generation](https://github.com/princeton-nlp/SimPO/tree/main/on_policy_data_gen) on the [HuggingFaceH4/ultrafeedback_binarized](https://huggingface.co/datasets/HuggingFaceH4/ultrafeedback_binarized) dataset using the [google/gemma-2-27b-it](https://huggingface.co/google/gemma-2-27b-it) model. We then selected prompts where the chosen reward was at least 0.01 higher than the rejected reward, resulting in 37,040 training data points.\n\nModel training was conducted using 8x80G A800 GPUs, leveraging the [alignment-handbook](https://github.com/huggingface/alignment-handbook) library. We used `deepspeed_zero_stage3` with optimizer offloading to the CPU. The `SimPOTrainer` arguments were as follows:\n\n```bash\n# SimPOTrainer arguments\nbf16: true\nbeta: 10\ngamma_beta_ratio: 0.5\ngradient_accumulation_steps: 8\ngradient_checkpointing: true\ngradient_checkpointing_kwargs:\n  use_reentrant: true\nhub_model_id: simpo-exps\nlearning_rate: 8.0e-7\nlog_level: info\nlogging_steps: 1\nlr_scheduler_type: cosine\nmax_length: 2048\nmax_prompt_length: 1800\nnum_train_epochs: 1\noptim: adamw_torch\noutput_dir: outputs/gemma-2-27b-it-SimPO\nrun_name: gemma-2-27b-it-SimPO\nper_device_train_batch_size: 2\npush_to_hub: false\nsave_strategy: \"steps\"\nsave_steps: 100\nsave_total_limit: 20\nseed: 42\nwarmup_ratio: 0.1\nsave_only_model: true\n```\n\n## Citation\n\ngemma model:\n```\n@article{gemma_2024,\n    title={Gemma},\n    url={https://www.kaggle.com/m/3301},\n    DOI={10.34740/KAGGLE/M/3301},\n    publisher={Kaggle},\n    author={Gemma Team},\n    year={2024}\n}\n```\n\nSimPO paper:\n```\n@article{meng2024simpo,\n  title={{SimPO}: Simple preference optimization with a reference-free reward},\n  author={Meng, Yu and Xia, Mengzhou and Chen, Danqi},\n  journal={arXiv preprint arXiv:2405.14734},\n  year={2024}\n}\n```\n\nUltraFeedback paper:\n```\n@article{cui2023ultrafeedback,\n  title={{UltraFeedback}: Boosting language models with high-quality feedback},\n  author={Cui, Ganqu and Yuan, Lifan and Ding, Ning and Yao, Guanming and Zhu, Wei and Ni, Yuan and Xie, Guotong and Liu, Zhiyuan and Sun, Maosong},\n  journal={arXiv preprint arXiv:2310.01377},\n  year={2023}\n}\n```\n\n",
    "related_quantizations": []
  },
  "tags": [
    "gguf",
    "arxiv:2405.14734",
    "arxiv:2310.01377",
    "endpoints_compatible",
    "region:us",
    "conversational"
  ],
  "likes": 0,
  "downloads": 103,
  "gated": false,
  "private": false,
  "last_modified": "2024-09-18T03:56:58.000Z",
  "created_at": "2024-09-17T15:52:52.000Z",
  "pipeline_tag": "",
  "library_name": ""
}

{
  "_id": "66e9a5d4ffe696289baeb2a2",
  "id": "RichardErkhov/AALF_-_gemma-2-27b-it-SimPO-37K-gguf",
  "modelId": "RichardErkhov/AALF_-_gemma-2-27b-it-SimPO-37K-gguf",
  "sha": "3c3cce221fb8828e0f8a32ed32762526a32cd3ab",
  "createdAt": "2024-09-17T15:52:52.000Z",
  "lastModified": "2024-09-18T03:56:58.000Z",
  "author": "RichardErkhov",
  "downloads": 103,
  "likes": 0,
  "gated": false,
  "private": false,
  "pipeline_tag": "",
  "library_name": "",
  "siblings_count": 24
}