richarderkhov/columbia-nlp_-_lion-gemma-2b-dpo-v1.0-gguf overview

{
  "metadata": {},
  "card_data": {
    "frontmatter": {},
    "hero_image_url": "",
    "summary": "The LION-series are trained using an **empirically optimized pipeline** that consists of three stages: SFT, DPO, and online preference learning (online DPO). We find simple techniques such as sequence packing, loss masking in SFT, increasing the preference dataset size in DPO, and online DPO training can significantly improve the performance of language models. Our best models (the LION-series) **exceed the performance of the official instruct models** tuned with closed-source data and algorithms. For training datasets, code, and evaluation scripts, please refer to our paper and codebase.",
    "quick_links": [],
    "benchmark_table_html": "",
    "readme_markdown": "Quantization made by Richard Erkhov.\n\n[Github](https://github.com/RichardErkhov)\n\n[Discord](https://discord.gg/pvy7H8DZMG)\n\n[Request more models](https://github.com/RichardErkhov/quant_request)\n\n\nLION-Gemma-2b-dpo-v1.0 - GGUF\n- Model creator: https://huggingface.co/Columbia-NLP/\n- Original model: https://huggingface.co/Columbia-NLP/LION-Gemma-2b-dpo-v1.0/\n\n\n| Name | Quant method | Size |\n| ---- | ---- | ---- |\n| [LION-Gemma-2b-dpo-v1.0.Q2_K.gguf](https://huggingface.co/RichardErkhov/Columbia-NLP_-_LION-Gemma-2b-dpo-v1.0-gguf/blob/main/LION-Gemma-2b-dpo-v1.0.Q2_K.gguf) | Q2_K | 1.08GB |\n| [LION-Gemma-2b-dpo-v1.0.IQ3_XS.gguf](https://huggingface.co/RichardErkhov/Columbia-NLP_-_LION-Gemma-2b-dpo-v1.0-gguf/blob/main/LION-Gemma-2b-dpo-v1.0.IQ3_XS.gguf) | IQ3_XS | 1.16GB |\n| [LION-Gemma-2b-dpo-v1.0.IQ3_S.gguf](https://huggingface.co/RichardErkhov/Columbia-NLP_-_LION-Gemma-2b-dpo-v1.0-gguf/blob/main/LION-Gemma-2b-dpo-v1.0.IQ3_S.gguf) | IQ3_S | 1.2GB |\n| [LION-Gemma-2b-dpo-v1.0.Q3_K_S.gguf](https://huggingface.co/RichardErkhov/Columbia-NLP_-_LION-Gemma-2b-dpo-v1.0-gguf/blob/main/LION-Gemma-2b-dpo-v1.0.Q3_K_S.gguf) | Q3_K_S | 1.2GB |\n| [LION-Gemma-2b-dpo-v1.0.IQ3_M.gguf](https://huggingface.co/RichardErkhov/Columbia-NLP_-_LION-Gemma-2b-dpo-v1.0-gguf/blob/main/LION-Gemma-2b-dpo-v1.0.IQ3_M.gguf) | IQ3_M | 1.22GB |\n| [LION-Gemma-2b-dpo-v1.0.Q3_K.gguf](https://huggingface.co/RichardErkhov/Columbia-NLP_-_LION-Gemma-2b-dpo-v1.0-gguf/blob/main/LION-Gemma-2b-dpo-v1.0.Q3_K.gguf) | Q3_K | 1.29GB |\n| [LION-Gemma-2b-dpo-v1.0.Q3_K_M.gguf](https://huggingface.co/RichardErkhov/Columbia-NLP_-_LION-Gemma-2b-dpo-v1.0-gguf/blob/main/LION-Gemma-2b-dpo-v1.0.Q3_K_M.gguf) | Q3_K_M | 1.29GB |\n| [LION-Gemma-2b-dpo-v1.0.Q3_K_L.gguf](https://huggingface.co/RichardErkhov/Columbia-NLP_-_LION-Gemma-2b-dpo-v1.0-gguf/blob/main/LION-Gemma-2b-dpo-v1.0.Q3_K_L.gguf) | Q3_K_L | 1.36GB |\n| [LION-Gemma-2b-dpo-v1.0.IQ4_XS.gguf](https://huggingface.co/RichardErkhov/Columbia-NLP_-_LION-Gemma-2b-dpo-v1.0-gguf/blob/main/LION-Gemma-2b-dpo-v1.0.IQ4_XS.gguf) | IQ4_XS | 1.4GB |\n| [LION-Gemma-2b-dpo-v1.0.Q4_0.gguf](https://huggingface.co/RichardErkhov/Columbia-NLP_-_LION-Gemma-2b-dpo-v1.0-gguf/blob/main/LION-Gemma-2b-dpo-v1.0.Q4_0.gguf) | Q4_0 | 1.44GB |\n| [LION-Gemma-2b-dpo-v1.0.IQ4_NL.gguf](https://huggingface.co/RichardErkhov/Columbia-NLP_-_LION-Gemma-2b-dpo-v1.0-gguf/blob/main/LION-Gemma-2b-dpo-v1.0.IQ4_NL.gguf) | IQ4_NL | 1.45GB |\n| [LION-Gemma-2b-dpo-v1.0.Q4_K_S.gguf](https://huggingface.co/RichardErkhov/Columbia-NLP_-_LION-Gemma-2b-dpo-v1.0-gguf/blob/main/LION-Gemma-2b-dpo-v1.0.Q4_K_S.gguf) | Q4_K_S | 1.45GB |\n| [LION-Gemma-2b-dpo-v1.0.Q4_K.gguf](https://huggingface.co/RichardErkhov/Columbia-NLP_-_LION-Gemma-2b-dpo-v1.0-gguf/blob/main/LION-Gemma-2b-dpo-v1.0.Q4_K.gguf) | Q4_K | 1.52GB |\n| [LION-Gemma-2b-dpo-v1.0.Q4_K_M.gguf](https://huggingface.co/RichardErkhov/Columbia-NLP_-_LION-Gemma-2b-dpo-v1.0-gguf/blob/main/LION-Gemma-2b-dpo-v1.0.Q4_K_M.gguf) | Q4_K_M | 1.52GB |\n| [LION-Gemma-2b-dpo-v1.0.Q4_1.gguf](https://huggingface.co/RichardErkhov/Columbia-NLP_-_LION-Gemma-2b-dpo-v1.0-gguf/blob/main/LION-Gemma-2b-dpo-v1.0.Q4_1.gguf) | Q4_1 | 1.56GB |\n| [LION-Gemma-2b-dpo-v1.0.Q5_0.gguf](https://huggingface.co/RichardErkhov/Columbia-NLP_-_LION-Gemma-2b-dpo-v1.0-gguf/blob/main/LION-Gemma-2b-dpo-v1.0.Q5_0.gguf) | Q5_0 | 1.68GB |\n| [LION-Gemma-2b-dpo-v1.0.Q5_K_S.gguf](https://huggingface.co/RichardErkhov/Columbia-NLP_-_LION-Gemma-2b-dpo-v1.0-gguf/blob/main/LION-Gemma-2b-dpo-v1.0.Q5_K_S.gguf) | Q5_K_S | 1.68GB |\n| [LION-Gemma-2b-dpo-v1.0.Q5_K.gguf](https://huggingface.co/RichardErkhov/Columbia-NLP_-_LION-Gemma-2b-dpo-v1.0-gguf/blob/main/LION-Gemma-2b-dpo-v1.0.Q5_K.gguf) | Q5_K | 1.71GB |\n| [LION-Gemma-2b-dpo-v1.0.Q5_K_M.gguf](https://huggingface.co/RichardErkhov/Columbia-NLP_-_LION-Gemma-2b-dpo-v1.0-gguf/blob/main/LION-Gemma-2b-dpo-v1.0.Q5_K_M.gguf) | Q5_K_M | 1.71GB |\n| [LION-Gemma-2b-dpo-v1.0.Q5_1.gguf](https://huggingface.co/RichardErkhov/Columbia-NLP_-_LION-Gemma-2b-dpo-v1.0-gguf/blob/main/LION-Gemma-2b-dpo-v1.0.Q5_1.gguf) | Q5_1 | 1.79GB |\n| [LION-Gemma-2b-dpo-v1.0.Q6_K.gguf](https://huggingface.co/RichardErkhov/Columbia-NLP_-_LION-Gemma-2b-dpo-v1.0-gguf/blob/main/LION-Gemma-2b-dpo-v1.0.Q6_K.gguf) | Q6_K | 1.92GB |\n| [LION-Gemma-2b-dpo-v1.0.Q8_0.gguf](https://huggingface.co/RichardErkhov/Columbia-NLP_-_LION-Gemma-2b-dpo-v1.0-gguf/blob/main/LION-Gemma-2b-dpo-v1.0.Q8_0.gguf) | Q8_0 | 2.49GB |\n\n\n\n\nOriginal model description:\n---\nlibrary_name: transformers\ntags: []\n---\n\n# Model Card for LION-Gemma-2b-dpo-v1.0\n\nThe LION-series are trained using an **empirically optimized pipeline** that consists of three stages: SFT, DPO, and online preference learning (online DPO). We find simple techniques such as sequence packing, loss masking in SFT, increasing the preference dataset size in DPO, and online DPO training can significantly improve the performance of language models. Our best models (the LION-series) **exceed the performance of the official instruct models** tuned with closed-source data and algorithms.\n\nFor training datasets, code, and evaluation scripts, please refer to our [paper](https://arxiv.org/abs/2407.06542) and [codebase](https://github.com/Columbia-NLP-Lab/LionAlignment).\n\n\n## Model description\n\nThis model is finetuned from [`Columbia-NLP/LION-Gemma-2b-sft-v1.0`](https://huggingface.co/Columbia-NLP/LION-Gemma-2b-sft-v1.0) using DPO from the LION pipeline.\n\n- **Model type:** [`gemma-2b`](https://huggingface.co/google/gemma-2b)\n- **Language(s) (NLP):** Primarily English\n- **License:** Gemma Terms of Use\n- **Finetuned from model:** [`Columbia-NLP/LION-Gemma-2b-sft-v1.0`](https://huggingface.co/Columbia-NLP/LION-Gemma-2b-sft-v1.0)\n\n\n## Performance\n\n| Model | Method | Size | Arena-Hard | AlpacaEval-2 | MT-Bench | OpenLLM |\n|-------------|--------|------|------:|------:|---------:|-------:|\n|[Gemma-2b](https://huggingface.co/google/gemma-2b) | - | 2B | - | - | - | 46.69 |\n|[Gemma-2b-it](https://huggingface.co/google/gemma-2b-it) | SFT+RLHF | 2B | 3.4 | 5.44 | 5.63 | 42.75 |\n|[Gemma-2b-zephyr](https://huggingface.co/wandb/gemma-2b-zephyr-dpo) | SFT+DPO | 2B | 0.9 | 2.65 | 4.13 | 46.92 |\n|[LLaMA-2-7b-chat](https://huggingface.co/meta-llama/Llama-2-7b-chat-hf) | SFT | 7B | 4.6 | 5.35 | 6.22 | 53.16 |\n|[Vicuna-7b-v1.5](https://huggingface.co/lmsys/vicuna-7b-v1.5) | SFT | 7B | 2.5 | 7.62 | 6.57 | 52.06 |\n|[LION-Gemma-2b-sft-v1.0 (ours)](https://huggingface.co/Columbia-NLP/LION-Gemma-2b-sft-v1.0) | SFT | 2B | 2.4 | 7.79 | 6.37 | 54.78 |\n|⮕ [LION-Gemma-2b-dpo-v1.0 (ours)](https://huggingface.co/Columbia-NLP/LION-Gemma-2b-dpo-v1.0) | SFT+DPO | 2B | 4.6 | 8.75 | 6.58 | 55.35 |\n|[LION-Gemma-2b-odpo-v1.0 (ours)](https://huggingface.co/Columbia-NLP/LION-Gemma-2b-odpo-v1.0) | SFT+DPO+ODPO | 2B | 5.0 | 9.57 | 6.75 | 55.98 |\n\n\n## Intended uses\n\nTo ensure reproducibility, please use the following chat templates:\n\n```python\nimport torch\nfrom transformers import pipeline\n\npipe = pipeline(\n    \"text-generation\",\n    model=\"Columbia-NLP/LION-Gemma-2b-dpo-v1.0\",\n    device_map=\"auto\",\n    torch_dtype=torch.bfloat16,\n)\nmessages = [\n    {\n        \"role\": \"system\",\n        \"content\": \"\",\n    },\n    {\n        \"role\": \"user\", \n        \"content\": \"Write a short paragraph where every sentence start with the letter A.\"\n    },\n]\noutputs = pipe(\n    messages,\n    max_new_tokens=128,\n    do_sample=True,\n    temperature=0.7,\n    top_p=0.7,\n    stop_sequence=\"<|im_end|>\",\n)\nprint(outputs[0][\"generated_text\"][-1][\"content\"])\n# Alice always aspired to achieve academic excellence.\n# After attending an esteemed academy, she acquired a plethora of knowledge and attended various extracurricular activities.\n# Always eager to apply her newfound skills, Alice undertook ambitious projects and attended various workshops.\n# As a result, Alice acquired a remarkable academic record and became an active member of her community.\n```\n\nto inspect the chat template/manually do generation:\n\n```python\ntokenizer = AutoTokenizer.from_pretrained(\"Columbia-NLP/LION-Gemma-2b-dpo-v1.0\")\nprompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)\nprint(prompt)\n# tokenize prompt and use model.generate\n```\n\n\n### Training details\n\nPlease refer to our [paper](https://arxiv.org/abs/2407.06542) and [codebase](https://github.com/Columbia-NLP-Lab/LionAlignment).\n\n\n## Citation Information\n\nIf you find this model useful in your work, please consider citing our paper:\n\n```\n@misc{yu2024lionsempiricallyoptimizedapproach,\n      title={LIONs: An Empirically Optimized Approach to Align Language Models}, \n      author={Xiao Yu and Qingyang Wu and Yu Li and Zhou Yu},\n      year={2024},\n      eprint={2407.06542},\n      archivePrefix={arXiv},\n      primaryClass={cs.CL},\n      url={https://arxiv.org/abs/2407.06542}, \n}\n```\n\n## Acknowledgements\n\nWe thank the Columbia-NLP group and [articulate.ai](https://www.articulateai.com/) for providing OpenAI API credits and computational resources to conduct our experiments.\n\n",
    "related_quantizations": []
  },
  "tags": [
    "gguf",
    "arxiv:2407.06542",
    "endpoints_compatible",
    "region:us",
    "conversational"
  ],
  "likes": 0,
  "downloads": 1602,
  "gated": false,
  "private": false,
  "last_modified": "2024-08-22T10:05:49.000Z",
  "created_at": "2024-08-22T09:33:13.000Z",
  "pipeline_tag": "",
  "library_name": ""
}

{
  "_id": "66c705d98d4d9cfebbc46cd0",
  "id": "RichardErkhov/Columbia-NLP_-_LION-Gemma-2b-dpo-v1.0-gguf",
  "modelId": "RichardErkhov/Columbia-NLP_-_LION-Gemma-2b-dpo-v1.0-gguf",
  "sha": "0bc9bf53b17fd267f1f4eb5fcd2dbe9698a1bb0b",
  "createdAt": "2024-08-22T09:33:13.000Z",
  "lastModified": "2024-08-22T10:05:49.000Z",
  "author": "RichardErkhov",
  "downloads": 1602,
  "likes": 0,
  "gated": false,
  "private": false,
  "pipeline_tag": "",
  "library_name": "",
  "siblings_count": 24
}