richarderkhov/jpacifico_-_chocolatine-14b-instruct-dpo-v1.2-gguf overview

{
  "metadata": {},
  "card_data": {
    "frontmatter": {},
    "hero_image_url": "https://github.com/jpacifico/Chocolatine-LLM/blob/main/Assets/chocolatine_14B_leaderboard_20240901.png?raw=false",
    "summary": "message = [ {\"role\": \"system\", \"content\": \"You are a helpful assistant chatbot.\"}, {\"role\": \"user\", \"content\": \"What is a Large Language Model?\"} ] tokenizer = AutoTokenizer.from_pretrained(new_model) prompt = tokenizer.apply_chat_template(message, add_generation_prompt=True, tokenize=False) # Create pipeline pipeline = transformers.pipeline( \"text-generation\", model=new_model, tokenizer=tokenizer ) # Generate text sequences = pipeline( prompt, do_sample=True, temperature=0.7, top_p=0.9, num_return_sequences=1, max_length=200, ) print(sequences[0]['generated_text']) ``` ### Limitations The Chocolatine model is a quick demonstration that a base model can be easily fine-tuned to achieve compelling performance. It does not have any moderation mechanism.",
    "quick_links": [],
    "benchmark_table_html": "",
    "readme_markdown": "Quantization made by Richard Erkhov.\n\n[Github](https://github.com/RichardErkhov)\n\n[Discord](https://discord.gg/pvy7H8DZMG)\n\n[Request more models](https://github.com/RichardErkhov/quant_request)\n\n\nChocolatine-14B-Instruct-DPO-v1.2 - GGUF\n- Model creator: https://huggingface.co/jpacifico/\n- Original model: https://huggingface.co/jpacifico/Chocolatine-14B-Instruct-DPO-v1.2/\n\n\n| Name | Quant method | Size |\n| ---- | ---- | ---- |\n| [Chocolatine-14B-Instruct-DPO-v1.2.Q2_K.gguf](https://huggingface.co/RichardErkhov/jpacifico_-_Chocolatine-14B-Instruct-DPO-v1.2-gguf/blob/main/Chocolatine-14B-Instruct-DPO-v1.2.Q2_K.gguf) | Q2_K | 4.79GB |\n| [Chocolatine-14B-Instruct-DPO-v1.2.IQ3_XS.gguf](https://huggingface.co/RichardErkhov/jpacifico_-_Chocolatine-14B-Instruct-DPO-v1.2-gguf/blob/main/Chocolatine-14B-Instruct-DPO-v1.2.IQ3_XS.gguf) | IQ3_XS | 5.41GB |\n| [Chocolatine-14B-Instruct-DPO-v1.2.IQ3_S.gguf](https://huggingface.co/RichardErkhov/jpacifico_-_Chocolatine-14B-Instruct-DPO-v1.2-gguf/blob/main/Chocolatine-14B-Instruct-DPO-v1.2.IQ3_S.gguf) | IQ3_S | 5.65GB |\n| [Chocolatine-14B-Instruct-DPO-v1.2.Q3_K_S.gguf](https://huggingface.co/RichardErkhov/jpacifico_-_Chocolatine-14B-Instruct-DPO-v1.2-gguf/blob/main/Chocolatine-14B-Instruct-DPO-v1.2.Q3_K_S.gguf) | Q3_K_S | 5.65GB |\n| [Chocolatine-14B-Instruct-DPO-v1.2.IQ3_M.gguf](https://huggingface.co/RichardErkhov/jpacifico_-_Chocolatine-14B-Instruct-DPO-v1.2-gguf/blob/main/Chocolatine-14B-Instruct-DPO-v1.2.IQ3_M.gguf) | IQ3_M | 6.03GB |\n| [Chocolatine-14B-Instruct-DPO-v1.2.Q3_K.gguf](https://huggingface.co/RichardErkhov/jpacifico_-_Chocolatine-14B-Instruct-DPO-v1.2-gguf/blob/main/Chocolatine-14B-Instruct-DPO-v1.2.Q3_K.gguf) | Q3_K | 6.45GB |\n| [Chocolatine-14B-Instruct-DPO-v1.2.Q3_K_M.gguf](https://huggingface.co/RichardErkhov/jpacifico_-_Chocolatine-14B-Instruct-DPO-v1.2-gguf/blob/main/Chocolatine-14B-Instruct-DPO-v1.2.Q3_K_M.gguf) | Q3_K_M | 6.45GB |\n| [Chocolatine-14B-Instruct-DPO-v1.2.Q3_K_L.gguf](https://huggingface.co/RichardErkhov/jpacifico_-_Chocolatine-14B-Instruct-DPO-v1.2-gguf/blob/main/Chocolatine-14B-Instruct-DPO-v1.2.Q3_K_L.gguf) | Q3_K_L | 6.98GB |\n| [Chocolatine-14B-Instruct-DPO-v1.2.IQ4_XS.gguf](https://huggingface.co/RichardErkhov/jpacifico_-_Chocolatine-14B-Instruct-DPO-v1.2-gguf/blob/main/Chocolatine-14B-Instruct-DPO-v1.2.IQ4_XS.gguf) | IQ4_XS | 7.02GB |\n| [Chocolatine-14B-Instruct-DPO-v1.2.Q4_0.gguf](https://huggingface.co/RichardErkhov/jpacifico_-_Chocolatine-14B-Instruct-DPO-v1.2-gguf/blob/main/Chocolatine-14B-Instruct-DPO-v1.2.Q4_0.gguf) | Q4_0 | 7.35GB |\n| [Chocolatine-14B-Instruct-DPO-v1.2.IQ4_NL.gguf](https://huggingface.co/RichardErkhov/jpacifico_-_Chocolatine-14B-Instruct-DPO-v1.2-gguf/blob/main/Chocolatine-14B-Instruct-DPO-v1.2.IQ4_NL.gguf) | IQ4_NL | 7.41GB |\n| [Chocolatine-14B-Instruct-DPO-v1.2.Q4_K_S.gguf](https://huggingface.co/RichardErkhov/jpacifico_-_Chocolatine-14B-Instruct-DPO-v1.2-gguf/blob/main/Chocolatine-14B-Instruct-DPO-v1.2.Q4_K_S.gguf) | Q4_K_S | 7.41GB |\n| [Chocolatine-14B-Instruct-DPO-v1.2.Q4_K.gguf](https://huggingface.co/RichardErkhov/jpacifico_-_Chocolatine-14B-Instruct-DPO-v1.2-gguf/blob/main/Chocolatine-14B-Instruct-DPO-v1.2.Q4_K.gguf) | Q4_K | 7.98GB |\n| [Chocolatine-14B-Instruct-DPO-v1.2.Q4_K_M.gguf](https://huggingface.co/RichardErkhov/jpacifico_-_Chocolatine-14B-Instruct-DPO-v1.2-gguf/blob/main/Chocolatine-14B-Instruct-DPO-v1.2.Q4_K_M.gguf) | Q4_K_M | 7.98GB |\n| [Chocolatine-14B-Instruct-DPO-v1.2.Q4_1.gguf](https://huggingface.co/RichardErkhov/jpacifico_-_Chocolatine-14B-Instruct-DPO-v1.2-gguf/blob/main/Chocolatine-14B-Instruct-DPO-v1.2.Q4_1.gguf) | Q4_1 | 8.16GB |\n| [Chocolatine-14B-Instruct-DPO-v1.2.Q5_0.gguf](https://huggingface.co/RichardErkhov/jpacifico_-_Chocolatine-14B-Instruct-DPO-v1.2-gguf/blob/main/Chocolatine-14B-Instruct-DPO-v1.2.Q5_0.gguf) | Q5_0 | 8.96GB |\n| [Chocolatine-14B-Instruct-DPO-v1.2.Q5_K_S.gguf](https://huggingface.co/RichardErkhov/jpacifico_-_Chocolatine-14B-Instruct-DPO-v1.2-gguf/blob/main/Chocolatine-14B-Instruct-DPO-v1.2.Q5_K_S.gguf) | Q5_K_S | 8.96GB |\n| [Chocolatine-14B-Instruct-DPO-v1.2.Q5_K.gguf](https://huggingface.co/RichardErkhov/jpacifico_-_Chocolatine-14B-Instruct-DPO-v1.2-gguf/blob/main/Chocolatine-14B-Instruct-DPO-v1.2.Q5_K.gguf) | Q5_K | 9.38GB |\n| [Chocolatine-14B-Instruct-DPO-v1.2.Q5_K_M.gguf](https://huggingface.co/RichardErkhov/jpacifico_-_Chocolatine-14B-Instruct-DPO-v1.2-gguf/blob/main/Chocolatine-14B-Instruct-DPO-v1.2.Q5_K_M.gguf) | Q5_K_M | 9.38GB |\n| [Chocolatine-14B-Instruct-DPO-v1.2.Q5_1.gguf](https://huggingface.co/RichardErkhov/jpacifico_-_Chocolatine-14B-Instruct-DPO-v1.2-gguf/blob/main/Chocolatine-14B-Instruct-DPO-v1.2.Q5_1.gguf) | Q5_1 | 9.76GB |\n| [Chocolatine-14B-Instruct-DPO-v1.2.Q6_K.gguf](https://huggingface.co/RichardErkhov/jpacifico_-_Chocolatine-14B-Instruct-DPO-v1.2-gguf/blob/main/Chocolatine-14B-Instruct-DPO-v1.2.Q6_K.gguf) | Q6_K | 10.67GB |\n| [Chocolatine-14B-Instruct-DPO-v1.2.Q8_0.gguf](https://huggingface.co/RichardErkhov/jpacifico_-_Chocolatine-14B-Instruct-DPO-v1.2-gguf/blob/main/Chocolatine-14B-Instruct-DPO-v1.2.Q8_0.gguf) | Q8_0 | 13.82GB |\n\n\n\n\nOriginal model description:\n---\nlibrary_name: transformers\nlicense: mit\nlanguage:\n- fr\n- en\ntags:\n- french\n- chocolatine\ndatasets:\n- jpacifico/french-orca-dpo-pairs-revised\npipeline_tag: text-generation\n---\n\n### Chocolatine-14B-Instruct-DPO-v1.2\n\nDPO fine-tuned of [microsoft/Phi-3-medium-4k-instruct](https://huggingface.co/microsoft/Phi-3-medium-4k-instruct) (14B params)  \nusing the [jpacifico/french-orca-dpo-pairs-revised](https://huggingface.co/datasets/jpacifico/french-orca-dpo-pairs-revised) rlhf dataset.  \nTraining in French also improves the model in English, surpassing the performances of its base model.  \nWindow context = 4k tokens  \n\n* **4-bit quantized version** is available here : [jpacifico/Chocolatine-14B-Instruct-DPO-v1.2-Q4_K_M-GGUF](https://huggingface.co/jpacifico/Chocolatine-14B-Instruct-DPO-v1.2-Q4_K_M-GGUF)\n\n### OpenLLM Leaderboard\n\nChocolatine is the best-performing 14B model on the [OpenLLM Leaderboard](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard) (2024/09/01)   \nand even the number one of the < 22B params models\n![image/png](https://github.com/jpacifico/Chocolatine-LLM/blob/main/Assets/chocolatine_14B_leaderboard_20240901.png?raw=false)  \n\n\n|      Metric       |Value|\n|-------------------|----:|\n|**Avg.**               |**33.3**|\n|IFEval     |68.52|\n|BBH        |49.85|\n|MATH Lvl 5 |17.98|\n|GPQA       |10.07|\n|MuSR       |12.35|\n|MMLU-PRO   |41.07|\n\n### MT-Bench-French\n\nChocolatine-14B-Instruct-DPO-v1.2 outperforms its previous versions and its base model Phi-3-medium-4k-instruct on [MT-Bench-French](https://huggingface.co/datasets/bofenghuang/mt-bench-french), used with [multilingual-mt-bench](https://github.com/Peter-Devine/multilingual_mt_bench) and GPT-4-Turbo as LLM-judge. \n\n```\n########## First turn ##########\n                                             score\nmodel                                 turn        \ngpt-4o-mini                           1     9.2875\nChocolatine-14B-Instruct-4k-DPO       1     8.6375\nChocolatine-14B-Instruct-DPO-v1.2     1     8.6125\nPhi-3.5-mini-instruct                 1     8.5250\nChocolatine-3B-Instruct-DPO-v1.2      1     8.3750\nPhi-3-medium-4k-instruct              1     8.2250\ngpt-3.5-turbo                         1     8.1375\nChocolatine-3B-Instruct-DPO-Revised   1     7.9875\nDaredevil-8B                          1     7.8875\nMeta-Llama-3.1-8B-Instruct            1     7.0500\nvigostral-7b-chat                     1     6.7875\nMistral-7B-Instruct-v0.3              1     6.7500\ngemma-2-2b-it                         1     6.4500\nFrench-Alpaca-7B-Instruct_beta        1     5.6875\nvigogne-2-7b-chat                     1     5.6625\n\n########## Second turn ##########\n                                               score\nmodel                                 turn          \ngpt-4o-mini                           2     8.912500\nChocolatine-14B-Instruct-DPO-v1.2     2     8.337500\nChocolatine-3B-Instruct-DPO-Revised   2     7.937500\nChocolatine-3B-Instruct-DPO-v1.2      2     7.862500\nPhi-3-medium-4k-instruct              2     7.750000\nChocolatine-14B-Instruct-4k-DPO       2     7.737500\ngpt-3.5-turbo                         2     7.679167\nPhi-3.5-mini-instruct                 2     7.575000\nDaredevil-8B                          2     7.087500\nMeta-Llama-3.1-8B-Instruct            2     6.787500\nMistral-7B-Instruct-v0.3              2     6.500000\nvigostral-7b-chat                     2     6.162500\ngemma-2-2b-it                         2     6.100000\nFrench-Alpaca-7B-Instruct_beta        2     5.487395\nvigogne-2-7b-chat                     2     2.775000\n\n########## Average ##########\n                                          score\nmodel                                          \ngpt-4o-mini                            9.100000\nChocolatine-14B-Instruct-DPO-v1.2      8.475000\nChocolatine-14B-Instruct-4k-DPO        8.187500\nChocolatine-3B-Instruct-DPO-v1.2       8.118750\nPhi-3.5-mini-instruct                  8.050000\nPhi-3-medium-4k-instruct               7.987500\nChocolatine-3B-Instruct-DPO-Revised    7.962500\ngpt-3.5-turbo                          7.908333\nDaredevil-8B                           7.487500\nMeta-Llama-3.1-8B-Instruct             6.918750\nMistral-7B-Instruct-v0.3               6.625000\nvigostral-7b-chat                      6.475000\ngemma-2-2b-it                          6.275000\nFrench-Alpaca-7B-Instruct_beta         5.587866\nvigogne-2-7b-chat                      4.218750\n```\n\n### Usage\n\nYou can run this model using my [Colab notebook](https://github.com/jpacifico/Chocolatine-LLM/blob/main/Chocolatine_14B_inference_test_colab.ipynb) \n\nYou can also run Chocolatine using the following code:\n\n```python\nimport transformers\nfrom transformers import AutoTokenizer\n\n# Format prompt\nmessage = [\n    {\"role\": \"system\", \"content\": \"You are a helpful assistant chatbot.\"},\n    {\"role\": \"user\", \"content\": \"What is a Large Language Model?\"}\n]\ntokenizer = AutoTokenizer.from_pretrained(new_model)\nprompt = tokenizer.apply_chat_template(message, add_generation_prompt=True, tokenize=False)\n\n# Create pipeline\npipeline = transformers.pipeline(\n    \"text-generation\",\n    model=new_model,\n    tokenizer=tokenizer\n)\n\n# Generate text\nsequences = pipeline(\n    prompt,\n    do_sample=True,\n    temperature=0.7,\n    top_p=0.9,\n    num_return_sequences=1,\n    max_length=200,\n)\nprint(sequences[0]['generated_text'])\n```\n\n### Limitations\n\nThe Chocolatine model is a quick demonstration that a base model can be easily fine-tuned to achieve compelling performance.  \nIt does not have any moderation mechanism.  \n\n- **Developed by:** Jonathan Pacifico, 2024\n- **Model type:** LLM \n- **Language(s) (NLP):** French, English\n- **License:** MIT\n\n",
    "related_quantizations": []
  },
  "tags": [
    "gguf",
    "endpoints_compatible",
    "region:us",
    "conversational"
  ],
  "likes": 0,
  "downloads": 108,
  "gated": false,
  "private": false,
  "last_modified": "2024-09-17T19:32:12.000Z",
  "created_at": "2024-09-17T13:58:50.000Z",
  "pipeline_tag": "",
  "library_name": ""
}

{
  "_id": "66e98b1a18307a56bd944a63",
  "id": "RichardErkhov/jpacifico_-_Chocolatine-14B-Instruct-DPO-v1.2-gguf",
  "modelId": "RichardErkhov/jpacifico_-_Chocolatine-14B-Instruct-DPO-v1.2-gguf",
  "sha": "e02ca86e1afa5b3d1c5d39180fc63d690c4efd34",
  "createdAt": "2024-09-17T13:58:50.000Z",
  "lastModified": "2024-09-17T19:32:12.000Z",
  "author": "RichardErkhov",
  "downloads": 108,
  "likes": 0,
  "gated": false,
  "private": false,
  "pipeline_tag": "",
  "library_name": "",
  "siblings_count": 24
}