richarderkhov/mayflowergmbh_-_wiedervereinigung-7b-dpo-gguf overview

{
  "metadata": {},
  "card_data": {
    "frontmatter": {},
    "hero_image_url": "https://huggingface.co/mayflowergmbh/Wiedervereinigung-7b/resolve/main/Wiedervereinigung-7b.png",
    "summary": "!image/png This is a dpo aligned merge of our favourite german models, scoring 7.11 on the mt-bench-de average. Since the original models based on mistral - three of them on the brilliant german LeoLM/leo-mistral-hessianai-7b - they are reunited in this merged model. Therefore the name, no nationalist ideas involved :-). To improve result quality they are dpo-trained with a german translation of slimorca dpo using hermeo-7B for reject results. If you are gpu-poor like me you can now use LLaMA-Factory to train with german datasets. Kudos to the authors of the original models at DiscoResearch and VAGOsolutions, Malte Ostendorff and Matthias Uhlig. We are your fan club. This model was brought to you and the nvidia bill was paid by Mayflower GmbH.",
    "quick_links": [],
    "benchmark_table_html": "",
    "readme_markdown": "Quantization made by Richard Erkhov.\n\n[Github](https://github.com/RichardErkhov)\n\n[Discord](https://discord.gg/pvy7H8DZMG)\n\n[Request more models](https://github.com/RichardErkhov/quant_request)\n\n\nWiedervereinigung-7b-dpo - GGUF\n- Model creator: https://huggingface.co/mayflowergmbh/\n- Original model: https://huggingface.co/mayflowergmbh/Wiedervereinigung-7b-dpo/\n\n\n| Name | Quant method | Size |\n| ---- | ---- | ---- |\n| [Wiedervereinigung-7b-dpo.Q2_K.gguf](https://huggingface.co/RichardErkhov/mayflowergmbh_-_Wiedervereinigung-7b-dpo-gguf/blob/main/Wiedervereinigung-7b-dpo.Q2_K.gguf) | Q2_K | 2.53GB |\n| [Wiedervereinigung-7b-dpo.IQ3_XS.gguf](https://huggingface.co/RichardErkhov/mayflowergmbh_-_Wiedervereinigung-7b-dpo-gguf/blob/main/Wiedervereinigung-7b-dpo.IQ3_XS.gguf) | IQ3_XS | 2.81GB |\n| [Wiedervereinigung-7b-dpo.IQ3_S.gguf](https://huggingface.co/RichardErkhov/mayflowergmbh_-_Wiedervereinigung-7b-dpo-gguf/blob/main/Wiedervereinigung-7b-dpo.IQ3_S.gguf) | IQ3_S | 2.96GB |\n| [Wiedervereinigung-7b-dpo.Q3_K_S.gguf](https://huggingface.co/RichardErkhov/mayflowergmbh_-_Wiedervereinigung-7b-dpo-gguf/blob/main/Wiedervereinigung-7b-dpo.Q3_K_S.gguf) | Q3_K_S | 2.95GB |\n| [Wiedervereinigung-7b-dpo.IQ3_M.gguf](https://huggingface.co/RichardErkhov/mayflowergmbh_-_Wiedervereinigung-7b-dpo-gguf/blob/main/Wiedervereinigung-7b-dpo.IQ3_M.gguf) | IQ3_M | 3.06GB |\n| [Wiedervereinigung-7b-dpo.Q3_K.gguf](https://huggingface.co/RichardErkhov/mayflowergmbh_-_Wiedervereinigung-7b-dpo-gguf/blob/main/Wiedervereinigung-7b-dpo.Q3_K.gguf) | Q3_K | 3.28GB |\n| [Wiedervereinigung-7b-dpo.Q3_K_M.gguf](https://huggingface.co/RichardErkhov/mayflowergmbh_-_Wiedervereinigung-7b-dpo-gguf/blob/main/Wiedervereinigung-7b-dpo.Q3_K_M.gguf) | Q3_K_M | 3.28GB |\n| [Wiedervereinigung-7b-dpo.Q3_K_L.gguf](https://huggingface.co/RichardErkhov/mayflowergmbh_-_Wiedervereinigung-7b-dpo-gguf/blob/main/Wiedervereinigung-7b-dpo.Q3_K_L.gguf) | Q3_K_L | 3.56GB |\n| [Wiedervereinigung-7b-dpo.IQ4_XS.gguf](https://huggingface.co/RichardErkhov/mayflowergmbh_-_Wiedervereinigung-7b-dpo-gguf/blob/main/Wiedervereinigung-7b-dpo.IQ4_XS.gguf) | IQ4_XS | 3.67GB |\n| [Wiedervereinigung-7b-dpo.Q4_0.gguf](https://huggingface.co/RichardErkhov/mayflowergmbh_-_Wiedervereinigung-7b-dpo-gguf/blob/main/Wiedervereinigung-7b-dpo.Q4_0.gguf) | Q4_0 | 3.83GB |\n| [Wiedervereinigung-7b-dpo.IQ4_NL.gguf](https://huggingface.co/RichardErkhov/mayflowergmbh_-_Wiedervereinigung-7b-dpo-gguf/blob/main/Wiedervereinigung-7b-dpo.IQ4_NL.gguf) | IQ4_NL | 3.87GB |\n| [Wiedervereinigung-7b-dpo.Q4_K_S.gguf](https://huggingface.co/RichardErkhov/mayflowergmbh_-_Wiedervereinigung-7b-dpo-gguf/blob/main/Wiedervereinigung-7b-dpo.Q4_K_S.gguf) | Q4_K_S | 3.86GB |\n| [Wiedervereinigung-7b-dpo.Q4_K.gguf](https://huggingface.co/RichardErkhov/mayflowergmbh_-_Wiedervereinigung-7b-dpo-gguf/blob/main/Wiedervereinigung-7b-dpo.Q4_K.gguf) | Q4_K | 4.07GB |\n| [Wiedervereinigung-7b-dpo.Q4_K_M.gguf](https://huggingface.co/RichardErkhov/mayflowergmbh_-_Wiedervereinigung-7b-dpo-gguf/blob/main/Wiedervereinigung-7b-dpo.Q4_K_M.gguf) | Q4_K_M | 4.07GB |\n| [Wiedervereinigung-7b-dpo.Q4_1.gguf](https://huggingface.co/RichardErkhov/mayflowergmbh_-_Wiedervereinigung-7b-dpo-gguf/blob/main/Wiedervereinigung-7b-dpo.Q4_1.gguf) | Q4_1 | 4.24GB |\n| [Wiedervereinigung-7b-dpo.Q5_0.gguf](https://huggingface.co/RichardErkhov/mayflowergmbh_-_Wiedervereinigung-7b-dpo-gguf/blob/main/Wiedervereinigung-7b-dpo.Q5_0.gguf) | Q5_0 | 4.65GB |\n| [Wiedervereinigung-7b-dpo.Q5_K_S.gguf](https://huggingface.co/RichardErkhov/mayflowergmbh_-_Wiedervereinigung-7b-dpo-gguf/blob/main/Wiedervereinigung-7b-dpo.Q5_K_S.gguf) | Q5_K_S | 4.65GB |\n| [Wiedervereinigung-7b-dpo.Q5_K.gguf](https://huggingface.co/RichardErkhov/mayflowergmbh_-_Wiedervereinigung-7b-dpo-gguf/blob/main/Wiedervereinigung-7b-dpo.Q5_K.gguf) | Q5_K | 4.78GB |\n| [Wiedervereinigung-7b-dpo.Q5_K_M.gguf](https://huggingface.co/RichardErkhov/mayflowergmbh_-_Wiedervereinigung-7b-dpo-gguf/blob/main/Wiedervereinigung-7b-dpo.Q5_K_M.gguf) | Q5_K_M | 4.78GB |\n| [Wiedervereinigung-7b-dpo.Q5_1.gguf](https://huggingface.co/RichardErkhov/mayflowergmbh_-_Wiedervereinigung-7b-dpo-gguf/blob/main/Wiedervereinigung-7b-dpo.Q5_1.gguf) | Q5_1 | 5.07GB |\n| [Wiedervereinigung-7b-dpo.Q6_K.gguf](https://huggingface.co/RichardErkhov/mayflowergmbh_-_Wiedervereinigung-7b-dpo-gguf/blob/main/Wiedervereinigung-7b-dpo.Q6_K.gguf) | Q6_K | 5.53GB |\n| [Wiedervereinigung-7b-dpo.Q8_0.gguf](https://huggingface.co/RichardErkhov/mayflowergmbh_-_Wiedervereinigung-7b-dpo-gguf/blob/main/Wiedervereinigung-7b-dpo.Q8_0.gguf) | Q8_0 | 7.17GB |\n\n\n\n\nOriginal model description:\n---\ntags:\n- merge\n- mergekit\n- lazymergekit\n- DiscoResearch/DiscoLM_German_7b_v1\n- DRXD1000/Phoenix\n- VAGOsolutions/SauerkrautLM-7b-v1-mistral\n- malteos/hermeo-7b\nbase_model:\n- DiscoResearch/DiscoLM_German_7b_v1\n- DRXD1000/Phoenix\n- VAGOsolutions/SauerkrautLM-7b-v1-mistral\n- malteos/hermeo-7b\nlicense: apache-2.0\nlanguage:\n- de\n- en\n---\n\n# Wiedervereinigung-7b-dpo\n![image/png](https://huggingface.co/mayflowergmbh/Wiedervereinigung-7b/resolve/main/Wiedervereinigung-7b.png)\n\nThis is a dpo aligned merge of our favourite german models, scoring 7.11 on the mt-bench-de average.\nSince the original models based on mistral - three of them on the brilliant german LeoLM/leo-mistral-hessianai-7b - they are reunited in this merged model. \nTherefore the name, no nationalist ideas involved :-). \n\nTo improve result quality they are dpo-trained with a german translation of slimorca dpo using hermeo-7B for reject results. \n\nIf you are gpu-poor like me you can now use [LLaMA-Factory](https://github.com/hiyouga/LLaMA-Factory) to train with german datasets.\n\nKudos to the authors of the original models at [DiscoResearch](https://huggingface.co/DiscoResearch) and [VAGOsolutions](https://huggingface.co/VAGOsolutions), [Malte Ostendorff](https://huggingface.co/malteos) \nand [Matthias Uhlig](https://huggingface.co/DRXD1000). We are your fan club.\n\nThis model was brought to you and the nvidia bill was paid by [Mayflower GmbH](https://mayflower.de/).\n\n## Benchmark results: mt-bench-de\n\nIs the merged model alone already good? Well, of course. But it is even better with the help of some dpo tuning.\n\n```json\n{\n    \"first_turn\": 7.3,\n    \"second_turn\": 6.925,\n    \"categories\": {\n        \"writing\": 8.425,\n        \"roleplay\": 8.6,\n        \"reasoning\": 5.4,\n        \"math\": 4.35,\n        \"coding\": 4.3,\n        \"extraction\": 7.975,\n        \"stem\": 8.5,\n        \"humanities\": 9.35\n    },\n    \"average\": 7.1125\n}\n```\n\n## Other Versions\n\nA big thank you to [LoneStriker](https://huggingface.co/LoneStriker) for the quantized models.\n\n| Name | Quant method | Bits | \n| ---- | ---- | ---- | \n[Wiedervereinigung-7b-dpo](https://huggingface.co/mayflowergmbh/Wiedervereinigung-7b-dpo)| Unquantized | 16 |\n[Wiedervereinigung-7b-dpo-GPTQ](https://huggingface.co/LoneStriker/Wiedervereinigung-7b-dpo-GPTQ)| GPTQ | 4 |\n[Wiedervereinigung-7b-dpo-AWQ](https://huggingface.co/LoneStriker/Wiedervereinigung-7b-dpo-AWQ)| AWQ | 4 |\n[Wiedervereinigung-7b-dpo-GGUF](https://huggingface.co/LoneStriker/Wiedervereinigung-7b-dpo-GGUF)| GGUF | 3-8 |\n[Wiedervereinigung-7b-dpo-8.0bpw-h8-exl2](https://huggingface.co/LoneStriker/Wiedervereinigung-7b-dpo-8.0bpw-h8-exl2)| EXL2 | 8 |\n[Wiedervereinigung-7b-dpo-6.0bpw-h6-exl2](https://huggingface.co/LoneStriker/Wiedervereinigung-7b-dpo-6.0bpw-h6-exl2)| EXL2 | 6 |\n[Wiedervereinigung-7b-dpo-5.0bpw-h6-exl2](https://huggingface.co/LoneStriker/Wiedervereinigung-7b-dpo-5.0bpw-h6-exl2)| EXL2 | 5 |\n[Wiedervereinigung-7b-dpo-4.0bpw-h6-exl2](https://huggingface.co/LoneStriker/Wiedervereinigung-7b-dpo-4.0bpw-h6-exl2)| EXL2 | 4 |\n[Wiedervereinigung-7b-dpo-3.0bpw-h6-exl2](https://huggingface.co/LoneStriker/Wiedervereinigung-7b-dpo-3.0bpw-h6-exl2)| EXL2 | 3 |\n\nWiedervereinigung-7b is a  [LazyMergekit](https://colab.research.google.com/drive/1obulZ1ROXHjYLn6PPZJwRR6GzgQogxxb?usp=sharing) merge of:\n* [DiscoResearch/DiscoLM_German_7b_v1](https://huggingface.co/DiscoResearch/DiscoLM_German_7b_v1)\n* [DRXD1000/Phoenix](https://huggingface.co/DRXD1000/Phoenix)\n* [VAGOsolutions/SauerkrautLM-7b-v1-mistral](https://huggingface.co/VAGOsolutions/SauerkrautLM-7b-v1-mistral)\n* [malteos/hermeo-7b](https://huggingface.co/malteos/hermeo-7b)\n\n\n## 🧩 Configuration\n\n```yaml\nmodels:\n  - model: LeoLM/leo-mistral-hessianai-7b\n    # No parameters necessary for base model\n  - model: DiscoResearch/DiscoLM_German_7b_v1\n    parameters:\n      density: 0.6\n      weight: 0.25\n  - model: DRXD1000/Phoenix\n    parameters:\n      density: 0.6\n      weight: 0.25\n  - model: VAGOsolutions/SauerkrautLM-7b-v1-mistral\n    parameters:\n      density: 0.6\n      weight: 0.25\n  - model: malteos/hermeo-7b\n    parameters:\n      density: 0.6\n      weight: 0.25\nmerge_method: dare_ties\nbase_model: LeoLM/leo-mistral-hessianai-7b\nparameters:\n  int8_mask: true\ndtype: bfloat16\n```\n\n\n## 💻 Usage\n\n```python\n!pip install -qU transformers accelerate\n\nfrom transformers import AutoTokenizer\nimport transformers\nimport torch\n\nmodel = \"mayflowergmbh/Wiedervereinigung-7b-dpo\"\nmessages = [{\"role\": \"user\", \"content\": \"Was ist ein deutsches Large Language Model?\"}]\n\ntokenizer = AutoTokenizer.from_pretrained(model)\nprompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)\npipeline = transformers.pipeline(\n    \"text-generation\",\n    model=model,\n    torch_dtype=torch.float16,\n    device_map=\"auto\",\n)\n\noutputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)\nprint(outputs[0][\"generated_text\"])\n```\n\n",
    "related_quantizations": []
  },
  "tags": [
    "gguf",
    "endpoints_compatible",
    "region:us",
    "conversational"
  ],
  "likes": 0,
  "downloads": 270,
  "gated": false,
  "private": false,
  "last_modified": "2024-08-01T13:35:05.000Z",
  "created_at": "2024-08-01T05:50:56.000Z",
  "pipeline_tag": "",
  "library_name": ""
}

{
  "_id": "66ab2240fc35e079a967a555",
  "id": "RichardErkhov/mayflowergmbh_-_Wiedervereinigung-7b-dpo-gguf",
  "modelId": "RichardErkhov/mayflowergmbh_-_Wiedervereinigung-7b-dpo-gguf",
  "sha": "0a2e8f2b03664c83ed0398ab5db63b6c1f3399e6",
  "createdAt": "2024-08-01T05:50:56.000Z",
  "lastModified": "2024-08-01T13:35:05.000Z",
  "author": "RichardErkhov",
  "downloads": 270,
  "likes": 0,
  "gated": false,
  "private": false,
  "pipeline_tag": "",
  "library_name": "",
  "siblings_count": 24
}