richarderkhov/jsgreenawalt_-_gemma-2-9b-it-advanced-v2.1-gguf overview

{
  "metadata": {},
  "card_data": {
    "frontmatter": {},
    "hero_image_url": "",
    "summary": "This is a merge of the 'smartest' advanced fine-tunes available for Gemma-2-9b-it. It includes WPO, SimPO, and SPPO. The merge was performed via the SOTA 'della' merge method. Merge parameters have been hand-tuned for best results. The Q8_0 quant is highly recommended until better quants come along.",
    "quick_links": [],
    "benchmark_table_html": "",
    "readme_markdown": "Quantization made by Richard Erkhov.\n\n[Github](https://github.com/RichardErkhov)\n\n[Discord](https://discord.gg/pvy7H8DZMG)\n\n[Request more models](https://github.com/RichardErkhov/quant_request)\n\n\ngemma-2-9B-it-advanced-v2.1 - GGUF\n- Model creator: https://huggingface.co/jsgreenawalt/\n- Original model: https://huggingface.co/jsgreenawalt/gemma-2-9B-it-advanced-v2.1/\n\n\n| Name | Quant method | Size |\n| ---- | ---- | ---- |\n| [gemma-2-9B-it-advanced-v2.1.Q2_K.gguf](https://huggingface.co/RichardErkhov/jsgreenawalt_-_gemma-2-9B-it-advanced-v2.1-gguf/blob/main/gemma-2-9B-it-advanced-v2.1.Q2_K.gguf) | Q2_K | 3.54GB |\n| [gemma-2-9B-it-advanced-v2.1.IQ3_XS.gguf](https://huggingface.co/RichardErkhov/jsgreenawalt_-_gemma-2-9B-it-advanced-v2.1-gguf/blob/main/gemma-2-9B-it-advanced-v2.1.IQ3_XS.gguf) | IQ3_XS | 3.86GB |\n| [gemma-2-9B-it-advanced-v2.1.IQ3_S.gguf](https://huggingface.co/RichardErkhov/jsgreenawalt_-_gemma-2-9B-it-advanced-v2.1-gguf/blob/main/gemma-2-9B-it-advanced-v2.1.IQ3_S.gguf) | IQ3_S | 4.04GB |\n| [gemma-2-9B-it-advanced-v2.1.Q3_K_S.gguf](https://huggingface.co/RichardErkhov/jsgreenawalt_-_gemma-2-9B-it-advanced-v2.1-gguf/blob/main/gemma-2-9B-it-advanced-v2.1.Q3_K_S.gguf) | Q3_K_S | 4.04GB |\n| [gemma-2-9B-it-advanced-v2.1.IQ3_M.gguf](https://huggingface.co/RichardErkhov/jsgreenawalt_-_gemma-2-9B-it-advanced-v2.1-gguf/blob/main/gemma-2-9B-it-advanced-v2.1.IQ3_M.gguf) | IQ3_M | 4.19GB |\n| [gemma-2-9B-it-advanced-v2.1.Q3_K.gguf](https://huggingface.co/RichardErkhov/jsgreenawalt_-_gemma-2-9B-it-advanced-v2.1-gguf/blob/main/gemma-2-9B-it-advanced-v2.1.Q3_K.gguf) | Q3_K | 4.43GB |\n| [gemma-2-9B-it-advanced-v2.1.Q3_K_M.gguf](https://huggingface.co/RichardErkhov/jsgreenawalt_-_gemma-2-9B-it-advanced-v2.1-gguf/blob/main/gemma-2-9B-it-advanced-v2.1.Q3_K_M.gguf) | Q3_K_M | 4.43GB |\n| [gemma-2-9B-it-advanced-v2.1.Q3_K_L.gguf](https://huggingface.co/RichardErkhov/jsgreenawalt_-_gemma-2-9B-it-advanced-v2.1-gguf/blob/main/gemma-2-9B-it-advanced-v2.1.Q3_K_L.gguf) | Q3_K_L | 4.78GB |\n| [gemma-2-9B-it-advanced-v2.1.IQ4_XS.gguf](https://huggingface.co/RichardErkhov/jsgreenawalt_-_gemma-2-9B-it-advanced-v2.1-gguf/blob/main/gemma-2-9B-it-advanced-v2.1.IQ4_XS.gguf) | IQ4_XS | 4.86GB |\n| [gemma-2-9B-it-advanced-v2.1.Q4_0.gguf](https://huggingface.co/RichardErkhov/jsgreenawalt_-_gemma-2-9B-it-advanced-v2.1-gguf/blob/main/gemma-2-9B-it-advanced-v2.1.Q4_0.gguf) | Q4_0 | 5.07GB |\n| [gemma-2-9B-it-advanced-v2.1.IQ4_NL.gguf](https://huggingface.co/RichardErkhov/jsgreenawalt_-_gemma-2-9B-it-advanced-v2.1-gguf/blob/main/gemma-2-9B-it-advanced-v2.1.IQ4_NL.gguf) | IQ4_NL | 5.1GB |\n| [gemma-2-9B-it-advanced-v2.1.Q4_K_S.gguf](https://huggingface.co/RichardErkhov/jsgreenawalt_-_gemma-2-9B-it-advanced-v2.1-gguf/blob/main/gemma-2-9B-it-advanced-v2.1.Q4_K_S.gguf) | Q4_K_S | 5.1GB |\n| [gemma-2-9B-it-advanced-v2.1.Q4_K.gguf](https://huggingface.co/RichardErkhov/jsgreenawalt_-_gemma-2-9B-it-advanced-v2.1-gguf/blob/main/gemma-2-9B-it-advanced-v2.1.Q4_K.gguf) | Q4_K | 5.37GB |\n| [gemma-2-9B-it-advanced-v2.1.Q4_K_M.gguf](https://huggingface.co/RichardErkhov/jsgreenawalt_-_gemma-2-9B-it-advanced-v2.1-gguf/blob/main/gemma-2-9B-it-advanced-v2.1.Q4_K_M.gguf) | Q4_K_M | 5.37GB |\n| [gemma-2-9B-it-advanced-v2.1.Q4_1.gguf](https://huggingface.co/RichardErkhov/jsgreenawalt_-_gemma-2-9B-it-advanced-v2.1-gguf/blob/main/gemma-2-9B-it-advanced-v2.1.Q4_1.gguf) | Q4_1 | 5.55GB |\n| [gemma-2-9B-it-advanced-v2.1.Q5_0.gguf](https://huggingface.co/RichardErkhov/jsgreenawalt_-_gemma-2-9B-it-advanced-v2.1-gguf/blob/main/gemma-2-9B-it-advanced-v2.1.Q5_0.gguf) | Q5_0 | 6.04GB |\n| [gemma-2-9B-it-advanced-v2.1.Q5_K_S.gguf](https://huggingface.co/RichardErkhov/jsgreenawalt_-_gemma-2-9B-it-advanced-v2.1-gguf/blob/main/gemma-2-9B-it-advanced-v2.1.Q5_K_S.gguf) | Q5_K_S | 6.04GB |\n| [gemma-2-9B-it-advanced-v2.1.Q5_K.gguf](https://huggingface.co/RichardErkhov/jsgreenawalt_-_gemma-2-9B-it-advanced-v2.1-gguf/blob/main/gemma-2-9B-it-advanced-v2.1.Q5_K.gguf) | Q5_K | 6.19GB |\n| [gemma-2-9B-it-advanced-v2.1.Q5_K_M.gguf](https://huggingface.co/RichardErkhov/jsgreenawalt_-_gemma-2-9B-it-advanced-v2.1-gguf/blob/main/gemma-2-9B-it-advanced-v2.1.Q5_K_M.gguf) | Q5_K_M | 6.19GB |\n| [gemma-2-9B-it-advanced-v2.1.Q5_1.gguf](https://huggingface.co/RichardErkhov/jsgreenawalt_-_gemma-2-9B-it-advanced-v2.1-gguf/blob/main/gemma-2-9B-it-advanced-v2.1.Q5_1.gguf) | Q5_1 | 6.52GB |\n| [gemma-2-9B-it-advanced-v2.1.Q6_K.gguf](https://huggingface.co/RichardErkhov/jsgreenawalt_-_gemma-2-9B-it-advanced-v2.1-gguf/blob/main/gemma-2-9B-it-advanced-v2.1.Q6_K.gguf) | Q6_K | 7.07GB |\n| [gemma-2-9B-it-advanced-v2.1.Q8_0.gguf](https://huggingface.co/RichardErkhov/jsgreenawalt_-_gemma-2-9B-it-advanced-v2.1-gguf/blob/main/gemma-2-9B-it-advanced-v2.1.Q8_0.gguf) | Q8_0 | 9.15GB |\n\n\n\n\nOriginal model description:\n---\nbase_model:\n- wzhouad/gemma-2-9b-it-WPO-HB\n- UCLA-AGI/Gemma-2-9B-It-SPPO-Iter3\n- google/gemma-2-9b-it\n- princeton-nlp/gemma-2-9b-it-SimPO\nlibrary_name: transformers\ntags:\n- mergekit\n- merge\n- merge\n\n---\n# Gemma Advanced V2.1\n\nThis is a merge of the 'smartest' advanced fine-tunes available for Gemma-2-9b-it. It includes WPO, SimPO, and SPPO. The merge was performed via the SOTA 'della' merge method. Merge parameters have been hand-tuned for best results. The Q8_0 quant is highly recommended until better quants come along.\n\n## Notes and observations:\n* The extreme temperature sensitivity from V1 has been fixed, no longer needs to be run at lower temperatures\n* Has a somewhat different writing style than any of the parent models\n* Great instruction following\n* Tracks plot details well and has good situational understanding\n* Seems to have a good understanding of psychology, emotions and creative writing\n* More 'sane' than base gemma-it, SPPO, or SimPO - not as prone to 'Cruella De Vil' or 'Evil Sorceress' like SPPO or SimPO, when portraying characters\n* Would likely serve as a good base for further merges\n* I'm looking for a job, if you're hiring. I'm a skilled Python developer who brings strong devops skills along with an ever-growing knowledge of machine learning pipelines and models. Message me if you want to talk about what I can bring to your team.\n* Overall, this feels like a very useful and successful merge.\n\n## Quantized GGUFs can be found here:\n* [My quants, Q8_0 tested  -  jsgreenawalt/gemma-2-9B-it-advanced-v2.1-GGUF](https://huggingface.co/jsgreenawalt/gemma-2-9B-it-advanced-v2.1-GGUF)\n* [iMatrix  -  mradermacher/gemma-2-9B-it-advanced-v2.1-i1-GGUF](https://huggingface.co/mradermacher/gemma-2-9B-it-advanced-v2.1-i1-GGUF)\n* [QuantFactory/gemma-2-9B-it-advanced-v2.1-GGUF](https://huggingface.co/QuantFactory/gemma-2-9B-it-advanced-v2.1-GGUF)\n* [mradermacher/gemma-2-9B-it-advanced-v2.1-GGUF](https://huggingface.co/mradermacher/gemma-2-9B-it-advanced-v2.1-GGUF)\n\nThanks to everyone who was kind enough to provide quants!\n\nI'll link to other quants as they appear.\n\n# sample ollama Modelfile\n```yaml\nFROM /path/to/file/gemma-2-9B-it-advanced-v2.1-Q8_0.gguf\nPARAMETER stop \"<start_of_turn>\"\nPARAMETER stop \"<end_of_turn>\"\nPARAMETER num_ctx 8192\nTEMPLATE \"\"\"<start_of_turn>user\n{{ if .System }}{{ .System }} {{ end }}{{ .Prompt }}<end_of_turn>\n<start_of_turn>model\n{{ .Response }}<end_of_turn>\"\"\"\n```\n\nThis is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).\n\n## Merge Details\n### Merge Method\n\nThis model was merged using the della merge method using [google/gemma-2-9b-it](https://huggingface.co/google/gemma-2-9b-it) as a base.\n\n### Models Merged\n\nThe following models were included in the merge:\n* [wzhouad/gemma-2-9b-it-WPO-HB](https://huggingface.co/wzhouad/gemma-2-9b-it-WPO-HB)\n* [princeton-nlp/gemma-2-9b-it-SimPO](https://huggingface.co/princeton-nlp/gemma-2-9b-it-SimPO)\n* [UCLA-AGI/Gemma-2-9B-It-SPPO-Iter3](https://huggingface.co/UCLA-AGI/Gemma-2-9B-It-SPPO-Iter3)\n\n### Configuration\n\nThe following YAML configuration was used to produce this model:\n\n```yaml\nmodels:\n  - model: google/gemma-2-9b-it \n  - model: wzhouad/gemma-2-9b-it-WPO-HB\n    parameters:\n      density: 0.55\n      weight: 0.6\n  - model: princeton-nlp/gemma-2-9b-it-SimPO \n    parameters:\n      density: 0.35\n      weight: 0.6\n  - model: UCLA-AGI/Gemma-2-9B-It-SPPO-Iter3\n    parameters:\n      density: 0.25\n      weight: 0.4\nmerge_method: della\nbase_model: google/gemma-2-9b-it\nparameters:\n  normalize: true\n  int8_mask: true\n  lambda: 1.0\n  epsilon: 0.1\ndtype: float16\n\n```\n\n\n",
    "related_quantizations": []
  },
  "tags": [
    "gguf",
    "endpoints_compatible",
    "region:us",
    "conversational"
  ],
  "likes": 4,
  "downloads": 103,
  "gated": false,
  "private": false,
  "last_modified": "2024-09-11T17:01:37.000Z",
  "created_at": "2024-09-11T13:50:10.000Z",
  "pipeline_tag": "",
  "library_name": ""
}

{
  "_id": "66e1a012ac703a318a438219",
  "id": "RichardErkhov/jsgreenawalt_-_gemma-2-9B-it-advanced-v2.1-gguf",
  "modelId": "RichardErkhov/jsgreenawalt_-_gemma-2-9B-it-advanced-v2.1-gguf",
  "sha": "e0e6454a1afc5d50b2caef6106fe6b8f430acf45",
  "createdAt": "2024-09-11T13:50:10.000Z",
  "lastModified": "2024-09-11T17:01:37.000Z",
  "author": "RichardErkhov",
  "downloads": 103,
  "likes": 4,
  "gated": false,
  "private": false,
  "pipeline_tag": "",
  "library_name": "",
  "siblings_count": 24
}