richarderkhov/opendfm_-_chemdfm-13b-v1.0-gguf overview

{
  "metadata": {},
  "card_data": {
    "frontmatter": {},
    "hero_image_url": "https://github.com/OpenDFM/ChemDFM/raw/main/docs/static/images/main.png",
    "summary": "!Main Image ChemDFM is the pioneering open-sourced dialogue foundation model for Chemistry and molecule science, which is build based on LLaMa-13B. ChemDFM outperforms the open-sourced LLMs in all the typical tasks of chemistry, and even reach comparable or higher performances of GPT-4. For more details, please refer to our paper.",
    "quick_links": [],
    "benchmark_table_html": "",
    "readme_markdown": "Quantization made by Richard Erkhov.\n\n[Github](https://github.com/RichardErkhov)\n\n[Discord](https://discord.gg/pvy7H8DZMG)\n\n[Request more models](https://github.com/RichardErkhov/quant_request)\n\n\nChemDFM-13B-v1.0 - GGUF\n- Model creator: https://huggingface.co/OpenDFM/\n- Original model: https://huggingface.co/OpenDFM/ChemDFM-13B-v1.0/\n\n\n| Name | Quant method | Size |\n| ---- | ---- | ---- |\n| [ChemDFM-13B-v1.0.Q2_K.gguf](https://huggingface.co/RichardErkhov/OpenDFM_-_ChemDFM-13B-v1.0-gguf/blob/main/ChemDFM-13B-v1.0.Q2_K.gguf) | Q2_K | 4.56GB |\n| [ChemDFM-13B-v1.0.IQ3_XS.gguf](https://huggingface.co/RichardErkhov/OpenDFM_-_ChemDFM-13B-v1.0-gguf/blob/main/ChemDFM-13B-v1.0.IQ3_XS.gguf) | IQ3_XS | 5.04GB |\n| [ChemDFM-13B-v1.0.IQ3_S.gguf](https://huggingface.co/RichardErkhov/OpenDFM_-_ChemDFM-13B-v1.0-gguf/blob/main/ChemDFM-13B-v1.0.IQ3_S.gguf) | IQ3_S | 5.31GB |\n| [ChemDFM-13B-v1.0.Q3_K_S.gguf](https://huggingface.co/RichardErkhov/OpenDFM_-_ChemDFM-13B-v1.0-gguf/blob/main/ChemDFM-13B-v1.0.Q3_K_S.gguf) | Q3_K_S | 5.31GB |\n| [ChemDFM-13B-v1.0.IQ3_M.gguf](https://huggingface.co/RichardErkhov/OpenDFM_-_ChemDFM-13B-v1.0-gguf/blob/main/ChemDFM-13B-v1.0.IQ3_M.gguf) | IQ3_M | 5.62GB |\n| [ChemDFM-13B-v1.0.Q3_K.gguf](https://huggingface.co/RichardErkhov/OpenDFM_-_ChemDFM-13B-v1.0-gguf/blob/main/ChemDFM-13B-v1.0.Q3_K.gguf) | Q3_K | 5.95GB |\n| [ChemDFM-13B-v1.0.Q3_K_M.gguf](https://huggingface.co/RichardErkhov/OpenDFM_-_ChemDFM-13B-v1.0-gguf/blob/main/ChemDFM-13B-v1.0.Q3_K_M.gguf) | Q3_K_M | 5.95GB |\n| [ChemDFM-13B-v1.0.Q3_K_L.gguf](https://huggingface.co/RichardErkhov/OpenDFM_-_ChemDFM-13B-v1.0-gguf/blob/main/ChemDFM-13B-v1.0.Q3_K_L.gguf) | Q3_K_L | 6.5GB |\n| [ChemDFM-13B-v1.0.IQ4_XS.gguf](https://huggingface.co/RichardErkhov/OpenDFM_-_ChemDFM-13B-v1.0-gguf/blob/main/ChemDFM-13B-v1.0.IQ4_XS.gguf) | IQ4_XS | 6.59GB |\n| [ChemDFM-13B-v1.0.Q4_0.gguf](https://huggingface.co/RichardErkhov/OpenDFM_-_ChemDFM-13B-v1.0-gguf/blob/main/ChemDFM-13B-v1.0.Q4_0.gguf) | Q4_0 | 6.91GB |\n| [ChemDFM-13B-v1.0.IQ4_NL.gguf](https://huggingface.co/RichardErkhov/OpenDFM_-_ChemDFM-13B-v1.0-gguf/blob/main/ChemDFM-13B-v1.0.IQ4_NL.gguf) | IQ4_NL | 6.95GB |\n| [ChemDFM-13B-v1.0.Q4_K_S.gguf](https://huggingface.co/RichardErkhov/OpenDFM_-_ChemDFM-13B-v1.0-gguf/blob/main/ChemDFM-13B-v1.0.Q4_K_S.gguf) | Q4_K_S | 6.96GB |\n| [ChemDFM-13B-v1.0.Q4_K.gguf](https://huggingface.co/RichardErkhov/OpenDFM_-_ChemDFM-13B-v1.0-gguf/blob/main/ChemDFM-13B-v1.0.Q4_K.gguf) | Q4_K | 7.37GB |\n| [ChemDFM-13B-v1.0.Q4_K_M.gguf](https://huggingface.co/RichardErkhov/OpenDFM_-_ChemDFM-13B-v1.0-gguf/blob/main/ChemDFM-13B-v1.0.Q4_K_M.gguf) | Q4_K_M | 7.37GB |\n| [ChemDFM-13B-v1.0.Q4_1.gguf](https://huggingface.co/RichardErkhov/OpenDFM_-_ChemDFM-13B-v1.0-gguf/blob/main/ChemDFM-13B-v1.0.Q4_1.gguf) | Q4_1 | 7.66GB |\n| [ChemDFM-13B-v1.0.Q5_0.gguf](https://huggingface.co/RichardErkhov/OpenDFM_-_ChemDFM-13B-v1.0-gguf/blob/main/ChemDFM-13B-v1.0.Q5_0.gguf) | Q5_0 | 8.41GB |\n| [ChemDFM-13B-v1.0.Q5_K_S.gguf](https://huggingface.co/RichardErkhov/OpenDFM_-_ChemDFM-13B-v1.0-gguf/blob/main/ChemDFM-13B-v1.0.Q5_K_S.gguf) | Q5_K_S | 8.41GB |\n| [ChemDFM-13B-v1.0.Q5_K.gguf](https://huggingface.co/RichardErkhov/OpenDFM_-_ChemDFM-13B-v1.0-gguf/blob/main/ChemDFM-13B-v1.0.Q5_K.gguf) | Q5_K | 8.65GB |\n| [ChemDFM-13B-v1.0.Q5_K_M.gguf](https://huggingface.co/RichardErkhov/OpenDFM_-_ChemDFM-13B-v1.0-gguf/blob/main/ChemDFM-13B-v1.0.Q5_K_M.gguf) | Q5_K_M | 8.65GB |\n| [ChemDFM-13B-v1.0.Q5_1.gguf](https://huggingface.co/RichardErkhov/OpenDFM_-_ChemDFM-13B-v1.0-gguf/blob/main/ChemDFM-13B-v1.0.Q5_1.gguf) | Q5_1 | 9.16GB |\n| [ChemDFM-13B-v1.0.Q6_K.gguf](https://huggingface.co/RichardErkhov/OpenDFM_-_ChemDFM-13B-v1.0-gguf/blob/main/ChemDFM-13B-v1.0.Q6_K.gguf) | Q6_K | 10.0GB |\n| [ChemDFM-13B-v1.0.Q8_0.gguf](https://huggingface.co/RichardErkhov/OpenDFM_-_ChemDFM-13B-v1.0-gguf/blob/main/ChemDFM-13B-v1.0.Q8_0.gguf) | Q8_0 | 12.96GB |\n\n\n\n\nOriginal model description:\n---\nlicense: agpl-3.0\npipeline_tag: text-generation\ntags:\n- chemistry\nlanguage:\n- en\n- zh\n---\n# ChemDFM-13B: An LLM-Based Dialogue Foundation Model for Chemistry\n\n![Main Image](https://github.com/OpenDFM/ChemDFM/raw/main/docs/static/images/main.png)\n\nChemDFM is the pioneering open-sourced dialogue foundation model for Chemistry and molecule science, which is build based on LLaMa-13B. ChemDFM outperforms the open-sourced LLMs in all the typical tasks of chemistry, and even reach comparable or higher performances of GPT-4. For more details, please refer to [our paper](https://arxiv.org/abs/2401.14818).\n\n## News\n\n* **2024-06-13**: The results on the comprehensive science benchmark [SciKnowEval](https://huggingface.co/datasets/hicai-zju/SciKnowEval) show that \"ChemDFM emerged as one of the top open-source models by continuing pre-training and fine-tuning on a vast corpus of scientific literature\".\n* **2024-04-17**: The evaluation data (including instructions) we used in our paper is released on [GitHub](https://github.com/OpenDFM/ChemDFM)\n* **2024-03-12**: The parameter of ChemDFM-13B is open-sourced!\n* **2024-01-26**: The paper of ChemDFM-13B is released on arXiv: [ChemDFM: Dialogue Foundation Model for Chemistry](https://arxiv.org/abs/2401.14818)\n\n## Usage Details\n\nThe online demo of ChemDFM will be up soon!\n\n### local inference\n\nTo load and run ChemDFM locally, here is an example:\n\n```python\nimport torch\nfrom transformers import LlamaTokenizer, LlamaForCausalLM, GenerationConfig\n\nmodel_name_or_id = \"X-LANCE/ChemDFM-13B-v1.0\"\ntokenizer = LlamaTokenizer.from_pretrained(model_name_or_id)\nmodel = LlamaForCausalLM.from_pretrained(model_name_or_id, torch_dtype=torch.float16, device_map=\"auto\")\n\ninput_text = \"Can you please give detailed descriptions of the molecule below?\\nCl.O=C1c2c(O)cccc2-c2nn(CCNCCO)c3ccc(NCCNCCO)c1c23\"\ninput_text = f\"[Round 0]\\nHuman: {input_text}\\nAssistant:\"\n\ninputs = tokenizer(input_text, return_tensors=\"pt\").to(\"cuda\")\ngeneration_config = GenerationConfig(\n    do_sample=True,\n    top_k=20,\n    top_p=0.9,\n    temperature=0.9,\n    max_new_tokens=1024,\n    repetition_penalty=1.05,\n    eos_token_id=tokenizer.eos_token_id\n)\n\noutputs = model.generate(**inputs, generation_config=generation_config)\ngenerated_text = tokenizer.batch_decode(outputs, skip_special_tokens=True)[0][len(input_text):]\nprint(generated_text.strip())\n```\n\n### input format\n\nTo get better responses, we recommend to preprocess your input and history with the dialogue templates which are used during instruction tuning of ChemDFM. Specifically, for an input queries\n```python\n{'current_query': current_query, 'history': [(query1, answer1), (query2, answer2), ...]}\n```\n, you can use the following code to preprocess the input and history:\n```python\ndef formatting_input(current_query, history):\n    input_text = ''\n    for idx, (query, answer) in history:\n        input_text += f\"[Round {idx}]\\nHuman: {query}\\nAssistant: {answer}\\n\"\n    input_text += f\"[Round {len(history)}]\\nHuman: {current_query}\\nAssistant:\"\n    return input_text\n```\n\n### SMILES preprocess\n\nWhen there involves SMILES notation in your input, we recommend to preprocess the SMILES with the `rdkit` package to canonicalize the SMILES. Here is an example:\n```python\nfrom rdkit import Chem\ndef canonicalize_smiles(smiles):\n    mol = Chem.MolFromSmiles(smiles)\n    if mol is None:\n        return None\n    return Chem.MolToSmiles(mol, isomericSmiles=True, kekuleSmiles=False)\n```\nor directly:\n```python\nfrom rdkit import Chem\ndef canonicalize_smiles(smiles):\n    return Chem.CanonSmiles(smiles, useChiral=True)\n```\n\n## Performance\n\n### Chemical Benchmarks\n\nWe evaluate the performance of ChemDFM-13B on multiple widely-used benchmarks in chemistry. The detail introduction of the benchmarks can be found in [our paper](https://arxiv.org/abs/2401.14818). The overall performance of ChemDFM-13B is shown below:\n\n![Objective Performance](https://github.com/OpenDFM/ChemDFM/raw/main/docs/static/images/objective_performances.png) \n\n### Human Evaluation\n\n![Human Evaluation of QA](https://github.com/OpenDFM/ChemDFM/raw/main/docs/static/images/human_evaluation_QA.png)\n\nWe mark <font color=#548235>the correct and relevant information</font> in the replies in green, <font color=#C55A11>the correct but irrelevant information</font> in yellow, and <font color=#C00000>the wrong information in red</font>. In addition, **the key points of the answer** are marked in bold if they appear in the reply. \n\nThe results show that while open-sourced LLMs perform well when asked about existing knowledge (Q1), *only ChemDFM can provide correct and comprehensive answers when questions involve new molecules and reactions* (Q2 [\\[Yin et al., 2023\\]](https://pubs.acs.org/doi/10.1021/jacs.3c07044) & Q3 [\\[Dargo et al., 2023\\]](https://www.sciencedirect.com/science/article/pii/S1385894723030966))\n\n![Human Evaluation of Dialogue](https://github.com/OpenDFM/ChemDFM/raw/main/docs/static/images/human_evaluation_dialogue.png)\n\nThe above conversation is also inspired by [Yin et al.\\[2023\\]](https://pubs.acs.org/doi/10.1021/jacs.3c07044). During the dialogue, the researcher wants to selectively oxidize one of the two carbonyl groups of a molecule. However, the\ninitial solution given by ChemDFM results in both carbonyl groups being oxidized. Through the correction given by the researcher, ChemDFM adjusts its proposal and provides two possible solutions. Finally, the researcher chooses to use protecting groups and ChemDFM further details its advice. *In the dialogue, ChemDFM shows promising capabilities regarding error correction (Round 2) and detailing (Round 3) when handling real-world research scenarios.*\n\n\n**For more examples and analysis, please refer to [our paper](https://arxiv.org/abs/2401.14818).**\n\n## Citation\n```bibtex\n@misc{zhao2024chemdfm,\n    title={ChemDFM: Dialogue Foundation Model for Chemistry},\n    author={Zihan Zhao and Da Ma and Lu Chen and Liangtai Sun and Zihao Li and Hongshen Xu and Zichen Zhu and Su Zhu and Shuai Fan and Guodong Shen and Xin Chen and Kai Yu},\n    year={2024},\n    eprint={2401.14818},\n    archivePrefix={arXiv},\n    primaryClass={cs.CL}\n}\n```\n\n## Disclaimer\nCurrent version of ChemDFM may generate incorrect or misleading information. Please use it with caution and verify the results with domain experts before making any decisions based on the results.\n\n## Contact\n\nIf you have any questions or further requests, please contact [Zihan Zhao](mailto:zhao_mengxin@sjtu.edu.cn) and [Lu Chen](mailto:chenlusz@sjtu.edu.cn).\n\n\n",
    "related_quantizations": []
  },
  "tags": [
    "gguf",
    "arxiv:2401.14818",
    "endpoints_compatible",
    "region:us"
  ],
  "likes": 2,
  "downloads": 107,
  "gated": false,
  "private": false,
  "last_modified": "2024-08-10T07:07:21.000Z",
  "created_at": "2024-08-10T04:29:41.000Z",
  "pipeline_tag": "",
  "library_name": ""
}

{
  "_id": "66b6ecb5a8a5a1978e398447",
  "id": "RichardErkhov/OpenDFM_-_ChemDFM-13B-v1.0-gguf",
  "modelId": "RichardErkhov/OpenDFM_-_ChemDFM-13B-v1.0-gguf",
  "sha": "59e8d357c8b7c4df214c93efbbe54229cf49d5d6",
  "createdAt": "2024-08-10T04:29:41.000Z",
  "lastModified": "2024-08-10T07:07:21.000Z",
  "author": "RichardErkhov",
  "downloads": 107,
  "likes": 2,
  "gated": false,
  "private": false,
  "pipeline_tag": "",
  "library_name": "",
  "siblings_count": 24
}