quantfactory/turkish-llama-8b-dpo-v0.1-gguf overview

{
  "metadata": {},
  "card_data": {
    "license": "llama3",
    "language": [
      "tr"
    ],
    "pipeline_tag": "text-generation",
    "base_model": "meta-llama/Meta-Llama-3-8B",
    "tags": [
      "Turkish",
      "turkish",
      "Llama",
      "Llama3"
    ],
    "frontmatter": {},
    "hero_image_url": "./cosmosLLaMa2_r2.png",
    "summary": "This is quantized version of ytu-ce-cosmos/Turkish-Llama-8b-DPO-v0.1 created using llama.cpp # Original Model Card  # Cosmos LLaMa Instruct-DPO This is the newest and the most advanced iteration of CosmosLLama. The model has been developed by merging two distinctly trained CosmosLLaMa-Instruct DPO models. The CosmosLLaMa-Instruct DPO is designed for text generation tasks, providing the ability to continue a given text snippet in a coherent and contextually relevant manner. Due to the diverse nature of the training data, which includes websites, books, and other text sources, this model can exhibit biases. Users should be aware of these biases and use the model responsibly. You can easily demo the model from here: https://cosmos.yildiz.edu.tr/cosmosllama #### Transformers pipeline ``python import transformers import torch model_id = \"ytu-ce-cosmos/Turkish-Llama-8b-DPO-v0.1\" pipeline = transformers.pipeline( \"text-generation\", model=model_id, model_kwargs={\"torch_dtype\": torch.bfloat16}, device_map=\"auto\", ) messages = [ {\"role\": \"system\", \"content\": \"Sen bir yapay zeka asistanısın. Kullanıcı sana bir görev verecek. Amacın görevi olabildiğince sadık bir şekilde tamamlamak. Görevi yerine getirirken adım adım düşün ve adımlarını gerekçelendir.\"}, {\"role\": \"user\", \"content\": \"Soru: Bir arabanın deposu 60 litre benzin alabiliyor. Araba her 100 kilometrede 8 litre benzin tüketiyor. Depo tamamen doluyken araba kaç kilometre yol alabilir?\"}, ] terminators = [ pipeline.tokenizer.eos_token_id, pipeline.tokenizer.convert_tokens_to_ids(\"\") ] outputs = pipeline( messages, max_new_tokens=256, eos_token_id=terminators, do_sample=True, temperature=0.6, top_p=0.9, ) print(outputs[0][\"generated_text\"][-1]) ` #### Transformers AutoModelForCausalLM `python from transformers import AutoTokenizer, AutoModelForCausalLM import torch model_id = \"ytu-ce-cosmos/Turkish-Llama-8b-DPO-v0.1\" tokenizer = AutoTokenizer.from_pretrained(model_id) model = AutoModelForCausalLM.from_pretrained( model_id, torch_dtype=torch.bfloat16, device_map=\"auto\", ) messages = [ {\"role\": \"system\", \"content\": \"Sen bir yapay zeka asistanısın. Kullanıcı sana bir görev verecek. Amacın görevi olabildiğince sadık bir şekilde tamamlamak. Görevi yerine getirirken adım adım düşün ve adımlarını gerekçelendir.\"}, {\"role\": \"user\", \"content\": \"Soru: Bir arabanın deposu 60 litre benzin alabiliyor. Araba her 100 kilometrede 8 litre benzin tüketiyor. Depo tamamen doluyken araba kaç kilometre yol alabilir?\"}, ] input_ids = tokenizer.apply_chat_template( messages, add_generation_prompt=True, return_tensors=\"pt\" ).to(model.device) terminators = [ tokenizer.eos_token_id, tokenizer.convert_tokens_to_ids(\"\") ] outputs = model.generate( input_ids, max_new_tokens=256, eos_token_id=terminators, do_sample=True, temperature=0.6, top_p=0.9, ) response = outputs[0][input_ids.shape[-1]:] print(tokenizer.decode(response, skip_special_tokens=True)) `` # Acknowledgments 1018512024 ### Contact COSMOS AI Research Group, Yildiz Technical University Computer Engineering Department  https://cosmos.yildiz.edu.tr/  cosmos@yildiz.edu.tr --- license: llama3 ---",
    "quick_links": [],
    "benchmark_table_html": "",
    "readme_markdown": "\n---\n\nlicense: llama3\nlanguage:\n- tr\npipeline_tag: text-generation\nbase_model: meta-llama/Meta-Llama-3-8B\ntags:\n- Turkish\n- turkish\n- Llama\n- Llama3\n\n---\n\n[![QuantFactory Banner](https://lh7-rt.googleusercontent.com/docsz/AD_4nXeiuCm7c8lEwEJuRey9kiVZsRn2W-b4pWlu3-X534V3YmVuVc2ZL-NXg2RkzSOOS2JXGHutDuyyNAUtdJI65jGTo8jT9Y99tMi4H4MqL44Uc5QKG77B0d6-JfIkZHFaUA71-RtjyYZWVIhqsNZcx8-OMaA?key=xt3VSDoCbmTY7o-cwwOFwQ)](https://hf.co/QuantFactory)\n\n\n# QuantFactory/Turkish-Llama-8b-DPO-v0.1-GGUF\nThis is quantized version of [ytu-ce-cosmos/Turkish-Llama-8b-DPO-v0.1](https://huggingface.co/ytu-ce-cosmos/Turkish-Llama-8b-DPO-v0.1) created using llama.cpp\n\n# Original Model Card\n\n\n<img src=\"./cosmosLLaMa2_r2.png\"/>\n\n\n# Cosmos LLaMa Instruct-DPO\n\nThis is the newest and the most advanced iteration of CosmosLLama. The model has been developed by merging two distinctly trained CosmosLLaMa-Instruct DPO models.\n\nThe CosmosLLaMa-Instruct DPO is designed for text generation tasks, providing the ability to continue a given text snippet in a coherent and contextually relevant manner. Due to the diverse nature of the training data, which includes websites, books, and other text sources, this model can exhibit biases. Users should be aware of these biases and use the model responsibly.\n\nYou can easily demo the model from here: https://cosmos.yildiz.edu.tr/cosmosllama\n\n#### Transformers pipeline\n\n```python\nimport transformers\nimport torch\n\nmodel_id = \"ytu-ce-cosmos/Turkish-Llama-8b-DPO-v0.1\"\n\npipeline = transformers.pipeline(\n    \"text-generation\",\n    model=model_id,\n    model_kwargs={\"torch_dtype\": torch.bfloat16},\n    device_map=\"auto\",\n)\n\nmessages = [\n    {\"role\": \"system\", \"content\": \"Sen bir yapay zeka asistanısın. Kullanıcı sana bir görev verecek. Amacın görevi olabildiğince sadık bir şekilde tamamlamak. Görevi yerine getirirken adım adım düşün ve adımlarını gerekçelendir.\"},\n    {\"role\": \"user\", \"content\": \"Soru: Bir arabanın deposu 60 litre benzin alabiliyor. Araba her 100 kilometrede 8 litre benzin tüketiyor. Depo tamamen doluyken araba kaç kilometre yol alabilir?\"},\n]\n\nterminators = [\n    pipeline.tokenizer.eos_token_id,\n    pipeline.tokenizer.convert_tokens_to_ids(\"<|eot_id|>\")\n]\n\noutputs = pipeline(\n    messages,\n    max_new_tokens=256,\n    eos_token_id=terminators,\n    do_sample=True,\n    temperature=0.6,\n    top_p=0.9,\n)\nprint(outputs[0][\"generated_text\"][-1])\n```\n\n#### Transformers AutoModelForCausalLM\n\n```python\nfrom transformers import AutoTokenizer, AutoModelForCausalLM\nimport torch\n\nmodel_id = \"ytu-ce-cosmos/Turkish-Llama-8b-DPO-v0.1\"\n\ntokenizer = AutoTokenizer.from_pretrained(model_id)\nmodel = AutoModelForCausalLM.from_pretrained(\n    model_id,\n    torch_dtype=torch.bfloat16,\n    device_map=\"auto\",\n)\n\nmessages = [\n    {\"role\": \"system\", \"content\": \"Sen bir yapay zeka asistanısın. Kullanıcı sana bir görev verecek. Amacın görevi olabildiğince sadık bir şekilde tamamlamak. Görevi yerine getirirken adım adım düşün ve adımlarını gerekçelendir.\"},\n    {\"role\": \"user\", \"content\": \"Soru: Bir arabanın deposu 60 litre benzin alabiliyor. Araba her 100 kilometrede 8 litre benzin tüketiyor. Depo tamamen doluyken araba kaç kilometre yol alabilir?\"},\n]\n\ninput_ids = tokenizer.apply_chat_template(\n    messages,\n    add_generation_prompt=True,\n    return_tensors=\"pt\"\n).to(model.device)\n\nterminators = [\n    tokenizer.eos_token_id,\n    tokenizer.convert_tokens_to_ids(\"<|eot_id|>\")\n]\n\noutputs = model.generate(\n    input_ids,\n    max_new_tokens=256,\n    eos_token_id=terminators,\n    do_sample=True,\n    temperature=0.6,\n    top_p=0.9,\n)\nresponse = outputs[0][input_ids.shape[-1]:]\nprint(tokenizer.decode(response, skip_special_tokens=True))\n```\n\n\n# Acknowledgments\n- Thanks to the generous support from the Hugging Face team, it is possible to download models from their S3 storage 🤗\n- Computing resources used in this work were provided by the National Center for High Performance Computing of Turkey (UHeM) under grant numbers 1016912023 and \n1018512024\n- Research supported with Cloud TPUs from Google's TPU Research Cloud (TRC)\n\n### Contact\nCOSMOS AI Research Group, Yildiz Technical University Computer Engineering Department <br>\nhttps://cosmos.yildiz.edu.tr/ <br>\ncosmos@yildiz.edu.tr\n\n---\nlicense: llama3\n---\n",
    "related_quantizations": []
  },
  "tags": [
    "gguf",
    "Turkish",
    "turkish",
    "Llama",
    "Llama3",
    "text-generation",
    "tr",
    "base_model:meta-llama/Meta-Llama-3-8B",
    "base_model:quantized:meta-llama/Meta-Llama-3-8B",
    "license:llama3",
    "endpoints_compatible",
    "region:us",
    "conversational"
  ],
  "likes": 5,
  "downloads": 206,
  "gated": false,
  "private": false,
  "last_modified": "2024-09-12T04:24:19.000Z",
  "created_at": "2024-09-12T01:51:48.000Z",
  "pipeline_tag": "text-generation",
  "library_name": ""
}

{
  "_id": "66e24934cf6d4b77197ec35b",
  "id": "QuantFactory/Turkish-Llama-8b-DPO-v0.1-GGUF",
  "modelId": "QuantFactory/Turkish-Llama-8b-DPO-v0.1-GGUF",
  "sha": "72f38e6c26cb84fb2962655d57d0777e2bc3ce4c",
  "createdAt": "2024-09-12T01:51:48.000Z",
  "lastModified": "2024-09-12T04:24:19.000Z",
  "author": "QuantFactory",
  "downloads": 206,
  "likes": 5,
  "gated": false,
  "private": false,
  "pipeline_tag": "text-generation",
  "library_name": "",
  "siblings_count": 16
}