duyntnet/solar-10.7b-instruct-v1.0-imatrix-gguf overview

{
  "metadata": {},
  "card_data": {
    "license": "other",
    "language": [
      "en"
    ],
    "pipeline_tag": "text-generation",
    "inference": false,
    "tags": [
      "transformers",
      "gguf",
      "imatrix",
      "SOLAR-10.7B-Instruct-v1.0"
    ],
    "frontmatter": {
      "license": "other",
      "language": [
        "en"
      ],
      "pipeline_tag": "text-generation",
      "inference": "false",
      "tags": [
        "transformers",
        "gguf",
        "imatrix",
        "SOLAR-10.7B-Instruct-v1.0"
      ]
    },
    "hero_image_url": "",
    "summary": "# **Usage Instructions** This model has been fine-tuned primarily for single-turn conversation, making it less suitable for multi-turn conversations such as chat. ### **Version** Make sure you have the correct version of the transformers library installed: ``sh pip install transformers==4.35.2 ` ### **Loading the Model** Use the following Python code to load the model: `python import torch from transformers import AutoModelForCausalLM, AutoTokenizer tokenizer = AutoTokenizer.from_pretrained(\"Upstage/SOLAR-10.7B-Instruct-v1.0\") model = AutoModelForCausalLM.from_pretrained( \"Upstage/SOLAR-10.7B-Instruct-v1.0\", device_map=\"auto\", torch_dtype=torch.float16, ) ` ### **Conducting Single-Turn Conversation** `python conversation = [ {'role': 'user', 'content': 'Hello?'} ] prompt = tokenizer.apply_chat_template(conversation, tokenize=False, add_generation_prompt=True) inputs = tokenizer(prompt, return_tensors=\"pt\").to(model.device) outputs = model.generate(**inputs, use_cache=True, max_length=4096) output_text = tokenizer.decode(outputs[0]) print(output_text) ` Below is an example of the output. `  ### User: Hello? ### Assistant: Hello, how can I assist you today? Please feel free to ask any questions or request help with a specific task. ``",
    "quick_links": [],
    "benchmark_table_html": "",
    "readme_markdown": "---\nlicense: other\nlanguage:\n- en\npipeline_tag: text-generation\ninference: false\ntags:\n- transformers\n- gguf\n- imatrix\n- SOLAR-10.7B-Instruct-v1.0\n---\nQuantizations of https://huggingface.co/upstage/SOLAR-10.7B-Instruct-v1.0\n\n# From original readme\n\n# **Usage Instructions**\n\nThis model has been fine-tuned primarily for single-turn conversation, making it less suitable for multi-turn conversations such as chat.\n\n### **Version**\n\nMake sure you have the correct version of the transformers library installed:\n\n```sh\npip install transformers==4.35.2\n```\n\n### **Loading the Model**\n\nUse the following Python code to load the model:\n\n```python\nimport torch\nfrom transformers import AutoModelForCausalLM, AutoTokenizer\n\ntokenizer = AutoTokenizer.from_pretrained(\"Upstage/SOLAR-10.7B-Instruct-v1.0\")\nmodel = AutoModelForCausalLM.from_pretrained(\n    \"Upstage/SOLAR-10.7B-Instruct-v1.0\",\n    device_map=\"auto\",\n    torch_dtype=torch.float16,\n)\n```\n\n### **Conducting Single-Turn Conversation**\n\n```python\nconversation = [ {'role': 'user', 'content': 'Hello?'} ] \n\nprompt = tokenizer.apply_chat_template(conversation, tokenize=False, add_generation_prompt=True)\n\ninputs = tokenizer(prompt, return_tensors=\"pt\").to(model.device) \noutputs = model.generate(**inputs, use_cache=True, max_length=4096)\noutput_text = tokenizer.decode(outputs[0]) \nprint(output_text)\n```\n\nBelow is an example of the output.\n```\n<s> ### User:\nHello?\n\n### Assistant:\nHello, how can I assist you today? Please feel free to ask any questions or request help with a specific task.</s>\n```",
    "related_quantizations": []
  },
  "tags": [
    "transformers",
    "gguf",
    "imatrix",
    "SOLAR-10.7B-Instruct-v1.0",
    "text-generation",
    "en",
    "license:other",
    "region:us",
    "conversational"
  ],
  "likes": 0,
  "downloads": 98,
  "gated": false,
  "private": false,
  "last_modified": "2024-05-04T14:30:03.000Z",
  "created_at": "2024-05-04T11:46:39.000Z",
  "pipeline_tag": "text-generation",
  "library_name": "transformers"
}

{
  "_id": "6636201f20663edd59c71675",
  "id": "duyntnet/SOLAR-10.7B-Instruct-v1.0-imatrix-GGUF",
  "modelId": "duyntnet/SOLAR-10.7B-Instruct-v1.0-imatrix-GGUF",
  "sha": "ea19334578bb3989499c8582a6f055ee33e60a59",
  "createdAt": "2024-05-04T11:46:39.000Z",
  "lastModified": "2024-05-04T14:30:03.000Z",
  "author": "duyntnet",
  "downloads": 98,
  "likes": 0,
  "gated": false,
  "private": false,
  "pipeline_tag": "text-generation",
  "library_name": "transformers",
  "siblings_count": 29
}