maziyarpanahi/calme-2.1-qwen2-72b-gguf overview

{
  "metadata": {},
  "card_data": {
    "pipeline_tag": "text-generation",
    "tags": [
      "qwen",
      "qwen-2",
      "quantized",
      "2-bit",
      "3-bit",
      "4-bit",
      "5-bit",
      "6-bit",
      "8-bit",
      "16-bit",
      "GGUF"
    ],
    "inference": false,
    "model_creator": "MaziyarPanahi",
    "model_name": "calme-2.1-qwen2-72b-GGUF",
    "quantized_by": "MaziyarPanahi",
    "license": "other",
    "license_name": "tongyi-qianwen",
    "license_link": "https://huggingface.co/Qwen/Qwen2-72B-Instruct/blob/main/LICENSE",
    "frontmatter": {
      "pipeline_tag": "text-generation",
      "tags": [
        "qwen",
        "qwen-2",
        "quantized",
        "2-bit",
        "3-bit",
        "4-bit",
        "5-bit",
        "6-bit",
        "8-bit",
        "16-bit",
        "GGUF"
      ],
      "inference": "false",
      "model_creator": "MaziyarPanahi",
      "model_name": "calme-2.1-qwen2-72b-GGUF",
      "quantized_by": "MaziyarPanahi",
      "license": "other",
      "license_name": "tongyi-qianwen",
      "license_link": "https://huggingface.co/Qwen/Qwen2-72B-Instruct/blob/main/LICENSE"
    },
    "hero_image_url": "",
    "summary": "The GGUF and quantized models here are based on MaziyarPanahi/calme-2.1-qwen2-72b model",
    "quick_links": [],
    "benchmark_table_html": "",
    "readme_markdown": "---\npipeline_tag: text-generation\ntags:\n- qwen\n- qwen-2\n- quantized\n- 2-bit\n- 3-bit\n- 4-bit\n- 5-bit\n- 6-bit\n- 8-bit\n- 16-bit\n- GGUF\ninference: false\nmodel_creator: MaziyarPanahi\nmodel_name: calme-2.1-qwen2-72b-GGUF\nquantized_by: MaziyarPanahi\nlicense: other\nlicense_name: tongyi-qianwen\nlicense_link: https://huggingface.co/Qwen/Qwen2-72B-Instruct/blob/main/LICENSE\n---\n\n\n# MaziyarPanahi/calme-2.1-qwen2-72b-GGUF\n\nThe GGUF and quantized models here are based on [MaziyarPanahi/calme-2.1-qwen2-72b](https://huggingface.co/MaziyarPanahi/calme-2.1-qwen2-72b) model\n\n## How to download\nYou can download only the quants you need instead of cloning the entire repository as follows:\n\n```\nhuggingface-cli download MaziyarPanahi/calme-2.1-qwen2-72b-GGUF --local-dir . --include '*Q2_K*gguf'\n```\n\n## Load GGUF models\n\n\n```sh\n./llama.cpp/main -m mode_name.Q2_K.gguf -p \"<|im_start|>user\\nJust say 1, 2, 3 hi and NOTHING else\\n<|im_end|>\\n<|im_start|>assistant\\n\" -n 1024\n```\n\n\n\n\n## Original README\n\n---\n\n# MaziyarPanahi/calme-2.1-qwen2-72b\n\nThis is a fine-tuned version of the `Qwen/Qwen2-72B-Instruct` model. It aims to improve the base model across all benchmarks.\n\n# ⚡ Quantized GGUF\n\nAll GGUF models are available here: [MaziyarPanahi/calme-2.1-qwen2-72b-GGUF](https://huggingface.co/MaziyarPanahi/calme-2.1-qwen2-72b-GGUF)\n\n# 🏆 [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)\n\n\n\n|    Tasks     |Version|Filter|n-shot|Metric|Value |   |Stderr|\n|--------------|------:|------|-----:|------|-----:|---|-----:|\n|truthfulqa_mc2|      2|none  |     0|acc   |0.6761|±  |0.0148|\n\n|  Tasks   |Version|Filter|n-shot|Metric|Value |   |Stderr|\n|----------|------:|------|-----:|------|-----:|---|-----:|\n|winogrande|      1|none  |     5|acc   |0.8248|±  |0.0107|\n\n|    Tasks    |Version|Filter|n-shot| Metric |Value |   |Stderr|\n|-------------|------:|------|-----:|--------|-----:|---|-----:|\n|arc_challenge|      1|none  |    25|acc     |0.6852|±  |0.0136|\n|             |       |none  |    25|acc_norm|0.7184|±  |0.0131|\n\n|Tasks|Version|     Filter     |n-shot|  Metric   |Value |   |Stderr|\n|-----|------:|----------------|-----:|-----------|-----:|---|-----:|\n|gsm8k|      3|strict-match    |     5|exact_match|0.8582|±  |0.0096|\n|     |       |flexible-extract|     5|exact_match|0.8893|±  |0.0086|\n\n# Prompt Template\n\nThis model uses `ChatML` prompt template:\n\n```\n<|im_start|>system\n{System}\n<|im_end|>\n<|im_start|>user\n{User}\n<|im_end|>\n<|im_start|>assistant\n{Assistant}\n````\n\n# How to use\n\n\n```python\n\n# Use a pipeline as a high-level helper\n\nfrom transformers import pipeline\n\nmessages = [\n    {\"role\": \"user\", \"content\": \"Who are you?\"},\n]\npipe = pipeline(\"text-generation\", model=\"MaziyarPanahi/calme-2.1-qwen2-72b\")\npipe(messages)\n\n\n# Load model directly\n\nfrom transformers import AutoTokenizer, AutoModelForCausalLM\n\ntokenizer = AutoTokenizer.from_pretrained(\"MaziyarPanahi/calme-2.1-qwen2-72b\")\nmodel = AutoModelForCausalLM.from_pretrained(\"MaziyarPanahi/calme-2.1-qwen2-72b\")\n```\n",
    "related_quantizations": []
  },
  "tags": [
    "gguf",
    "qwen",
    "qwen-2",
    "quantized",
    "2-bit",
    "3-bit",
    "4-bit",
    "5-bit",
    "6-bit",
    "8-bit",
    "16-bit",
    "GGUF",
    "text-generation",
    "license:other",
    "region:us",
    "imatrix",
    "conversational"
  ],
  "likes": 13,
  "downloads": 139,
  "gated": false,
  "private": false,
  "last_modified": "2024-08-02T03:56:57.000Z",
  "created_at": "2024-06-27T00:43:46.000Z",
  "pipeline_tag": "text-generation",
  "library_name": ""
}

{
  "_id": "667cb5c2dc3fa2e006130eb7",
  "id": "MaziyarPanahi/calme-2.1-qwen2-72b-GGUF",
  "modelId": "MaziyarPanahi/calme-2.1-qwen2-72b-GGUF",
  "sha": "47ece891a30143f32a40f4ec2360ccf5df911f14",
  "createdAt": "2024-06-27T00:43:46.000Z",
  "lastModified": "2024-08-02T03:56:57.000Z",
  "author": "MaziyarPanahi",
  "downloads": 139,
  "likes": 13,
  "gated": false,
  "private": false,
  "pipeline_tag": "text-generation",
  "library_name": "",
  "siblings_count": 123
}