richarderkhov/ntqai_-_nxcode-cq-7b-orpo-gguf overview

{
  "metadata": {},
  "card_data": {
    "frontmatter": {},
    "hero_image_url": "https://cdn-uploads.huggingface.co/production/uploads/5ee1b417636bdb3834e2da19/etbfTJuVdAub2evNP_E4g.png",
    "summary": "Quantization made by Richard Erkhov. Github Discord Request more models Nxcode-CQ-7B-orpo - GGUF | Name | Quant method | Size | | ---- | ---- | ---- | | Nxcode-CQ-7B-orpo.Q2_K.gguf | Q2_K | 2.84GB | | Nxcode-CQ-7B-orpo.IQ3_XS.gguf | IQ3_XS | 3.13GB | | Nxcode-CQ-7B-orpo.IQ3_S.gguf | IQ3_S | 3.27GB | | Nxcode-CQ-7B-orpo.Q3_K_S.gguf | Q3_K_S | 3.26GB | | Nxcode-CQ-7B-orpo.IQ3_M.gguf | IQ3_M | 3.36GB | | Nxcode-CQ-7B-orpo.Q3_K.gguf | Q3_K | 3.55GB | | Nxcode-CQ-7B-orpo.Q3_K_M.gguf | Q3_K_M | 3.55GB | | Nxcode-CQ-7B-orpo.Q3_K_L.gguf | Q3_K_L | 3.71GB | | Nxcode-CQ-7B-orpo.IQ4_XS.gguf | IQ4_XS | 3.79GB | | Nxcode-CQ-7B-orpo.Q4_0.gguf | Q4_0 | 3.89GB | | Nxcode-CQ-7B-orpo.IQ4_NL.gguf | IQ4_NL | 3.94GB | | Nxcode-CQ-7B-orpo.Q4_K_S.gguf | Q4_K_S | 4.11GB | | Nxcode-CQ-7B-orpo.Q4_K.gguf | Q4_K | 4.41GB | | Nxcode-CQ-7B-orpo.Q4_K_M.gguf | Q4_K_M | 4.41GB | | Nxcode-CQ-7B-orpo.Q4_1.gguf | Q4_1 | 4.29GB | | Nxcode-CQ-7B-orpo.Q5_0.gguf | Q5_0 | 4.69GB | | Nxcode-CQ-7B-orpo.Q5_K_S.gguf | Q5_K_S | 4.79GB | | Nxcode-CQ-7B-orpo.Q5_K.gguf | Q5_K | 5.06GB | | Nxcode-CQ-7B-orpo.Q5_K_M.gguf | Q5_K_M | 5.06GB | | Nxcode-CQ-7B-orpo.Q5_1.gguf | Q5_1 | 5.09GB | | Nxcode-CQ-7B-orpo.Q6_K.gguf | Q6_K | 5.94GB | | Nxcode-CQ-7B-orpo.Q8_0.gguf | Q8_0 | 7.18GB | Original model description: --- license_name: tongyi-qianwen-research license_link: https://huggingface.co/Qwen/CodeQwen1.5-7B/blob/main/LICENSE tags: pipeline_tag: text-generation license: other ---",
    "quick_links": [],
    "benchmark_table_html": "",
    "readme_markdown": "Quantization made by Richard Erkhov.\n\n[Github](https://github.com/RichardErkhov)\n\n[Discord](https://discord.gg/pvy7H8DZMG)\n\n[Request more models](https://github.com/RichardErkhov/quant_request)\n\n\nNxcode-CQ-7B-orpo - GGUF\n- Model creator: https://huggingface.co/NTQAI/\n- Original model: https://huggingface.co/NTQAI/Nxcode-CQ-7B-orpo/\n\n\n| Name | Quant method | Size |\n| ---- | ---- | ---- |\n| [Nxcode-CQ-7B-orpo.Q2_K.gguf](https://huggingface.co/RichardErkhov/NTQAI_-_Nxcode-CQ-7B-orpo-gguf/blob/main/Nxcode-CQ-7B-orpo.Q2_K.gguf) | Q2_K | 2.84GB |\n| [Nxcode-CQ-7B-orpo.IQ3_XS.gguf](https://huggingface.co/RichardErkhov/NTQAI_-_Nxcode-CQ-7B-orpo-gguf/blob/main/Nxcode-CQ-7B-orpo.IQ3_XS.gguf) | IQ3_XS | 3.13GB |\n| [Nxcode-CQ-7B-orpo.IQ3_S.gguf](https://huggingface.co/RichardErkhov/NTQAI_-_Nxcode-CQ-7B-orpo-gguf/blob/main/Nxcode-CQ-7B-orpo.IQ3_S.gguf) | IQ3_S | 3.27GB |\n| [Nxcode-CQ-7B-orpo.Q3_K_S.gguf](https://huggingface.co/RichardErkhov/NTQAI_-_Nxcode-CQ-7B-orpo-gguf/blob/main/Nxcode-CQ-7B-orpo.Q3_K_S.gguf) | Q3_K_S | 3.26GB |\n| [Nxcode-CQ-7B-orpo.IQ3_M.gguf](https://huggingface.co/RichardErkhov/NTQAI_-_Nxcode-CQ-7B-orpo-gguf/blob/main/Nxcode-CQ-7B-orpo.IQ3_M.gguf) | IQ3_M | 3.36GB |\n| [Nxcode-CQ-7B-orpo.Q3_K.gguf](https://huggingface.co/RichardErkhov/NTQAI_-_Nxcode-CQ-7B-orpo-gguf/blob/main/Nxcode-CQ-7B-orpo.Q3_K.gguf) | Q3_K | 3.55GB |\n| [Nxcode-CQ-7B-orpo.Q3_K_M.gguf](https://huggingface.co/RichardErkhov/NTQAI_-_Nxcode-CQ-7B-orpo-gguf/blob/main/Nxcode-CQ-7B-orpo.Q3_K_M.gguf) | Q3_K_M | 3.55GB |\n| [Nxcode-CQ-7B-orpo.Q3_K_L.gguf](https://huggingface.co/RichardErkhov/NTQAI_-_Nxcode-CQ-7B-orpo-gguf/blob/main/Nxcode-CQ-7B-orpo.Q3_K_L.gguf) | Q3_K_L | 3.71GB |\n| [Nxcode-CQ-7B-orpo.IQ4_XS.gguf](https://huggingface.co/RichardErkhov/NTQAI_-_Nxcode-CQ-7B-orpo-gguf/blob/main/Nxcode-CQ-7B-orpo.IQ4_XS.gguf) | IQ4_XS | 3.79GB |\n| [Nxcode-CQ-7B-orpo.Q4_0.gguf](https://huggingface.co/RichardErkhov/NTQAI_-_Nxcode-CQ-7B-orpo-gguf/blob/main/Nxcode-CQ-7B-orpo.Q4_0.gguf) | Q4_0 | 3.89GB |\n| [Nxcode-CQ-7B-orpo.IQ4_NL.gguf](https://huggingface.co/RichardErkhov/NTQAI_-_Nxcode-CQ-7B-orpo-gguf/blob/main/Nxcode-CQ-7B-orpo.IQ4_NL.gguf) | IQ4_NL | 3.94GB |\n| [Nxcode-CQ-7B-orpo.Q4_K_S.gguf](https://huggingface.co/RichardErkhov/NTQAI_-_Nxcode-CQ-7B-orpo-gguf/blob/main/Nxcode-CQ-7B-orpo.Q4_K_S.gguf) | Q4_K_S | 4.11GB |\n| [Nxcode-CQ-7B-orpo.Q4_K.gguf](https://huggingface.co/RichardErkhov/NTQAI_-_Nxcode-CQ-7B-orpo-gguf/blob/main/Nxcode-CQ-7B-orpo.Q4_K.gguf) | Q4_K | 4.41GB |\n| [Nxcode-CQ-7B-orpo.Q4_K_M.gguf](https://huggingface.co/RichardErkhov/NTQAI_-_Nxcode-CQ-7B-orpo-gguf/blob/main/Nxcode-CQ-7B-orpo.Q4_K_M.gguf) | Q4_K_M | 4.41GB |\n| [Nxcode-CQ-7B-orpo.Q4_1.gguf](https://huggingface.co/RichardErkhov/NTQAI_-_Nxcode-CQ-7B-orpo-gguf/blob/main/Nxcode-CQ-7B-orpo.Q4_1.gguf) | Q4_1 | 4.29GB |\n| [Nxcode-CQ-7B-orpo.Q5_0.gguf](https://huggingface.co/RichardErkhov/NTQAI_-_Nxcode-CQ-7B-orpo-gguf/blob/main/Nxcode-CQ-7B-orpo.Q5_0.gguf) | Q5_0 | 4.69GB |\n| [Nxcode-CQ-7B-orpo.Q5_K_S.gguf](https://huggingface.co/RichardErkhov/NTQAI_-_Nxcode-CQ-7B-orpo-gguf/blob/main/Nxcode-CQ-7B-orpo.Q5_K_S.gguf) | Q5_K_S | 4.79GB |\n| [Nxcode-CQ-7B-orpo.Q5_K.gguf](https://huggingface.co/RichardErkhov/NTQAI_-_Nxcode-CQ-7B-orpo-gguf/blob/main/Nxcode-CQ-7B-orpo.Q5_K.gguf) | Q5_K | 5.06GB |\n| [Nxcode-CQ-7B-orpo.Q5_K_M.gguf](https://huggingface.co/RichardErkhov/NTQAI_-_Nxcode-CQ-7B-orpo-gguf/blob/main/Nxcode-CQ-7B-orpo.Q5_K_M.gguf) | Q5_K_M | 5.06GB |\n| [Nxcode-CQ-7B-orpo.Q5_1.gguf](https://huggingface.co/RichardErkhov/NTQAI_-_Nxcode-CQ-7B-orpo-gguf/blob/main/Nxcode-CQ-7B-orpo.Q5_1.gguf) | Q5_1 | 5.09GB |\n| [Nxcode-CQ-7B-orpo.Q6_K.gguf](https://huggingface.co/RichardErkhov/NTQAI_-_Nxcode-CQ-7B-orpo-gguf/blob/main/Nxcode-CQ-7B-orpo.Q6_K.gguf) | Q6_K | 5.94GB |\n| [Nxcode-CQ-7B-orpo.Q8_0.gguf](https://huggingface.co/RichardErkhov/NTQAI_-_Nxcode-CQ-7B-orpo-gguf/blob/main/Nxcode-CQ-7B-orpo.Q8_0.gguf) | Q8_0 | 7.18GB |\n\n\n\n\nOriginal model description:\n---\nlicense_name: tongyi-qianwen-research\nlicense_link: https://huggingface.co/Qwen/CodeQwen1.5-7B/blob/main/LICENSE\ntags:\n- code\npipeline_tag: text-generation\nlicense: other\n---\n\n<a href=\"https://ntq.com.vn\" target=\"_blank\"><img src=\"https://cdn-uploads.huggingface.co/production/uploads/5ee1b417636bdb3834e2da19/etbfTJuVdAub2evNP_E4g.png\" width=\"200\"/></a>\n\n## Introduction\n\nNxcode-CQ-7B-orpo is an [Monolithic Preference Optimization without Reference Model](https://arxiv.org/abs/2403.07691) fine-tune of Qwen/CodeQwen1.5-7B on 100k samples of high-quality ranking data.\n\n## [Evalplus](https://github.com/evalplus/evalplus)\n\n| EvalPlus | pass@1 |\n| --- | --- |\n| HumanEval | 86.6 |\n| HumanEval+ | 83.5 |\n| MBPP(v0.2.0) | 82.3 |\n| MBPP+(v0.2.0) | 70.4 |\n\nWe use a simple template to generate the solution for evalplus:\n\n```python\n\"Complete the following Python function:\\n{prompt}\"\n```\n\n[Evalplus Leaderboard](https://evalplus.github.io/leaderboard.html)\n| Models | HumanEval | HumanEval+|\n|------ | ------  | ------ |\n| GPT-4-Turbo (April 2024)|  90.2| 86.6|\n| GPT-4 (May 2023)|  88.4| 81.17|\n| GPT-4-Turbo (Nov 2023)|  85.4| 79.3| \n| CodeQwen1.5-7B-Chat|  83.5| 78.7| \n| claude-3-opus (Mar 2024)|  82.9| 76.8|\n| DeepSeek-Coder-33B-instruct|  81.1| 75.0|\n| WizardCoder-33B-V1.1|  79.9| 73.2|\n| OpenCodeInterpreter-DS-33B|  79.3| 73.8|\n| speechless-codellama-34B-v2.0|  77.4| 72|\n| GPT-3.5-Turbo (Nov 2023)| 76.8| 70.7|\n| Llama3-70B-instruct| 76.2| 70.7|\n\n## Bigcode Leaderboard\n\n[Bigcode Leaderboard](https://huggingface.co/spaces/bigcode/bigcode-models-leaderboard)\n\n**09/05/2024**\n\nTop 1 average score.\n\nTop 2 winrate.\n\n![image/png](https://cdn-uploads.huggingface.co/production/uploads/5ee1b417636bdb3834e2da19/OQonD6a7aNjnN9SsTkFp-.png)\n\n\n## Quickstart\n\nHere provides a code snippet with `apply_chat_template` to show you how to load the tokenizer and model and how to generate contents. You should upgrade the transformers if you receive an error when loading the tokenizer\n```python\nfrom transformers import AutoModelForCausalLM, AutoTokenizer\ndevice = \"cuda\" # the device to load the model onto\n\nmodel = AutoModelForCausalLM.from_pretrained(\n    \"NTQAI/Nxcode-CQ-7B-orpo\",\n    torch_dtype=\"auto\",\n    device_map=\"auto\"\n)\ntokenizer = AutoTokenizer.from_pretrained(\"NTQAI/Nxcode-CQ-7B-orpo\")\n\nprompt = \"\"\"Complete the following Python function:\nfrom typing import List\n\n\ndef has_close_elements(numbers: List[float], threshold: float) -> bool:\n    \"\"\" Check if in given list of numbers, are any two numbers closer to each other than\n    given threshold.\n    >>> has_close_elements([1.0, 2.0, 3.0], 0.5)\n    False\n    >>> has_close_elements([1.0, 2.8, 3.0, 4.0, 5.0, 2.0], 0.3)\n    True\n    \"\"\"\n\"\"\"\nmessages = [\n    {\"role\": \"user\", \"content\": prompt}\n]\n\ninputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors=\"pt\").to(model.device)\noutputs = model.generate(inputs, max_new_tokens=512, do_sample=False, top_k=50, top_p=0.95, num_return_sequences=1, eos_token_id=tokenizer.eos_token_id)\nres = tokenizer.decode(outputs[0][len(inputs[0]):], skip_special_tokens=True)\n\n```\n\n### Contact information\nFor personal communication related to this project, please contact Nha Nguyen Van (nha.nguyen@ntq-solution.com.vn).\n\n",
    "related_quantizations": []
  },
  "tags": [
    "gguf",
    "arxiv:2403.07691",
    "endpoints_compatible",
    "region:us",
    "conversational"
  ],
  "likes": 0,
  "downloads": 988,
  "gated": false,
  "private": false,
  "last_modified": "2024-06-25T21:33:39.000Z",
  "created_at": "2024-06-25T17:27:51.000Z",
  "pipeline_tag": "",
  "library_name": ""
}

{
  "_id": "667afe174ae2385fd1af23a7",
  "id": "RichardErkhov/NTQAI_-_Nxcode-CQ-7B-orpo-gguf",
  "modelId": "RichardErkhov/NTQAI_-_Nxcode-CQ-7B-orpo-gguf",
  "sha": "64d43569391130a6696f9e40820a4e6ae1eb574e",
  "createdAt": "2024-06-25T17:27:51.000Z",
  "lastModified": "2024-06-25T21:33:39.000Z",
  "author": "RichardErkhov",
  "downloads": 988,
  "likes": 0,
  "gated": false,
  "private": false,
  "pipeline_tag": "",
  "library_name": "",
  "siblings_count": 24
}