richarderkhov/cardinaloperations_-_orlm-llama-3-8b-gguf overview

{
  "metadata": {},
  "card_data": {
    "library_name": "transformers",
    "license": "llama3",
    "pipeline_tag": "text-generation",
    "frontmatter": {
      "library_name": "transformers",
      "license": "llama3",
      "pipeline_tag": "text-generation"
    },
    "hero_image_url": "",
    "summary": "This repository contains the quantized model weights for ORLM-LLaMA-3-8B, a fine-tuned version of Meta's Llama 3 8B model, specifically trained for generating optimization models and corresponding Python code.  It's presented in the paper ORLM: A Customizable Framework in Training Large Models for Automated Optimization Modeling.  This model significantly improves the capability of automating optimization modeling and solving, achieving state-of-the-art performance on several benchmarks. The official repo can be found here: https://github.com/Cardinal-Operations/ORLM Quantization made by Richard Erkhov. Github Discord Request more models ORLM-LLaMA-3-8B - GGUF | Name | Quant method | Size | | ---- | ---- | ---- | | ORLM-LLaMA-3-8B.Q2_K.gguf | Q2_K | 2.96GB | | ORLM-LLaMA-3-8B.IQ3_XS.gguf | IQ3_XS | 3.28GB | | ORLM-LLaMA-3-8B.IQ3_S.gguf | IQ3_S | 3.43GB | | ORLM-LLaMA-3-8B.Q3_K_S.gguf | Q3_K_S | 3.41GB | | ORLM-LLaMA-3-8B.IQ3_M.gguf | IQ3_M | 3.52GB | | ORLM-LLaMA-3-8B.Q3_K.gguf | Q3_K | 3.74GB | | ORLM-LLaMA-3-8B.Q3_K_M.gguf | Q3_K_M | 3.74GB | | ORLM-LLaMA-3-8B.Q3_K_L.gguf | Q3_K_L | 4.03GB | | ORLM-LLaMA-3-8B.IQ4_XS.gguf | IQ4_XS | 4.18GB | | ORLM-LLaMA-3-8B.Q4_0.gguf | Q4_0 | 4.34GB | | ORLM-LLaMA-3-8B.IQ4_NL.gguf | IQ4_NL | 4.38GB | | ORLM-LLaMA-3-8B.Q4_K_S.gguf | Q4_K_S | 4.37GB | | ORLM-LLaMA-3-8B.Q4_K.gguf | Q4_K | 4.58GB | | ORLM-LLaMA-3-8B.Q4_K_M.gguf | Q4_K_M | 4.58GB | | ORLM-LLaMA-3-8B.Q4_1.gguf | Q4_1 | 4.78GB | | ORLM-LLaMA-3-8B.Q5_0.gguf | Q5_0 | 5.21GB | | ORLM-LLaMA-3-8B.Q5_K_S.gguf | Q5_K_S | 5.21GB | | ORLM-LLaMA-3-8B.Q5_K.gguf | Q5_K | 5.34GB | | ORLM-LLaMA-3-8B.Q5_K_M.gguf | Q5_K_M | 5.34GB | | ORLM-LLaMA-3-8B.Q5_1.gguf | Q5_1 | 5.65GB | | ORLM-LLaMA-3-8B.Q6_K.gguf | Q6_K | 6.14GB | | ORLM-LLaMA-3-8B.Q8_0.gguf | Q8_0 | 7.95GB | Original model description: --- license: llama3 --- see our paper in https://arxiv.org/abs/2405.17743 github repo: https://github.com/Cardinal-Operations/ORLM",
    "quick_links": [],
    "benchmark_table_html": "",
    "readme_markdown": "---\nlibrary_name: transformers\nlicense: llama3\npipeline_tag: text-generation\n---\n\n# ORLM-LLaMA-3-8B: A Large Language Model for Optimization Modeling\n\nThis repository contains the quantized model weights for [ORLM-LLaMA-3-8B](https://huggingface.co/CardinalOperations/ORLM-LLaMA-3-8B), a fine-tuned version of Meta's Llama 3 8B model, specifically trained for generating optimization models and corresponding Python code.  It's presented in the paper [ORLM: A Customizable Framework in Training Large Models for Automated Optimization Modeling](https://huggingface.co/papers/2405.17743).  This model significantly improves the capability of automating optimization modeling and solving, achieving state-of-the-art performance on several benchmarks.\n\nThe official repo can be found here: https://github.com/Cardinal-Operations/ORLM\n\nQuantization made by Richard Erkhov.\n\n[Github](https://github.com/RichardErkhov)\n\n[Discord](https://discord.gg/pvy7H8DZMG)\n\n[Request more models](https://github.com/RichardErkhov/quant_request)\n\n\nORLM-LLaMA-3-8B - GGUF\n- Model creator: https://huggingface.co/CardinalOperations/\n- Original model: https://huggingface.co/CardinalOperations/ORLM-LLaMA-3-8B/\n\n\n| Name | Quant method | Size |\n| ---- | ---- | ---- |\n| [ORLM-LLaMA-3-8B.Q2_K.gguf](https://huggingface.co/RichardErkhov/CardinalOperations_-_ORLM-LLaMA-3-8B-gguf/blob/main/ORLM-LLaMA-3-8B.Q2_K.gguf) | Q2_K | 2.96GB |\n| [ORLM-LLaMA-3-8B.IQ3_XS.gguf](https://huggingface.co/RichardErkhov/CardinalOperations_-_ORLM-LLaMA-3-8B-gguf/blob/main/ORLM-LLaMA-3-8B.IQ3_XS.gguf) | IQ3_XS | 3.28GB |\n| [ORLM-LLaMA-3-8B.IQ3_S.gguf](https://huggingface.co/RichardErkhov/CardinalOperations_-_ORLM-LLaMA-3-8B-gguf/blob/main/ORLM-LLaMA-3-8B.IQ3_S.gguf) | IQ3_S | 3.43GB |\n| [ORLM-LLaMA-3-8B.Q3_K_S.gguf](https://huggingface.co/RichardErkhov/CardinalOperations_-_ORLM-LLaMA-3-8B-gguf/blob/main/ORLM-LLaMA-3-8B.Q3_K_S.gguf) | Q3_K_S | 3.41GB |\n| [ORLM-LLaMA-3-8B.IQ3_M.gguf](https://huggingface.co/RichardErkhov/CardinalOperations_-_ORLM-LLaMA-3-8B-gguf/blob/main/ORLM-LLaMA-3-8B.IQ3_M.gguf) | IQ3_M | 3.52GB |\n| [ORLM-LLaMA-3-8B.Q3_K.gguf](https://huggingface.co/RichardErkhov/CardinalOperations_-_ORLM-LLaMA-3-8B-gguf/blob/main/ORLM-LLaMA-3-8B.Q3_K.gguf) | Q3_K | 3.74GB |\n| [ORLM-LLaMA-3-8B.Q3_K_M.gguf](https://huggingface.co/RichardErkhov/CardinalOperations_-_ORLM-LLaMA-3-8B-gguf/blob/main/ORLM-LLaMA-3-8B.Q3_K_M.gguf) | Q3_K_M | 3.74GB |\n| [ORLM-LLaMA-3-8B.Q3_K_L.gguf](https://huggingface.co/RichardErkhov/CardinalOperations_-_ORLM-LLaMA-3-8B-gguf/blob/main/ORLM-LLaMA-3-8B.Q3_K_L.gguf) | Q3_K_L | 4.03GB |\n| [ORLM-LLaMA-3-8B.IQ4_XS.gguf](https://huggingface.co/RichardErkhov/CardinalOperations_-_ORLM-LLaMA-3-8B-gguf/blob/main/ORLM-LLaMA-3-8B.IQ4_XS.gguf) | IQ4_XS | 4.18GB |\n| [ORLM-LLaMA-3-8B.Q4_0.gguf](https://huggingface.co/RichardErkhov/CardinalOperations_-_ORLM-LLaMA-3-8B-gguf/blob/main/ORLM-LLaMA-3-8B.Q4_0.gguf) | Q4_0 | 4.34GB |\n| [ORLM-LLaMA-3-8B.IQ4_NL.gguf](https://huggingface.co/RichardErkhov/CardinalOperations_-_ORLM-LLaMA-3-8B-gguf/blob/main/ORLM-LLaMA-3-8B.IQ4_NL.gguf) | IQ4_NL | 4.38GB |\n| [ORLM-LLaMA-3-8B.Q4_K_S.gguf](https://huggingface.co/RichardErkhov/CardinalOperations_-_ORLM-LLaMA-3-8B-gguf/blob/main/ORLM-LLaMA-3-8B.Q4_K_S.gguf) | Q4_K_S | 4.37GB |\n| [ORLM-LLaMA-3-8B.Q4_K.gguf](https://huggingface.co/RichardErkhov/CardinalOperations_-_ORLM-LLaMA-3-8B-gguf/blob/main/ORLM-LLaMA-3-8B.Q4_K.gguf) | Q4_K | 4.58GB |\n| [ORLM-LLaMA-3-8B.Q4_K_M.gguf](https://huggingface.co/RichardErkhov/CardinalOperations_-_ORLM-LLaMA-3-8B-gguf/blob/main/ORLM-LLaMA-3-8B.Q4_K_M.gguf) | Q4_K_M | 4.58GB |\n| [ORLM-LLaMA-3-8B.Q4_1.gguf](https://huggingface.co/RichardErkhov/CardinalOperations_-_ORLM-LLaMA-3-8B-gguf/blob/main/ORLM-LLaMA-3-8B.Q4_1.gguf) | Q4_1 | 4.78GB |\n| [ORLM-LLaMA-3-8B.Q5_0.gguf](https://huggingface.co/RichardErkhov/CardinalOperations_-_ORLM-LLaMA-3-8B-gguf/blob/main/ORLM-LLaMA-3-8B.Q5_0.gguf) | Q5_0 | 5.21GB |\n| [ORLM-LLaMA-3-8B.Q5_K_S.gguf](https://huggingface.co/RichardErkhov/CardinalOperations_-_ORLM-LLaMA-3-8B-gguf/blob/main/ORLM-LLaMA-3-8B.Q5_K_S.gguf) | Q5_K_S | 5.21GB |\n| [ORLM-LLaMA-3-8B.Q5_K.gguf](https://huggingface.co/RichardErkhov/CardinalOperations_-_ORLM-LLaMA-3-8B-gguf/blob/main/ORLM-LLaMA-3-8B.Q5_K.gguf) | Q5_K | 5.34GB |\n| [ORLM-LLaMA-3-8B.Q5_K_M.gguf](https://huggingface.co/RichardErkhov/CardinalOperations_-_ORLM-LLaMA-3-8B-gguf/blob/main/ORLM-LLaMA-3-8B.Q5_K_M.gguf) | Q5_K_M | 5.34GB |\n| [ORLM-LLaMA-3-8B.Q5_1.gguf](https://huggingface.co/RichardErkhov/CardinalOperations_-_ORLM-LLaMA-3-8B-gguf/blob/main/ORLM-LLaMA-3-8B.Q5_1.gguf) | Q5_1 | 5.65GB |\n| [ORLM-LLaMA-3-8B.Q6_K.gguf](https://huggingface.co/RichardErkhov/CardinalOperations_-_ORLM-LLaMA-3-8B-gguf/blob/main/ORLM-LLaMA-3-8B.Q6_K.gguf) | Q6_K | 6.14GB |\n| [ORLM-LLaMA-3-8B.Q8_0.gguf](https://huggingface.co/RichardErkhov/CardinalOperations_-_ORLM-LLaMA-3-8B-gguf/blob/main/ORLM-LLaMA-3-8B.Q8_0.gguf) | Q8_0 | 7.95GB |\n\n\n\n\nOriginal model description:\n---\nlicense: llama3\n---\nsee our paper in https://arxiv.org/abs/2405.17743\n\ngithub repo: https://github.com/Cardinal-Operations/ORLM\n\n## Model Details\n\nLLaMA-3-8B-ORLM is fully fine-tuned on the OR-Instruct data and built with Meta [LLaMA-3-8B](https://huggingface.co/meta-llama/Meta-Llama-3-8B) model. \nMore training details can be seen at https://arxiv.org/abs/2405.17743\n\n## Model Usage\n\nPrompting Template:\n```text\nBelow is an operations research question. Build a mathematical model and corresponding python code using `coptpy` that appropriately addresses the question.\n\n# Question:\n{Question}\n\n# Response:\n```\n\nPlease replace the `{Question}` with any natural language OR question.\n\n## Examples\n\n```text\n--------------------------OR question wrapped in the template--------------------------\nBelow is an operations research question. Build a mathematical model and corresponding python code using `coptpy` that appropriately addresses the question.\n\n# Question:\nA lab has 1000 units of medicinal ingredients to make two pills, a large pill and a small pill. A large pill requires 3 units of medicinal ingredients and 2 units of filler. A small pill requires 2 units of medicinal ingredients and 1 unit of filler. The lab has to make at least 100 large pills. However, since small pills are more popular at least 60% of the total number of pills must be small. How many of each should be made to minimize the total number of filler material needed?\n\n# Response:\n--------------------------top-1 completion output by LLaMA-3-8B-ORLM under greedy decoding--------------------------\n## Mathematical Model:\nTo solve the lab's pill production problem, we need to construct a linear programming model to minimize the total number of filler material needed while satisfying the production requirements for the pills and the proportion constraint. Here is the model formulation:\n\n### Decision Variables:\n- \\(x\\): Number of large pills to be made.\n- \\(y\\): Number of small pills to be made.\n\n### Objective Function:\n- Minimize the total number of filler material needed: \\[Z = 2x + y\\]\n\n### Constraints:\n1. Medicinal ingredient usage constraint: \\[3x + 2y \\leq 1000\\]\n2. Large pill production requirement: \\[x \\geq 100\\]\n3. Proportion constraint for small pills: \\[y \\geq 0.6(x + y)\\]\n4. Non-negativity constraint: \\[x, y \\geq 0\\]\n\n### Integer Constraints:\n- \\(x, y\\) must be integers, as the number of pills cannot be fractional.\n\nUsing this model, we can apply linear programming techniques to find the optimal number of large and small pills to be made, minimizing the total number of filler material needed while satisfying the production requirements for the pills and the proportion constraint. This will provide the lab with an economically efficient pill production strategy.\n\n## Python Code Solution Using `coptpy`:\nHere is a Python script using the `coptpy` library to solve the problem:\n\n\\`\\`\\`python\nimport coptpy as cp\nfrom coptpy import COPT\n\n# Create a COPT environment\nenv = cp.Envr()\n\n# Create the model\nmodel = env.createModel(\"PillProduction\")\n\n# Add decision variables\nx = model.addVar(vtype=COPT.INTEGER, name=\"x\")  # Number of large pills\ny = model.addVar(vtype=COPT.INTEGER, name=\"y\")  # Number of small pills\n\n# Objective function: Minimize the total number of filler material needed\nmodel.setObjective(2 * x + y, sense=COPT.MINIMIZE)\n\n# Constraints\nmodel.addConstr(3 * x + 2 * y <= 1000, name=\"MaterialLimit\")\nmodel.addConstr(x >= 100, name=\"LargePillRequirement\")\nmodel.addConstr(y >= 0.6 * (x + y), name=\"SmallPillProportion\")\n\n# Solve the model\nmodel.solve()\n\n# Output results\nif model.status == COPT.OPTIMAL:\n    print(\"Minimum total number of filler material needed: {:.0f} units\".format(model.objval))\n    print(\"Number of large pills to be made: {:.0f}\".format(x.x))\n    print(\"Number of small pills to be made: {:.0f}\".format(y.x))\nelse:\n    print(\"No optimal solution found.\")\n\\`\\`\\`\n\nIn this script, we first create a `COPT` environment and model. Then, we add two integer decision variables `x` and `y`, representing the number of large and small pills to be made, respectively.\n\nNext, we set the objective function to minimize the total number of filler material needed, which is `2 * x + y`.\n\nThen, we add three constraints:\n1. Medicinal ingredient usage constraint: `3 * x + 2 * y <= 1000`\n2. Large pill production requirement: `x >= 100`\n3. Proportion constraint for small pills: `y >= 0.6 * (x + y)`\n\nFinally, we solve the model and output the results of the optimal solution, including the minimum total number of filler material needed and the number of large and small pills to be made.\n\nThis script provides a complete example of using the `coptpy` library to solve the lab's pill production problem, while satisfying all the constraints mentioned in the problem.\n```\n\n## Performances\n\nBelow is the comparison of performance on the NL4OPT, MAMO, and IndustryOR benchmarks. Values marked with a <sup>*</sup> are directly copied from original papers, with blanks where data were not reported. The highest results are highlighted in bold.\n\n| **Method**                                     | **NL4OPT**              | **MAMO EasyLP**       | **MAMO ComplexLP**  | **IndustryOR**    | **Micro Avg**   | **Macro Avg**   |\n|------------------------------------------------|-------------------------|-----------------------|----------------------|-------------------|-----------------|-----------------|\n| *Methods based on PLMs*                        |                         |                       |                      |                   |                 |                 |\n| `tag-BART`                                     | 47.9%<sup>*</sup>               | -                     | -                    | -                 | -               | -               |\n| *Methods based on GPT-3.5*                     |                         |                       |                      |                   |                 |                 |\n| `Standard`                                     | 42.4%<sup>*</sup>                | -                     | -                    | -                 | -               | -               |\n| `Reflexion`                                    | 50.7%<sup>*</sup>                | -                     | -                    | -                 | -               | -               |\n| `Chain-of-Experts`                             | 58.9%<sup>*</sup>                | -                     | -                    | -                 | -               | -               |\n| *Methods based on GPT-4*                       |                         |                       |                      |                   |                 |                 |\n| `Standard`                                     | 47.3%<sup>*</sup>                | 66.5%<sup>*</sup>              | 14.6%<sup>*</sup>             | 28.0%             | 50.2%           | 39.1%           |\n| `Reflexion`                                    | 53.0%<sup>*</sup>                | -                     | -                    | -                 | -               | -               |\n| `Chain-of-Experts`                             | 64.2%<sup>*</sup>                | -                     | -                    | -                 | -               | -               |\n| `OptiMUS`                                      | 78.8%<sup>*</sup>                | -                     | -                    | -                 | -               | -               |\n| *ORLMs based on open-source LLMs*              |                         |                       |                      |                   |                 |                 |\n| `ORLM-Mistral-7B`                              | 84.4%                   | 81.4%                 | 32.0%                | 27.0%             | 68.8%           | 56.2%           |\n| `ORLM-Deepseek-Math-7B-Base`                   | **86.5%**               | 82.2%                 | **37.9%**            | 33.0%             | 71.2%           | 59.9%           |\n| `ORLM-LLaMA-3-8B`                              | 85.7%                   | **82.3%**             | 37.4%                | **38.0%**         | **71.4%**       | **60.8%**       |\n\n\n## Citation\n\n```bibtex\n@article{tang2024orlm,\n  title={ORLM: A Customizable Framework in Training Large Models for Automated Optimization Modeling},\n  author={Tang, Zhengyang and Huang, Chenyu and Zheng, Xin and Hu, Shixi and Wang, Zizhuo and Ge, Dongdong and Wang, Benyou},\n  journal={arXiv preprint arXiv:2405.17743},\n  year={2024}\n}\n```\n\n```bibtex\n@article{llama3modelcard,\n  title={Llama 3 Model Card},\n  author={AI@Meta},\n  year={2024},\n  url = {https://github.com/meta-llama/llama3/blob/main/MODEL_CARD.md}\n}\n```\n\n## License\nThe use of this model is governed by the [META LLAMA 3 COMMUNITY LICENSE AGREEMENT](https://llama.meta.com/llama3/license/).",
    "related_quantizations": []
  },
  "tags": [
    "transformers",
    "gguf",
    "text-generation",
    "arxiv:2405.17743",
    "license:llama3",
    "endpoints_compatible",
    "region:us"
  ],
  "likes": 0,
  "downloads": 376,
  "gated": false,
  "private": false,
  "last_modified": "2025-05-31T21:51:51.000Z",
  "created_at": "2024-08-21T11:59:56.000Z",
  "pipeline_tag": "text-generation",
  "library_name": "transformers"
}

{
  "_id": "66c5d6bcd16aa41ecc84f255",
  "id": "RichardErkhov/CardinalOperations_-_ORLM-LLaMA-3-8B-gguf",
  "modelId": "RichardErkhov/CardinalOperations_-_ORLM-LLaMA-3-8B-gguf",
  "sha": "b74d354b7797d533d6a4701d47a7bf4604d3fc8f",
  "createdAt": "2024-08-21T11:59:56.000Z",
  "lastModified": "2025-05-31T21:51:51.000Z",
  "author": "RichardErkhov",
  "downloads": 376,
  "likes": 0,
  "gated": false,
  "private": false,
  "pipeline_tag": "text-generation",
  "library_name": "transformers",
  "siblings_count": 24
}