enlistedghost/mistral-small-3.1-24b-instruct-2503-gguf overview

{
  "metadata": {},
  "card_data": {
    "license": "apache-2.0",
    "datasets": [
      "mistralai/MM-MT-Bench"
    ],
    "language": [
      "en",
      "ru",
      "uk",
      "ar",
      "es",
      "de",
      "tr",
      "sv",
      "sr",
      "ro",
      "pl",
      "ne",
      "ms",
      "fa",
      "id",
      "zh",
      "ja",
      "pt",
      "fr",
      "bn",
      "hi",
      "vi"
    ],
    "metrics": [
      "code_eval"
    ],
    "base_model": [
      "mistralai/Mistral-Small-3.1-24B-Instruct-2503"
    ],
    "new_version": "EnlistedGhost/Mistral-Small-3.1-24B-Instruct-2503-GGUF",
    "pipeline_tag": "image-text-to-text",
    "library_name": "transformers",
    "tags": [
      "Mistral-Small",
      "MistralAI",
      "Multilingual",
      "Ollama",
      "Llama.cpp",
      "GGUF",
      "Quantized",
      "Multi-Modal",
      "Vision",
      "mmproj",
      "Image-Text-to-Text"
    ],
    "frontmatter": {
      "license": "apache-2.0",
      "datasets": [
        "mistralai/MM-MT-Bench"
      ],
      "language": [
        "en",
        "ru",
        "uk",
        "ar",
        "es",
        "de",
        "tr",
        "sv",
        "sr",
        "ro",
        "pl",
        "ne",
        "ms",
        "fa",
        "id",
        "zh",
        "ja",
        "pt",
        "fr",
        "bn",
        "hi",
        "vi"
      ],
      "metrics": [
        "code_eval"
      ],
      "base_model": [
        "mistralai/Mistral-Small-3.1-24B-Instruct-2503"
      ],
      "new_version": "EnlistedGhost/Mistral-Small-3.1-24B-Instruct-2503-GGUF",
      "pipeline_tag": "image-text-to-text",
      "library_name": "transformers",
      "tags": [
        "Mistral-Small",
        "MistralAI",
        "Multilingual",
        "Ollama",
        "Llama.cpp",
        "GGUF",
        "Quantized",
        "Multi-Modal",
        "Vision",
        "mmproj",
        "Image-Text-to-Text"
      ]
    },
    "hero_image_url": "https://huggingface.co/api/resolve-cache/models/EnlistedGhost/Mistral-Small-3.1-24B-Instruct-2503-GGUF/1b5df8518e9317b8e82c606331f98eac7e6b4599/images%2FMistral_Small_3_1.jpeg?%2FEnlistedGhost%2FMistral-Small-3.1-24B-Instruct-2503-GGUF%2Fresolve%2Fmain%2Fimages%2FMistral_Small_3_1.jpeg=&etag=%22e8010bcf5402b9c637d49a480062861f083094be-inline%22",
    "summary": "**This release contains:**  Llama.cpp and Ollama compatible GGUF converted and Quantized model files *(Compatible with both Ollama, and Llama.cpp)*  *(More information and updates just landed! More on the way* (^.^) *- Thank you for taking the time to view this release!)* **Quantized GGUF version of:** **Original Model Link:** ----------------------------------------------",
    "quick_links": [],
    "benchmark_table_html": "",
    "readme_markdown": "---\nlicense: apache-2.0\ndatasets:\n- mistralai/MM-MT-Bench\nlanguage:\n- en\n- ru\n- uk\n- ar\n- es\n- de\n- tr\n- sv\n- sr\n- ro\n- pl\n- ne\n- ms\n- fa\n- id\n- zh\n- ja\n- pt\n- fr\n- bn\n- hi\n- vi\nmetrics:\n- code_eval\nbase_model:\n- mistralai/Mistral-Small-3.1-24B-Instruct-2503\nnew_version: EnlistedGhost/Mistral-Small-3.1-24B-Instruct-2503-GGUF\npipeline_tag: image-text-to-text\nlibrary_name: transformers\ntags:\n- Mistral-Small\n- MistralAI\n- Multilingual\n- Ollama\n- Llama.cpp\n- GGUF\n- Quantized\n- Multi-Modal\n- Vision\n- mmproj\n- Image-Text-to-Text\n---\n\n\n<img src=\"https://huggingface.co/api/resolve-cache/models/EnlistedGhost/Mistral-Small-3.1-24B-Instruct-2503-GGUF/1b5df8518e9317b8e82c606331f98eac7e6b4599/images%2FMistral_Small_3_1.jpeg?%2FEnlistedGhost%2FMistral-Small-3.1-24B-Instruct-2503-GGUF%2Fresolve%2Fmain%2Fimages%2FMistral_Small_3_1.jpeg=&etag=%22e8010bcf5402b9c637d49a480062861f083094be-inline%22\" alt=\"Loading...\" width=\"241\" height=\"192\">\n\n## -----------------------------------------------<br /> - Model Details and Specifications: -<br />-----------------------------------------------\n\n# Mistral Small 3.1 24B Instruct 2503 GGUF (Ollama & Llama.cpp)\n\n**This release contains:** <br />\nLlama.cpp and Ollama compatible GGUF converted and Quantized model files \n*(Compatible with both Ollama, and Llama.cpp)* <br />\n*(More information and updates just landed! More on the way* (^.^) *- Thank you for taking the time to view this release!)*\n\n**Quantized GGUF version of:**\n- mistralai/Mistral-Small-3.1-24B-Instruct-2503 <br /> *(by MistralAI)*\n\n**Original Model Link:**\n- [mistralai/Mistral-Small-3.1-24B-Instruct-2503](https://huggingface.co/mistralai/Mistral-Small-3.1-24B-Instruct-2503)\n\n----------------------------------------------\n\n## ---------------------------------------------------<br /> - Conversion and GGUF Quantization: -<br />---------------------------------------------------\n\n**Software used to convert Safetensors to GGUF:**\n- <a href=\"https://github.com/ggml-org/llama.cpp/\">llama.cpp, release verson: (b7540)</a>\n\n**Software used to create Quantized GGUF Files:**\n- <a href=\"https://github.com/ggml-org/llama.cpp/\">llama.cpp, release verson: (b7540)</a> \n\n**Specific GitHub Commit Point:**\n- <a href=\"https://github.com/ggml-org/llama.cpp/commit/85c40c9b02941ebf1add1469af75f1796d513ef4\">b7540</a>\n\n**Converted to GGUF and Quantized by:**\n- [EnlistedGhost](https://huggingface.co/EnlistedGhost)\n\n----------------------------------------------\n\n## -------------------------------<br /> ---- Updates & News ---- <br /> -------------------------------\n\n**Model Updates (as of: January 25th, 2026)**\n- Uploaded: Remaining GGUF Converted and Quantized model files\n- Updated: ModelCard <br /> \n(this page)\n- Planned Updates: Yes, more updates coming! Thank you whoever you are for your patience (^.^)\n\n---------------------------------------------\n\n### --------------------------------------<br /> ---- How to run this Model ---- <br /> --------------------------------------\n\n**Compatible Software (Required to use this Model**) <br />\nYou can run this model by using either Ollama (or) Llama.cpp <br />\n*(Below are instruction on running these GGUF files with Ollama)*\n\n**How to run this Model using Ollama** <br />\nYou can run this model by using the \"ollama run\" command.<br />\nSimply copy & paste one of the commands from the list below into<br /> \nyour console, terminal or power-shell window.\n| Quant Type | File Size | Command |\n|:-----------|:----------|:--------|\n| QX_X | 0.00 GB | (Currently Uploading Files, Check again very soon!) |\n\n**Vision Projector (Files)** <br />\n*mmproj (Vision Projector) Files*\n| Quant Type | File Size | Download Link |\n|:-----------|:----------|:--------|\n| Q8_0 | 465 MB | |\n| F16 | 870 MB | |\n| F32 | 1.74 GB | |\n\n\n-----------------------------------------------\n\n\n## ---------------------------<br /> ---- Original Info ---- <br /> ---------------------------\n*(Crossposted from the link in the above section: \"Model Details\"):*\n<br />\n<br />\n\n\nBuilding upon Mistral Small 3 (2501), Mistral Small 3.1 (2503) **adds state-of-the-art vision understanding** and enhances **long context capabilities up to 128k tokens** without compromising text performance. \nWith 24 billion parameters, this model achieves top-tier capabilities in both text and vision tasks.  \nThis model is an instruction-finetuned version of: [Mistral-Small-3.1-24B-Base-2503](https://huggingface.co/mistralai/Mistral-Small-3.1-24B-Base-2503).\n\nMistral Small 3.1 can be deployed locally and is exceptionally \"knowledge-dense,\" fitting within a single RTX 4090 or a 32GB RAM MacBook once quantized.  \n\nIt is ideal for:\n- Fast-response conversational agents.\n- Low-latency function calling.\n- Subject matter experts via fine-tuning.\n- Local inference for hobbyists and organizations handling sensitive data.\n- Programming and math reasoning.\n- Long document understanding.\n- Visual understanding.\n\nFor enterprises requiring specialized capabilities (increased context, specific modalities, domain-specific knowledge, etc.), we will release commercial models beyond what Mistral AI contributes to the community.\n\nLearn more about Mistral Small 3.1 in our [blog post](https://mistral.ai/news/mistral-small-3-1/).\n\n## Key Features\n- **Vision:** Vision capabilities enable the model to analyze images and provide insights based on visual content in addition to text.\n- **Multilingual:** Supports dozens of languages, including English, French, German, Greek, Hindi, Indonesian, Italian, Japanese, Korean, Malay, Nepali, Polish, Portuguese, Romanian, Russian, Serbian, Spanish, Swedish, Turkish, Ukrainian, Vietnamese, Arabic, Bengali, Chinese, Farsi.\n- **Agent-Centric:** Offers best-in-class agentic capabilities with native function calling and JSON outputting.\n- **Advanced Reasoning:** State-of-the-art conversational and reasoning capabilities.\n- **Apache 2.0 License:** Open license allowing usage and modification for both commercial and non-commercial purposes.\n- **Context Window:** A 128k context window.\n- **System Prompt:** Maintains strong adherence and support for system prompts.\n- **Tokenizer:** Utilizes a Tekken tokenizer with a 131k vocabulary size.\n\n## Benchmark Results\n\nWhen available, we report numbers previously published by other model providers, otherwise we re-evaluate them using our own evaluation harness.\n\n### Pretrain Evals\n\n| Model                          | MMLU (5-shot) | MMLU Pro (5-shot CoT) | TriviaQA   | GPQA Main (5-shot CoT)| MMMU      |\n|--------------------------------|---------------|-----------------------|------------|-----------------------|-----------|\n| **Small 3.1 24B Base**         | **81.01%**    | **56.03%**            | 80.50%     | **37.50%**            | **59.27%**|\n| Gemma 3 27B PT                 | 78.60%        | 52.20%                | **81.30%** | 24.30%                | 56.10%    |\n\n### Instruction Evals\n\n#### Text\n\n| Model                          | MMLU      | MMLU Pro (5-shot CoT) | MATH                   | GPQA Main (5-shot CoT) | GPQA Diamond (5-shot CoT )| MBPP      | HumanEval | SimpleQA (TotalAcc)|\n|--------------------------------|-----------|-----------------------|------------------------|------------------------|---------------------------|-----------|-----------|--------------------|\n| **Small 3.1 24B Instruct**     | 80.62%    | 66.76%                | 69.30%                 | **44.42%**             | **45.96%**                | 74.71%    | **88.41%**| **10.43%**         |\n| Gemma 3 27B IT                 | 76.90%    | **67.50%**            | **89.00%**             | 36.83%                 | 42.40%                    | 74.40%    | 87.80%    | 10.00%             |\n| GPT4o Mini                     | **82.00%**| 61.70%                | 70.20%                 | 40.20%                 | 39.39%                    | 84.82%    | 87.20%    | 9.50%              |\n| Claude 3.5 Haiku               | 77.60%    | 65.00%                | 69.20%                 | 37.05%                 | 41.60%                    | **85.60%**| 88.10%    | 8.02%              |\n| Cohere Aya-Vision 32B          | 72.14%    | 47.16%                | 41.98%                 | 34.38%                 | 33.84%                    | 70.43%    | 62.20%    | 7.65%              |\n\n#### Vision\n\n| Model                          | MMMU       | MMMU PRO  | Mathvista | ChartQA   | DocVQA    | AI2D        | MM MT Bench |\n|--------------------------------|------------|-----------|-----------|-----------|-----------|-------------|-------------|\n| **Small 3.1 24B Instruct**     | 64.00%     | **49.25%**| **68.91%**| 86.24%    | **94.08%**| **93.72%**  | **7.3**     |\n| Gemma 3 27B IT                 | **64.90%** | 48.38%    | 67.60%    | 76.00%    | 86.60%    | 84.50%      | 7           |\n| GPT4o Mini                     | 59.40%     | 37.60%    | 56.70%    | 76.80%    | 86.70%    | 88.10%      | 6.6         |\n| Claude 3.5 Haiku               | 60.50%     | 45.03%    | 61.60%    | **87.20%**| 90.00%    | 92.10%      | 6.5         |\n| Cohere Aya-Vision 32B          | 48.20%     | 31.50%    | 50.10%    | 63.04%    | 72.40%    | 82.57%      | 4.1         |\n\n### Multilingual Evals\n\n| Model                          | Average    | European   | East Asian | Middle Eastern |\n|--------------------------------|------------|------------|------------|----------------|\n| **Small 3.1 24B Instruct**     | **71.18%** | **75.30%** | **69.17%** | 69.08%         |\n| Gemma 3 27B IT                 | 70.19%     | 74.14%     | 65.65%     | 70.76%         |\n| GPT4o Mini                     | 70.36%     | 74.21%     | 65.96%     | **70.90%**     |\n| Claude 3.5 Haiku               | 70.16%     | 73.45%     | 67.05%     | 70.00%         |\n| Cohere Aya-Vision 32B          | 62.15%     | 64.70%     | 57.61%     | 64.12%         |\n\n### Long Context Evals\n\n| Model                          | LongBench v2    | RULER 32K   | RULER 128K |\n|--------------------------------|-----------------|-------------|------------|\n| **Small 3.1 24B Instruct**     | **37.18%**      | **93.96%**  | 81.20%     |\n| Gemma 3 27B IT                 | 34.59%          | 91.10%      | 66.00%     |\n| GPT4o Mini                     | 29.30%          | 90.20%      | 65.8%      |\n| Claude 3.5 Haiku               | 35.19%          | 92.60%      | **91.90%** |",
    "related_quantizations": []
  },
  "tags": [
    "transformers",
    "gguf",
    "Mistral-Small",
    "MistralAI",
    "Multilingual",
    "Ollama",
    "Llama.cpp",
    "GGUF",
    "Quantized",
    "Multi-Modal",
    "Vision",
    "mmproj",
    "Image-Text-to-Text",
    "image-text-to-text",
    "en",
    "ru",
    "uk",
    "ar",
    "es",
    "de",
    "tr",
    "sv",
    "sr",
    "ro",
    "pl",
    "ne",
    "ms",
    "fa",
    "id",
    "zh",
    "ja",
    "pt",
    "fr",
    "bn",
    "hi",
    "vi",
    "dataset:mistralai/MM-MT-Bench",
    "base_model:mistralai/Mistral-Small-3.1-24B-Instruct-2503",
    "base_model:quantized:mistralai/Mistral-Small-3.1-24B-Instruct-2503",
    "license:apache-2.0",
    "endpoints_compatible",
    "region:us",
    "conversational"
  ],
  "likes": 0,
  "downloads": 506,
  "gated": false,
  "private": false,
  "last_modified": "2026-01-26T02:37:22.000Z",
  "created_at": "2025-12-02T22:36:26.000Z",
  "pipeline_tag": "image-text-to-text",
  "library_name": "transformers"
}

{
  "_id": "692f69ea5e5d56233e98e355",
  "id": "EnlistedGhost/Mistral-Small-3.1-24B-Instruct-2503-GGUF",
  "modelId": "EnlistedGhost/Mistral-Small-3.1-24B-Instruct-2503-GGUF",
  "sha": "756791fd59dcc7158fdf50ed21a679d73ca6c1c7",
  "createdAt": "2025-12-02T22:36:26.000Z",
  "lastModified": "2026-01-26T02:37:22.000Z",
  "author": "EnlistedGhost",
  "downloads": 506,
  "likes": 0,
  "gated": false,
  "private": false,
  "pipeline_tag": "image-text-to-text",
  "library_name": "transformers",
  "siblings_count": 25
}