baichuan-inc/baichuan-m2-32b-q4_k_m-gguf overview

{
  "metadata": {},
  "card_data": {
    "base_model": [
      "Qwen/Qwen2.5-32B"
    ],
    "language": [
      "en",
      "zh"
    ],
    "library_name": "transformers",
    "license": "apache-2.0",
    "tags": [
      "chat"
    ],
    "pipeline_tag": "text-generation",
    "paper": 2509.02208,
    "frontmatter": {
      "base_model": [
        "Qwen/Qwen2.5-32B"
      ],
      "language": [
        "en",
        "zh"
      ],
      "library_name": "transformers",
      "license": "apache-2.0",
      "tags": [
        "chat"
      ],
      "pipeline_tag": "text-generation",
      "paper": "2509.02208"
    },
    "hero_image_url": "https://img.shields.io/badge/License-Apache%202.0-blue.svg",
    "summary": "# Baichuan-M2-32B-Q4_K_M-GGUF This repository contains the model presented in Baichuan-M2: Scaling Medical Capability with Large Verifier System. ![License](https://opensource.org/licenses/Apache-2.0) ![Hugging Face](https://huggingface.co/baichuan-inc/Baichuan-M2-32B) ![M2 GPTQ-4bit](https://huggingface.co/baichuan-inc/Baichuan-M2-32B-GPTQ-Int4) ![Huawei Ascend 8bit](https://modelers.cn/models/Baichuan/Baichuan-M2-32B-W8A8)",
    "quick_links": [],
    "benchmark_table_html": "",
    "readme_markdown": "---\nbase_model:\n- Qwen/Qwen2.5-32B\nlanguage:\n- en\n- zh\nlibrary_name: transformers\nlicense: apache-2.0\ntags:\n- chat\npipeline_tag: text-generation\npaper: 2509.02208\n---\n# Baichuan-M2-32B-Q4_K_M-GGUF\n\nThis repository contains the model presented in [Baichuan-M2: Scaling Medical Capability with Large Verifier System](https://huggingface.co/papers/2509.02208).\n\n[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)\n[![Hugging Face](https://img.shields.io/badge/🤗%20Hugging%20Face-Model-yellow)](https://huggingface.co/baichuan-inc/Baichuan-M2-32B)\n[![M2 GPTQ-4bit](https://img.shields.io/badge/🤗%20M2%20GPTQ--4bit-Model-orange)](https://huggingface.co/baichuan-inc/Baichuan-M2-32B-GPTQ-Int4)\n[![Huawei Ascend 8bit](https://img.shields.io/badge/✨%20Huawei%20Ascend%208bit-Model-green)](https://modelers.cn/models/Baichuan/Baichuan-M2-32B-W8A8)\n\n## 🌟 Model Overview\n\nBaichuan-M2-32B is Baichuan AI's medical-enhanced reasoning model, the second medical model released by Baichuan. Designed for real-world medical reasoning tasks, this model builds upon Qwen2.5-32B with an innovative Large Verifier System. Through domain-specific fine-tuning on real-world medical questions, it achieves breakthrough medical performance while maintaining strong general capabilities.\n\n**Model Features:**\n\nBaichuan-M2 incorporates three core technical innovations: First, through the **Large Verifier System**, it combines medical scenario characteristics to design a comprehensive medical verification framework, including patient simulators and multi-dimensional verification mechanisms; second, through **medical domain adaptation enhancement** via Mid-Training, it achieves lightweight and efficient medical domain adaptation while preserving general capabilities; finally, it employs a **multi-stage reinforcement learning** strategy, decomposing complex RL tasks into hierarchical training stages to progressively enhance the model's medical knowledge, reasoning, and patient interaction capabilities.\n\n**Core Highlights:**\n- 🏆 **World's Leading Open-Source Medical Model**: Outperforms all open-source models and many proprietary models on HealthBench, achieving medical capabilities closest to GPT-5\n- 🧠 **Doctor-Thinking Alignment**: Trained on real clinical cases and patient simulators, with clinical diagnostic thinking and robust patient interaction capabilities\n- ⚡ **Efficient Deployment**: Supports 4-bit quantization for single-RTX4090 deployment, with 58.5% higher token throughput in MTP version for single-user scenarios\n\n## 📊 Performance Metrics\n\n### HealthBench Scores\n\n| Model Name | HealthBench | HealthBench-Hard | HealthBench-Consensus |\n|------------|-------------|------------------|-----------------------|\n| Baichuan-M2 | 60.1 | 34.7 | 91.5 |\n| gpt-oss-120b | 57.6 | 30 | 90 |\n| Qwen3-235B-A22B-Thinking-2507 | 55.2 | 25.9 | 90.6 |\n| Deepseek-R1-0528 | 53.6 | 22.6 | 91.5 |\n| GLM-4.5 | 47.8 | 18.7 | 85.3 |\n| Kimi-K2 | 43 | 10.7 | 90.9 |\n| gpt-oss-20b | 42.5 | 10.8 | 82.6 |\n\n### General Performance\n\n| Benchmark | Baichuan-M2-32B | Qwen3-32B (Thinking) |\n|-----------|-----------------|-----------|\n| AIME24 | 83.4 | 81.4 |\n| AIME25 | 72.9 | 72.9 |\n| Arena-Hard-v2.0 | 45.8 | 44.5 |\n| CFBench | 77.6 | 75.7 |\n| WritingBench | 8.56 | 7.90 |\n\n*Note: AIME uses max_tokens=64k, others use 32k; temperature=0.6 for all tests.*\n\n## 🔧 Technical Features\n\n📗 **Technical Blog**: [Blog - Baichuan-M2](https://www.baichuan-ai.com/blog/baichuan-M2)\n\n📑 **Technical Report**: [Arxiv - Baichuan-M2](https://arxiv.org/abs/2509.02208)\n\n### Large Verifier System\n- **Patient Simulator**: Virtual patient system based on real clinical cases\n- **Multi-Dimensional Verification**: 8 dimensions including medical accuracy, response completeness, and follow-up awareness\n- **Dynamic Scoring**: Real-time generation of adaptive evaluation criteria for complex clinical scenarios\n### Medical Domain Adaptation\n- **Mid-Training**: Medical knowledge injection while preserving general capabilities\n- **Reinforcement Learning**: Multi-stage RL strategy optimization\n- **General-Specialized Balance**: Carefully balanced medical, general, and mathematical composite training data\n\n## ⚙️ Quick Start\n\nFor deploying the Q4_K_M quantized model, you can use [llama.cpp](https://github.com/ggml-org/llama.cpp) or [ollama](https://github.com/ollama/ollama), please visit their website to get the specific operational steps for deploying the model.\nTaking ollama as an example.\n1. Ensure that Ollama is already installed\n2. Download the model: baichuan-m2-32b-q4_k_m.gguf\n3. Create and edit the Modelfile\n```shell\nFROM /path/to/baichuan-m2-32b-q4_k_m.gguf\n\nTEMPLATE \"\"\"{{- if .System -}}<<|im_start|>>system\n{{ .System }}<<|im_end|>>\n{{- end -}}\n{{- range .Messages -}}\n<<|im_start|>>{{ .Role }}\n{{ .Content }}<<|im_end|>>\n{{- end -}}\n<<|im_start|>>assistant\n\"\"\"\n\nPARAMETER stop \"<<|im_end|>>\"\nPARAMETER stop \"<<|im_start|>>\"\nPARAMETER temperature 0.6\nPARAMETER top_p 0.9\n```\n4. Create the model in Ollama\n```shell\nollama create baichuan-m2-q4km -f Modelfile\n```\n5. Launch the model, and you can begin chatting with it\n```shell\nollama run baichuan-m2-q4km\n```\n\n\n\n\n\n## ⚠️ Usage Notices\n1. **Medical Disclaimer**: For research and reference only; cannot replace professional medical diagnosis or treatment\n2. **Intended Use Cases**: Medical education, health consultation, clinical decision support\n3. **Safe Use**: Recommended under guidance of medical professionals\n\n## 📄 License\nLicensed under the [Apache License 2.0](LICENSE). Research and commercial use permitted.\n\n## 🤝 Acknowledgements\n- Base Model: Qwen2.5-32B\n- Training Framework: verl\n- Inference Engines: vLLM, SGLang\n- Quantization: AutoRound, GPTQ\nThank you to the open-source community. We commit to continuous contribution and advancement of healthcare AI.\n\n## 📞 Contact Us\n- Resources: [Baichuan AI Website](https://www.baichuan-ai.com)\n- Technical Support: [GitHub](https://github.com/baichuan-inc)\n\n---\n<div align=\"center\">\n\n**Empowering Healthcare with AI, Making Health Accessible to All**\n\n</div>",
    "related_quantizations": []
  },
  "tags": [
    "transformers",
    "gguf",
    "chat",
    "text-generation",
    "en",
    "zh",
    "arxiv:2509.02208",
    "base_model:Qwen/Qwen2.5-32B",
    "base_model:quantized:Qwen/Qwen2.5-32B",
    "license:apache-2.0",
    "endpoints_compatible",
    "region:us",
    "imatrix",
    "conversational"
  ],
  "likes": 3,
  "downloads": 122,
  "gated": false,
  "private": false,
  "last_modified": "2026-02-09T08:09:14.000Z",
  "created_at": "2026-02-06T11:43:48.000Z",
  "pipeline_tag": "text-generation",
  "library_name": "transformers"
}

{
  "_id": "6985d3f4abc04cee8e55687d",
  "id": "baichuan-inc/Baichuan-M2-32B-Q4_K_M-GGUF",
  "modelId": "baichuan-inc/Baichuan-M2-32B-Q4_K_M-GGUF",
  "sha": "9a9d5df2af9c251798b05b635e950a856d09489e",
  "createdAt": "2026-02-06T11:43:48.000Z",
  "lastModified": "2026-02-09T08:09:14.000Z",
  "author": "baichuan-inc",
  "downloads": 122,
  "likes": 3,
  "gated": false,
  "private": false,
  "pipeline_tag": "text-generation",
  "library_name": "transformers",
  "siblings_count": 4
}