aaryank/glm-4.7-gguf overview

{
  "metadata": {},
  "card_data": {
    "base_model": "zai-org/GLM-4.7",
    "base_model_relation": "quantized",
    "language": [
      "en",
      "zh"
    ],
    "library_name": "gguf",
    "license": "mit",
    "pipeline_tag": "text-generation",
    "tags": [
      "text-generation-inference",
      "glm",
      "chat",
      "agents",
      "code",
      "thinking",
      "gguf"
    ],
    "frontmatter": {
      "base_model": "zai-org/GLM-4.7",
      "base_model_relation": "quantized",
      "language": [
        "en",
        "zh"
      ],
      "library_name": "gguf",
      "license": "mit",
      "pipeline_tag": "text-generation",
      "tags": [
        "text-generation-inference",
        "glm",
        "chat",
        "agents",
        "code",
        "thinking",
        "gguf"
      ]
    },
    "hero_image_url": "https://raw.githubusercontent.com/zai-org/GLM-4.5/refs/heads/main/resources/logo.svg",
    "summary": "> [!IMPORTANT] > **I am currently looking for open positions!** 🤗 > If you find this model useful or are looking for a talented AI/LLM Engineer, please reach out to me on LinkedIn: **Aaryan Kapoor**.",
    "quick_links": [],
    "benchmark_table_html": "",
    "readme_markdown": "---\nbase_model: zai-org/GLM-4.7\nbase_model_relation: quantized\nlanguage:\n  - en\n  - zh\nlibrary_name: gguf\nlicense: mit\npipeline_tag: text-generation\ntags:\n  - text-generation-inference\n  - glm\n  - chat\n  - agents\n  - code\n  - thinking\n  - gguf\n---\n\n# GLM-4.7-GGUF\n\n<div align=\"center\">\n<img src=\"https://raw.githubusercontent.com/zai-org/GLM-4.5/refs/heads/main/resources/logo.svg\" width=\"15%\"/>\n</div>\n\n> [!IMPORTANT]\n> **I am currently looking for open positions!** 🤗\n> If you find this model useful or are looking for a talented AI/LLM Engineer, please reach out to me on LinkedIn: **[Aaryan Kapoor](https://www.linkedin.com/in/theaaryankapoor/)**.\n\n\n## Description\n\nThis repository contains **GGUF** format model files for [Zhipu AI's GLM-4.7](https://huggingface.co/zai-org/GLM-4.7).\n\n**GLM-4.7** is a powerful open-weights model designed for complex reasoning, agentic coding, and tool use. It supports \"Thinking\" (Chain of Thought) natively within its chat template.\n\n**Performances on Benchmarks.** More detailed comparisons of GLM-4.7 with other models GPT-5-High, GPT-5.1-High, Claude Sonnet 4.5, Gemini 3.0 Pro, DeepSeek-V3.2, Kimi K2 Thinking, on 17 benchmarks (including 8 reasoning, 5 coding, and 3 agents benchmarks) can be seen in the below table.\n\n| Benchmark                      | GLM-4.7 | GLM-4.6 | Kimi K2 Thinking | DeepSeek-V3.2 | Gemini 3.0 Pro | Claude Sonnet 4.5 | GPT-5-High | GPT-5.1-High |\n|:-------------------------------|:-------:|:-------:|:----------------:|:-------------:|:--------------:|:-----------------:|:----------:|:------------:|\n| MMLU-Pro                       |  84.3   |  83.2   |       84.6       |     85.0      |      90.1      |       88.2        |    87.5    |     87.0     |\n| GPQA-Diamond                   |  85.7   |  81.0   |       84.5       |     82.4      |      91.9      |       83.4        |    85.7    |     88.1     |\n| HLE                            |  24.8   |  17.2   |       23.9       |     25.1      |      37.5      |       13.7        |    26.3    |     25.7     |\n| HLE (w/ Tools)                 |  42.8   |  30.4   |       44.9       |     40.8      |      45.8      |       32.0        |    35.2    |     42.7     |\n| AIME 2025                      |  95.7   |  93.9   |       94.5       |     93.1      |      95.0      |       87.0        |    94.6    |     94.0     |\n| HMMT Feb. 2025                 |  97.1   |  89.2   |       89.4       |     92.5      |      97.5      |       79.2        |    88.3    |     96.3     |\n| HMMT Nov. 2025                 |  93.5   |  87.7   |       89.2       |     90.2      |      93.3      |       81.7        |    89.2    |      -       |\n| IMOAnswerBench                 |  82.0   |  73.5   |       78.6       |     78.3      |      83.3      |       65.8        |    76.0    |      -       |\n| LiveCodeBench-v6               |  84.9   |  82.8   |       83.1       |     83.3      |      90.7      |       64.0        |    87.0    |     87.0     |\n| SWE-bench Verified             |  73.8   |  68.0   |       71.3       |     73.1      |      76.2      |       77.2        |    74.9    |     76.3     |\n| SWE-bench Multilingual         |  66.7   |  53.8   |       61.1       |     70.2      |       -        |       68.0        |    55.3    |      -       |\n| Terminal Bench Hard            |  33.3   |  23.6   |       30.6       |   35.4   |      39.0      |       33.3        |    30.5    |     43.0     |\n| Terminal Bench 2.0             |  41.0   |  24.5   |       35.7       |     46.4      |      54.2      |       42.8        |    35.2    |     47.6     |\n| BrowseComp                     |  52.0   |  45.1   |        -         |     51.4      |       -        |       24.1        |    54.9    |     50.8     |\n| BrowseComp (w/ Context Manage) |  67.5   |  57.5   |       60.2       |     67.6      |      59.2      |         -         |     -      |      -       |\n| BrowseComp-Zh                  |  66.6   |  49.5   |       62.3       |     65.0      |       -        |       42.4        |    63.0    |      -       |\n| τ²-Bench                       |  87.4   |  75.2   |       74.3       |     85.3      |      90.7      |       87.2        |    82.4    |     82.7     |\n\n\n\n## How to Run (llama.cpp)\n\n**Important:** This model uses \"Thinking\" (Chain of Thought), which consumes significant context. You must increase the generation limit (`-n`) and specify stop tokens to prevent infinite loops.\n\n### 1. CLI Inference (Interactive Chat)\n\n```bash\n./llama-cli -m GLM-4.7.Q4_K_M.gguf \\\n  -n 2048 \\                  # Allow enough tokens for \"Thinking\"\n  -c 8192 \\                  # Adjust context based on VRAM\n  --temp 0.7 \\               # Recommended for reasoning\n  --top-p 0.9 \\\n  -ngl 99 \\                  # Offload layers to GPU (Reduce if OOM)\n  -r \"<|user|>,<|observation|>\" \\  # CRITICAL: Prevents infinite generation loops\n  -cnv \\                     # Enable Conversation Mode\n  -p \"Hello\"\n```\n\n> **Note:** If you want to see the internal \"Thinking\" process (the text between `<think>` tags), add the `--special` flag to the command.\n\n### 2. Server Mode (API)\n\nRunning a persistent server is recommended for this size model to avoid reloading times.\n\n```bash\n./llama-server -m GLM-4.7.Q4_K_M.gguf \\\n  --port 8080 \\\n  -ngl 99 \\\n  -c 8192 \\\n  -n 2048 \\\n  --alias glm4\n```\n\n**API Request Example (JSON):**\n\nWhen using the API, ensure you include the stop tokens in your payload:\n\n```json\n{\n  \"model\": \"glm4\",\n  \"messages\": [\n    { \"role\": \"user\", \"content\": \"Explain quantum computing.\" }\n  ],\n  \"stop\": [\"<|user|>\", \"<|observation|>\"],\n  \"max_tokens\": 2048\n}\n```\n\n### Hardware Requirements\n\n*   **Full GPU Offloading (`-ngl 99`):** Requires ~130GB VRAM for Q4_K_M (e.g., 2x A100 80GB or Mac Studio Ultra).\n*   **Split Offloading:** For single A100 (80GB) cards, use **Q2_K** or **IQ2_XXS** and set `-ngl 40` (adjust based on available VRAM) to split the model between GPU and System RAM.\n**Default Settings (Most Tasks)**\n\n* temperature: `1.0`\n* top-p: `0.95`\n* max new tokens: `131072`\n\nFor multi-turn agentic tasks (τ²-Bench and Terminal Bench 2), please turn on [Preserved Thinking mode](https://docs.z.ai/guides/capabilities/thinking-mode).\n\n\n### CLI Example\n\n```bash\n./llama-cli -m GLM-4.7.Q4_K_M.gguf \\\n  -c 8192 \\\n  --temp 1.0 \\\n  --top-p 0.95 \\\n  -p \"[gMASK]<sop><|system|>\\nYou are a helpful assistant.<|user|>\\nWrite a Python script to calculate Fibonacci numbers.<|assistant|>\\n<think>\" \\\n  -cnv",
    "related_quantizations": []
  },
  "tags": [
    "gguf",
    "text-generation-inference",
    "glm",
    "chat",
    "agents",
    "code",
    "thinking",
    "text-generation",
    "en",
    "zh",
    "base_model:zai-org/GLM-4.7",
    "base_model:quantized:zai-org/GLM-4.7",
    "license:mit",
    "endpoints_compatible",
    "region:us",
    "conversational"
  ],
  "likes": 14,
  "downloads": 192,
  "gated": false,
  "private": false,
  "last_modified": "2025-12-23T05:33:53.000Z",
  "created_at": "2025-12-22T17:49:12.000Z",
  "pipeline_tag": "text-generation",
  "library_name": "gguf"
}

{
  "_id": "69498498d268f0122e8aaa56",
  "id": "AaryanK/GLM-4.7-GGUF",
  "modelId": "AaryanK/GLM-4.7-GGUF",
  "sha": "39166c2598d8026bf7ee7089fb76c2dedff62516",
  "createdAt": "2025-12-22T17:49:12.000Z",
  "lastModified": "2025-12-23T05:33:53.000Z",
  "author": "AaryanK",
  "downloads": 192,
  "likes": 14,
  "gated": false,
  "private": false,
  "pipeline_tag": "text-generation",
  "library_name": "gguf",
  "siblings_count": 9
}