aessedai/glm-4.7-gguf overview

{
  "metadata": {},
  "card_data": {
    "base_model": [
      "zai-org/GLM-4.7"
    ],
    "frontmatter": {
      "base_model": [
        "zai-org/GLM-4.7"
      ]
    },
    "hero_image_url": "ppl_ratio_vs_kld.png \"Chart showing PPL vs KLD analysis of quants\"",
    "summary": "This repo contains specialized MoE-quants for GLM-4.7. The idea being that given the huge size of the FFN tensors compared to the rest of the tensors in the model, it should be possible to achieve a better quality while keeping the overall size of the entire model smaller compared to a similar naive quantization. To that end, the quantization type default is kept in high quality (Q8_0 to Q5_K) and the FFN UP + FFN GATE tensors are quanted down along with the FFN DOWN tensors. The mixture convention is as follows: [Default Type]-[FFN_UP]-[FFN_GATE]-[FFN_DOWN], eg: Q8_0-Q4_K-Q4_K-Q5_K. This means: I've mapped these mixes to the closest BPW I could reasonably discern. | Quant | Size | Mixture | PPL | KLD | | :--------- | :--------- | :------- | :------- | :--------- | | Q8_0 | 354.79 GiB (8.50 BPW) | Q8_0 | 8.6821 ± 0.15706 | 0 | | Q5_K_M | 250.15 GiB (6.00 BPW) | Q8_0-Q5_K-Q5_K-Q6_K | 8.6823 ± 0.15710 | 0.01157 ± 0.00068 | | Q4_K_M | 209.77 GiB (5.03 BPW) | Q8_0-Q4_K-Q4_K-Q5_K | 8.7467 ± 0.15845 | 0.01726 ± 0.00058 | | IQ4_XS | 165.28 GiB (3.96 BPW) | Q8_0-IQ3_S-IQ3_S-IQ4_XS | 8.8664 ± 0.16071 | 0.04375 ± 0.00107 | | IQ2_M  | 107.12 GiB (2.57 BPW) | Q5_K-IQ2_XXS-IQ2_XXS-IQ3_XXS | 9.8248 ± 0.17931 | 0.19464 ± 0.00315 | !ppl_ratio_vs_kld",
    "quick_links": [],
    "benchmark_table_html": "",
    "readme_markdown": "---\nbase_model:\n- zai-org/GLM-4.7\n---\nThis repo contains specialized MoE-quants for GLM-4.7. The idea being that given the huge size of the FFN tensors compared to the rest of the tensors in the model, it should be possible to achieve a better quality while keeping the overall size of the entire model smaller compared to a similar naive quantization. To that end, the quantization type default is kept in high quality (Q8_0 to Q5_K) and the FFN UP + FFN GATE tensors are quanted down along with the FFN DOWN tensors.\n\nThe mixture convention is as follows: `[Default Type]-[FFN_UP]-[FFN_GATE]-[FFN_DOWN]`, eg: `Q8_0-Q4_K-Q4_K-Q5_K`. This means:\n- Q8_0 is the default type (attention, shared expert, etc.)\n- Q4_K was used for the FFN_UP and FFN_GATE conditional expert tensors\n- Q5_K was used for the FFN_DOWN conditional expert tensors\n\nI've mapped these mixes to the closest BPW I could reasonably discern.\n\n| Quant | Size | Mixture | PPL | KLD | \n| :--------- | :--------- | :------- | :------- | :--------- |\n| Q8_0 | 354.79 GiB (8.50 BPW) | Q8_0 | 8.6821 ± 0.15706 | 0 |\n| Q5_K_M | 250.15 GiB (6.00 BPW) | Q8_0-Q5_K-Q5_K-Q6_K | 8.6823 ± 0.15710 | 0.01157 ± 0.00068 |\n| Q4_K_M | 209.77 GiB (5.03 BPW) | Q8_0-Q4_K-Q4_K-Q5_K | 8.7467 ± 0.15845 | 0.01726 ± 0.00058 |\n| IQ4_XS | 165.28 GiB (3.96 BPW) | Q8_0-IQ3_S-IQ3_S-IQ4_XS | 8.8664 ± 0.16071 | 0.04375 ± 0.00107 |\n| IQ2_M  | 107.12 GiB (2.57 BPW) | Q5_K-IQ2_XXS-IQ2_XXS-IQ3_XXS | 9.8248 ± 0.17931 | 0.19464 ± 0.00315 |\n\n![ppl_ratio_vs_kld](ppl_ratio_vs_kld.png \"Chart showing PPL vs KLD analysis of quants\")\n\n## Shout Outs\n* [Bartowski's imatrix_v3 corpus](https://gist.github.com/bartowski1182/eb213dccb3571f863da82e99418f81e8)\n* [Ubergarm's ik_llama quants (for SOTA quant schemas)](https://huggingface.co/ubergarm/GLM-4.7-GGUF)",
    "related_quantizations": [
      {
        "id": "timowu/GLM-4.7-IQ4_XS",
        "author": "timowu",
        "downloads": null,
        "likes": null,
        "pipeline_tag": "",
        "library_name": "",
        "tags": []
      }
    ]
  },
  "tags": [
    "gguf",
    "base_model:zai-org/GLM-4.7",
    "base_model:quantized:zai-org/GLM-4.7",
    "endpoints_compatible",
    "region:us",
    "imatrix",
    "conversational"
  ],
  "likes": 12,
  "downloads": 180,
  "gated": false,
  "private": false,
  "last_modified": "2025-12-25T05:49:11.000Z",
  "created_at": "2025-12-22T22:09:05.000Z",
  "pipeline_tag": "",
  "library_name": ""
}

{
  "_id": "6949c1819ff59b2b56891dc2",
  "id": "AesSedai/GLM-4.7-GGUF",
  "modelId": "AesSedai/GLM-4.7-GGUF",
  "sha": "ece6e23521945af7e6dd0ccffecc85165953939e",
  "createdAt": "2025-12-22T22:09:05.000Z",
  "lastModified": "2025-12-25T05:49:11.000Z",
  "author": "AesSedai",
  "downloads": 180,
  "likes": 12,
  "gated": false,
  "private": false,
  "pipeline_tag": "",
  "library_name": "",
  "siblings_count": 27
}