dinerburger/qwen3.5-27b-gguf overview

{
  "metadata": {},
  "card_data": {
    "license": "apache-2.0",
    "base_model": [
      "Qwen/Qwen3.5-27B"
    ],
    "frontmatter": {
      "license": "apache-2.0",
      "base_model": [
        "Qwen/Qwen3.5-27B"
      ]
    },
    "hero_image_url": "",
    "summary": "This is an experimental 4-bit quantization of the dense Qwen3.5-27B, using the unsloth imatrix data, but with the following special rules applied: IQ4_NL script: `` QUANT=\"IQ4_NL\" llama-quantize \\ --output-tensor-type q8_0 \\ --token-embedding-type q8_0 \\ --tensor-type attn_qkv=q8_0 \\ --tensor-type attn_k=bf16 \\ --tensor-type attn_v=bf16 \\ --tensor-type attn_q=q8_0 \\ --tensor-type attn_output=q8_0 \\ --tensor-type attn_gate=q8_0 \\ --tensor-type ssm_ba=bf16 \\ --tensor-type ssm_beta=bf16 \\ --tensor-type ssm_alpha=bf16 \\ --tensor-type ssm_out=q8_0 \\ --imatrix Qwen3.5-27B-imatrix.gguf_file \\ Qwen3.5-27B-BF16-00001-of-00002.gguf \\ Qwen3.5-27B.${QUANT}.gguf \\ ${QUANT} ` IQ4_XS script: ` QUANT=\"IQ4_XS\" llama-quantize \\ --output-tensor-type Q6_K \\ --token-embedding-type Q6_K \\ --tensor-type attn_qkv=q8_0 \\ --tensor-type attn_k=bf16 \\ --tensor-type attn_v=bf16 \\ --tensor-type attn_q=Q6_K \\ --tensor-type attn_output=q8_0 \\ --tensor-type attn_gate=q8_0 \\ --tensor-type ssm_ba=bf16 \\ --tensor-type ssm_beta=bf16 \\ --tensor-type ssm_alpha=bf16 \\ --tensor-type ssm_out=q8_0 \\ --tensor-type ffn_down=Q5_K \\ --imatrix Qwen3.5-27B-imatrix.gguf_file \\ BF16/Qwen3.5-27B-BF16-00001-of-00002.gguf \\ Qwen3.5-27B.${QUANT}.gguf \\ ${QUANT} ` **BONUS TRACK BONUS TRACK** For users of ik_llama.cpp, I've added an iq4_k version as well: ` QUANT=\"iq4_k\" llama-quantize \\ --output-tensor-type iq6_k \\ --token-embedding-type iq6_k \\ --custom-q attn_qkv=iq6_k \\ --custom-q attn_k=bf16 \\ --custom-q attn_v=bf16 \\ --custom-q attn_q=iq6_k \\ --custom-q attn_output=iq6_k \\ --custom-q attn_gate=iq6_k \\ --custom-q ssm_ba=bf16 \\ --custom-q ssm_beta=bf16 \\ --custom-q ssm_alpha=bf16 \\ --custom-q ssm_out=q8_0 \\ --custom-q ffn_down=iq5_k \\ --imatrix Qwen3.5-27B-imatrix.dat \\ BF16/Qwen3.5-27B-BF16-00001-of-00002.gguf \\ Qwen3.5-27B.${QUANT}.ik.gguf \\ ${QUANT} ``",
    "quick_links": [],
    "benchmark_table_html": "",
    "readme_markdown": "---\nlicense: apache-2.0\nbase_model:\n- Qwen/Qwen3.5-27B\n---\n\nThis is an experimental 4-bit quantization of the dense [Qwen3.5-27B](https://huggingface.co/Qwen/Qwen3.5-27B), using the [unsloth imatrix data](https://huggingface.co/unsloth/Qwen3.5-27B-GGUF/blob/main/imatrix_unsloth.gguf_file), but with the following special rules applied:\n\nIQ4_NL script:\n```\nQUANT=\"IQ4_NL\"\nllama-quantize \\\n  --output-tensor-type q8_0 \\\n  --token-embedding-type q8_0 \\\n  --tensor-type attn_qkv=q8_0 \\\n  --tensor-type attn_k=bf16 \\\n  --tensor-type attn_v=bf16 \\\n  --tensor-type attn_q=q8_0 \\\n  --tensor-type attn_output=q8_0 \\\n  --tensor-type attn_gate=q8_0 \\\n  --tensor-type ssm_ba=bf16 \\\n  --tensor-type ssm_beta=bf16 \\\n  --tensor-type ssm_alpha=bf16 \\\n  --tensor-type ssm_out=q8_0 \\\n  --imatrix Qwen3.5-27B-imatrix.gguf_file \\\n  Qwen3.5-27B-BF16-00001-of-00002.gguf \\\n  Qwen3.5-27B.${QUANT}.gguf \\\n  ${QUANT}\n```\n\nIQ4_XS script:\n```\nQUANT=\"IQ4_XS\"\nllama-quantize \\\n  --output-tensor-type Q6_K \\\n  --token-embedding-type Q6_K \\\n  --tensor-type attn_qkv=q8_0 \\\n  --tensor-type attn_k=bf16 \\\n  --tensor-type attn_v=bf16 \\\n  --tensor-type attn_q=Q6_K \\\n  --tensor-type attn_output=q8_0 \\\n  --tensor-type attn_gate=q8_0 \\\n  --tensor-type ssm_ba=bf16 \\\n  --tensor-type ssm_beta=bf16 \\\n  --tensor-type ssm_alpha=bf16 \\\n  --tensor-type ssm_out=q8_0 \\\n  --tensor-type ffn_down=Q5_K \\\n  --imatrix Qwen3.5-27B-imatrix.gguf_file \\\n  BF16/Qwen3.5-27B-BF16-00001-of-00002.gguf \\\n  Qwen3.5-27B.${QUANT}.gguf \\\n  ${QUANT}\n```\n\n**BONUS TRACK BONUS TRACK**\nFor users of ik_llama.cpp, I've added an iq4_k version as well:\n```\nQUANT=\"iq4_k\"\nllama-quantize \\\n  --output-tensor-type iq6_k \\\n  --token-embedding-type iq6_k \\\n  --custom-q attn_qkv=iq6_k \\\n  --custom-q attn_k=bf16 \\\n  --custom-q attn_v=bf16 \\\n  --custom-q attn_q=iq6_k \\\n  --custom-q attn_output=iq6_k \\\n  --custom-q attn_gate=iq6_k \\\n  --custom-q ssm_ba=bf16 \\\n  --custom-q ssm_beta=bf16 \\\n  --custom-q ssm_alpha=bf16 \\\n  --custom-q ssm_out=q8_0 \\\n  --custom-q ffn_down=iq5_k \\\n  --imatrix Qwen3.5-27B-imatrix.dat \\\n  BF16/Qwen3.5-27B-BF16-00001-of-00002.gguf \\\n  Qwen3.5-27B.${QUANT}.ik.gguf \\\n  ${QUANT}\n```",
    "related_quantizations": []
  },
  "tags": [
    "gguf",
    "base_model:Qwen/Qwen3.5-27B",
    "base_model:quantized:Qwen/Qwen3.5-27B",
    "license:apache-2.0",
    "endpoints_compatible",
    "region:us",
    "imatrix",
    "conversational"
  ],
  "likes": 5,
  "downloads": 2230,
  "gated": false,
  "private": false,
  "last_modified": "2026-03-22T12:49:23.000Z",
  "created_at": "2026-02-27T16:47:11.000Z",
  "pipeline_tag": "",
  "library_name": ""
}

{
  "_id": "69a1ca8fb83b7eeb97826e5e",
  "id": "dinerburger/Qwen3.5-27B-GGUF",
  "modelId": "dinerburger/Qwen3.5-27B-GGUF",
  "sha": "3329e638f45f8ede983a0d405131d612db48474d",
  "createdAt": "2026-02-27T16:47:11.000Z",
  "lastModified": "2026-03-22T12:49:23.000Z",
  "author": "dinerburger",
  "downloads": 2230,
  "likes": 5,
  "gated": false,
  "private": false,
  "pipeline_tag": "",
  "library_name": "",
  "siblings_count": 6
}