ghost-actual/qwen3.5-27b-claude-opus-4.6-distilled-heretic-gguf overview

{
  "metadata": {},
  "card_data": {
    "language": [
      "en"
    ],
    "tags": [
      "qwen3_5",
      "qwen3.5",
      "gguf",
      "llama-cpp",
      "reasoning",
      "chain-of-thought",
      "abliterated",
      "uncensored",
      "heretic",
      "claude"
    ],
    "base_model": [
      "ZonoDilu/Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled"
    ],
    "pipeline_tag": "text-generation",
    "frontmatter": {
      "language": [
        "en"
      ],
      "tags": [
        "qwen3_5",
        "qwen3.5",
        "gguf",
        "llama-cpp",
        "reasoning",
        "chain-of-thought",
        "abliterated",
        "uncensored",
        "heretic",
        "claude"
      ],
      "base_model": [
        "ZonoDilu/Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled"
      ],
      "pipeline_tag": "text-generation"
    },
    "hero_image_url": "",
    "summary": "# Qwen3.5-27B-Claude-Opus-4.6-Distilled-heretic-GGUF GGUF quantizations of ghost-actual/Qwen3.5-27B-Claude-Opus-4.6-Distilled-heretic. Qwen3.5-27B with Claude Opus 4.6 reasoning distillation, abliterated via Heretic.",
    "quick_links": [],
    "benchmark_table_html": "",
    "readme_markdown": "---\nlanguage:\n- en\ntags:\n- qwen3_5\n- qwen3.5\n- gguf\n- llama-cpp\n- reasoning\n- chain-of-thought\n- abliterated\n- uncensored\n- heretic\n- claude\nbase_model:\n- ZonoDilu/Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled\npipeline_tag: text-generation\n---\n# Qwen3.5-27B-Claude-Opus-4.6-Distilled-heretic-GGUF\n\nGGUF quantizations of [ghost-actual/Qwen3.5-27B-Claude-Opus-4.6-Distilled-heretic](https://huggingface.co/ghost-actual/Qwen3.5-27B-Claude-Opus-4.6-Distilled-heretic).\n\nQwen3.5-27B with Claude Opus 4.6 reasoning distillation, abliterated via [Heretic](https://github.com/p-e-w/heretic).\n\n## Available Quants\n\n| Filename | Quant | Size | BPW | Notes |\n|---|---|---|---|---|\n| heretic-27b-Q4_K_M.gguf | Q4_K_M | ~15.4 GB | 4.92 | **Recommended** — best balance of quality and VRAM |\n\n\n## VRAM Requirements (Q4_K_M)\n\n| Context | VRAM | Fits on |\n|---|---|---|\n| 16K | ~18 GB | RTX 3090, 4090, A5000 |\n| 32K | ~19 GB | RTX 3090 Ti, A6000 |\n| 65K | ~21 GB | RTX 3090 Ti, A6000 |\n\nQwen3.5's hybrid DeltaNet architecture means KV cache only covers ~25% of layers. Context scaling is very VRAM-efficient compared to pure transformer models.\n\n## Usage with llama.cpp\n```bash\nllama-server \\\n    -m heretic-27b-Q4_K_M.gguf \\\n    -ngl 99 \\\n    --ctx-size 16384 \\\n    --flash-attn on \\\n    --jinja\n```\n\n### With vision (mmproj)\nBuild the mmproj from the [base model weights](https://huggingface.co/ghost-actual/Qwen3.5-27B-Claude-Opus-4.6-Distilled-heretic):\n```bash\npython convert_hf_to_gguf.py \\\n    ghost-actual/Qwen3.5-27B-Claude-Opus-4.6-Distilled-heretic \\\n    --mmproj --outfile heretic-27b-mmproj-F16.gguf\n\nllama-server \\\n    -m heretic-27b-Q4_K_M.gguf \\\n    --mmproj heretic-27b-mmproj-F16.gguf \\\n    -ngl 99 --ctx-size 16384 --flash-attn on --jinja\n```\n\n### Recommended settings\n```\ntemperature: 0.6\ntop_p: 0.95\ntop_k: 20\npresence_penalty: 1.5\nrepetition_penalty: 1.05\n```\n\n## Abliteration Stats\n\n- **Tool**: Heretic v1.2.0\n- **Refusals**: 13/100\n- **KL Divergence**: 1264\n\n## Architecture\n\nQwen3.5 hybrid Gated DeltaNet + conventional attention. 64 layers in 3:1 pattern. 262K native context. Native multimodal vision. See the [full model card](https://huggingface.co/ghost-actual/Qwen3.5-27B-Claude-Opus-4.6-Distilled-heretic) for details.\n\n## Made by\n\nGhost — [ghost-actual](https://huggingface.co/ghost-actual)",
    "related_quantizations": []
  },
  "tags": [
    "gguf",
    "qwen3_5",
    "qwen3.5",
    "llama-cpp",
    "reasoning",
    "chain-of-thought",
    "abliterated",
    "uncensored",
    "heretic",
    "claude",
    "text-generation",
    "en",
    "base_model:ZonoDilu/Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled",
    "base_model:quantized:ZonoDilu/Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled",
    "endpoints_compatible",
    "region:us",
    "conversational"
  ],
  "likes": 3,
  "downloads": 308,
  "gated": false,
  "private": false,
  "last_modified": "2026-03-08T03:49:12.000Z",
  "created_at": "2026-03-07T19:39:57.000Z",
  "pipeline_tag": "text-generation",
  "library_name": ""
}

{
  "_id": "69ac7f0df44506f6b70d4714",
  "id": "ghost-actual/Qwen3.5-27B-Claude-Opus-4.6-Distilled-heretic-GGUF",
  "modelId": "ghost-actual/Qwen3.5-27B-Claude-Opus-4.6-Distilled-heretic-GGUF",
  "sha": "becaa4f40547761cc7d9e4302356acfa67e38a33",
  "createdAt": "2026-03-07T19:39:57.000Z",
  "lastModified": "2026-03-08T03:49:12.000Z",
  "author": "ghost-actual",
  "downloads": 308,
  "likes": 3,
  "gated": false,
  "private": false,
  "pipeline_tag": "text-generation",
  "library_name": "",
  "siblings_count": 4
}