wangzhang/qwen3.5-122b-a10b-abliterated-gguf overview

{
  "metadata": {},
  "card_data": {
    "library_name": "llama.cpp",
    "license": "apache-2.0",
    "license_link": "https://huggingface.co/Qwen/Qwen3.5-122B-A10B/blob/main/LICENSE",
    "pipeline_tag": "text-generation",
    "base_model": [
      "wangzhang/Qwen3.5-122B-A10B-abliterated"
    ],
    "tags": [
      "abliterix",
      "uncensored",
      "decensored",
      "abliterated",
      "moe",
      "gguf"
    ],
    "frontmatter": {},
    "hero_image_url": "",
    "summary": "GGUF quantized versions of wangzhang/Qwen3.5-122B-A10B-abliterated, a decensored Qwen/Qwen3.5-122B-A10B created using Abliterix.",
    "quick_links": [],
    "benchmark_table_html": "",
    "readme_markdown": "---\r\nlibrary_name: llama.cpp\r\nlicense: apache-2.0\r\nlicense_link: https://huggingface.co/Qwen/Qwen3.5-122B-A10B/blob/main/LICENSE\r\npipeline_tag: text-generation\r\nbase_model:\r\n- wangzhang/Qwen3.5-122B-A10B-abliterated\r\ntags:\r\n- abliterix\r\n- uncensored\r\n- decensored\r\n- abliterated\r\n- moe\r\n- gguf\r\n---\r\n# Qwen3.5-122B-A10B-abliterated-GGUF\r\n\r\nGGUF quantized versions of [wangzhang/Qwen3.5-122B-A10B-abliterated](https://huggingface.co/wangzhang/Qwen3.5-122B-A10B-abliterated), a decensored [Qwen/Qwen3.5-122B-A10B](https://huggingface.co/Qwen/Qwen3.5-122B-A10B) created using [Abliterix](https://github.com/wuwangzhang1216/abliterix).\r\n\r\n## Available Quantizations\r\n\r\n| File | Quantization | Size | Quality | Use Case |\r\n|------|-------------|------|---------|----------|\r\n| [Q4_K_M](Qwen3.5-122B-A10B-abliterated-Q4_K_M.gguf) | Q4_K_M | 70 GB | High | 1x 80GB GPU or CPU with 96GB+ RAM |\r\n| [Q8_0](Qwen3.5-122B-A10B-abliterated-Q8_0.gguf) | Q8_0 | 121 GB | Very High | 2x 80GB GPUs or CPU with 160GB+ RAM |\r\n\r\n## About the Source Model\r\n\r\n- **95% refusal reduction**: Reduced from 100/100 refusals to just **5/100 (5%)**\r\n- **MoE architecture**: Qwen3.5-122B-A10B activates only ~10B parameters per token — 14B-class speed with 122B-class knowledge\r\n- **Minimal capability loss**: KL divergence of just **0.0878** from the original model\r\n- **50-trial Optuna TPE optimization**: Automated Bayesian hyperparameter search with multi-objective Pareto optimization\r\n- **Orthogonalized abliteration**: Surgical removal of refusal directions without degrading general intelligence\r\n\r\nSee [wangzhang/Qwen3.5-122B-A10B-abliterated](https://huggingface.co/wangzhang/Qwen3.5-122B-A10B-abliterated) for full details on the abliteration process and steering parameters.\r\n\r\n## Usage\r\n\r\n### llama.cpp\r\n\r\n```bash\r\n# Q4_K_M (recommended for single GPU)\r\n./llama-cli -m Qwen3.5-122B-A10B-abliterated-Q4_K_M.gguf -p \"Your prompt here\" -n 512\r\n\r\n# Q8_0 (higher quality)\r\n./llama-cli -m Qwen3.5-122B-A10B-abliterated-Q8_0.gguf -p \"Your prompt here\" -n 512\r\n```\r\n\r\n### llama-server\r\n\r\n```bash\r\n./llama-server -m Qwen3.5-122B-A10B-abliterated-Q4_K_M.gguf --host 0.0.0.0 --port 8080\r\n```\r\n\r\n### Ollama\r\n\r\n```bash\r\n# Create a Modelfile\r\necho \"FROM ./Qwen3.5-122B-A10B-abliterated-Q4_K_M.gguf\" > Modelfile\r\nollama create qwen3.5-122b-abliterated -f Modelfile\r\nollama run qwen3.5-122b-abliterated\r\n```\r\n\r\n## VRAM / RAM Requirements\r\n\r\n| Quantization | Full GPU Offload | Partial Offload (32 layers) | CPU Only |\r\n|-------------|-----------------|---------------------------|----------|\r\n| Q4_K_M | ~74 GB (1x 80GB) | ~40 GB GPU + 40 GB RAM | ~80 GB RAM |\r\n| Q8_0 | ~130 GB (2x 80GB) | ~70 GB GPU + 70 GB RAM | ~140 GB RAM |\r\n\r\n## Disclaimer\r\n\r\nThis model is provided for research purposes only. The creator is not responsible for any misuse.\r\n\r\n## Credits\r\n\r\n- Base model: [Qwen/Qwen3.5-122B-A10B](https://huggingface.co/Qwen/Qwen3.5-122B-A10B) by Alibaba Qwen team\r\n- Abliteration: [wangzhang/Qwen3.5-122B-A10B-abliterated](https://huggingface.co/wangzhang/Qwen3.5-122B-A10B-abliterated)\r\n- Abliteration framework: [Abliterix](https://github.com/wuwangzhang1216/abliterix)\r\n- Quantization: [llama.cpp](https://github.com/ggml-org/llama.cpp)\r\n",
    "related_quantizations": []
  },
  "tags": [
    "llama.cpp",
    "gguf",
    "abliterix",
    "uncensored",
    "decensored",
    "abliterated",
    "moe",
    "text-generation",
    "base_model:wangzhang/Qwen3.5-122B-A10B-abliterated",
    "base_model:quantized:wangzhang/Qwen3.5-122B-A10B-abliterated",
    "license:apache-2.0",
    "endpoints_compatible",
    "region:us",
    "conversational"
  ],
  "likes": 5,
  "downloads": 311,
  "gated": true,
  "private": false,
  "last_modified": "2026-03-30T05:03:02.000Z",
  "created_at": "2026-03-14T20:18:48.000Z",
  "pipeline_tag": "text-generation",
  "library_name": "llama.cpp"
}

{
  "_id": "69b5c2a83809eebb0612d79c",
  "id": "wangzhang/Qwen3.5-122B-A10B-abliterated-GGUF",
  "modelId": "wangzhang/Qwen3.5-122B-A10B-abliterated-GGUF",
  "sha": "62b39fe0c703287c247148359770b7f6f159cc58",
  "createdAt": "2026-03-14T20:18:48.000Z",
  "lastModified": "2026-03-30T05:03:02.000Z",
  "author": "wangzhang",
  "downloads": 311,
  "likes": 5,
  "gated": true,
  "private": false,
  "pipeline_tag": "text-generation",
  "library_name": "llama.cpp",
  "siblings_count": 4
}