khazarai/qwen3-4b-qwen3.6-plus-reasoning-slerp-gguf overview

{
  "metadata": {},
  "card_data": {
    "base_model": [
      "khazarai/Qwen3-4B-Qwen3.6-plus-Reasoning-Slerp"
    ],
    "tags": [
      "mergekit",
      "merge"
    ],
    "license": "apache-2.0",
    "pipeline_tag": "text-generation",
    "language": [
      "en"
    ],
    "datasets": [
      "khazarai/qwen3.6-plus-high-reasoning-500x",
      "khazarai/kimi-2.5-high-reasoning-250x"
    ],
    "frontmatter": {
      "base_model": [
        "khazarai/Qwen3-4B-Qwen3.6-plus-Reasoning-Slerp"
      ],
      "tags": [
        "mergekit",
        "merge"
      ],
      "license": "apache-2.0",
      "pipeline_tag": "text-generation",
      "language": [
        "en"
      ],
      "datasets": [
        "khazarai/qwen3.6-plus-high-reasoning-500x",
        "khazarai/kimi-2.5-high-reasoning-250x"
      ]
    },
    "hero_image_url": "benchmark/Merged_Model.png",
    "summary": "!alt=\"General Benchmark Comparison Chart\" *Note: The sharp drop in \"Creative Writing\" is an expected and accepted trade-off to maximize extreme logical reasoning and coding precision.* This model is a highly experimental and optimized reasoning model created through a surgical SLERP merge of two powerful 4B reasoning models. The goal of this merge was to combine the deep analytical capabilities of Kimi with the mathematical and structural precision of Qwen, while mitigating the catastrophic forgetting commonly seen in SFT model merges. After multiple iterations and layer-by-layer tensor analysis, we achieved a **\"1+1=3 Synergy Effect\"** in Logical Inference and Planning, outperforming both base models and the Qwen Thinking model. ### The \"Golden Path\" (V5) Strategy Standard SLERP merges often destroy RAG capabilities and syntax adherence. To solve this, this model utilizes a custom merge configuration: 1.  **RAG/Vocabulary Fix:** embed_tokens and lm_head are strictly pinned to 1.0 (Qwen). The model reads and speaks purely using Qwen's vocabulary, completely eliminating the RAG degradation problem. 2.  **Gradient Attention:** The intermediate attention and MLP layers follow a smooth gradient [0, 0.1, 0.2, 0.3, 0.5, 0.7, 0.8, 0.9, 1] to prevent weight interference in deep reasoning steps.",
    "quick_links": [],
    "benchmark_table_html": "",
    "readme_markdown": "---\nbase_model:\n- khazarai/Qwen3-4B-Qwen3.6-plus-Reasoning-Slerp\ntags:\n- mergekit\n- merge\nlicense: apache-2.0\npipeline_tag: text-generation\nlanguage:\n- en\ndatasets:\n- khazarai/qwen3.6-plus-high-reasoning-500x\n- khazarai/kimi-2.5-high-reasoning-250x\n---\n\n# khazarai/Qwen3-4B-Qwen3.6-plus-Reasoning-Slerp\n\n\n![alt=\"General Benchmark Comparison Chart\"](benchmark/Merged_Model.png)\n\n*Note: The sharp drop in \"Creative Writing\" is an expected and accepted trade-off to maximize extreme logical reasoning and coding precision.*\n\nThis model is a highly experimental and optimized reasoning model created through a surgical SLERP merge of two powerful 4B reasoning models. The goal of this merge was to combine the deep analytical capabilities of Kimi with the mathematical and structural precision of Qwen, while mitigating the catastrophic forgetting commonly seen in SFT model merges.\nAfter multiple iterations and layer-by-layer tensor analysis, we achieved a **\"1+1=3 Synergy Effect\"** in Logical Inference and Planning, outperforming both base models and the Qwen Thinking model.\n\n\n### The \"Golden Path\" (V5) Strategy\nStandard SLERP merges often destroy RAG capabilities and syntax adherence. To solve this, this model utilizes a custom merge configuration:\n\n1.  **RAG/Vocabulary Fix:** `embed_tokens` and `lm_head` are strictly pinned to `1.0` (Qwen). The model reads and speaks purely using Qwen's vocabulary, completely eliminating the RAG degradation problem.\n2.  **Gradient Attention:** The intermediate attention and MLP layers follow a smooth gradient `[0, 0.1, 0.2, 0.3, 0.5, 0.7, 0.8, 0.9, 1]` to prevent weight interference in deep reasoning steps.\n\n## Benchmark Performance (Multi-Domain Reasoning)\n\n\n| Model | Score |\n| :--- | :--- |\n| **khazarai/Qwen3-4B-Qwen3.6-plus-Reasoning-Slerp** | **77.18** |\n| khazarai/Qwen3-4B-Kimi2.5-Reasoning-Distilled | 76.09 |\n| khazarai/Qwen3-4B-Qwen3.6-plus-Reasoning-Distilled | 75.64 |\n| Qwen/Qwen3-4B-Thinking-2507 | 73.73 |\n\n- **Benchmark**: khazarai/Multi-Domain-Reasoning-Benchmark\n- **Total Questions**: 100\n\n\n## 💡 Intended Use Cases\n\n* **Ideal for:** Complex logical deductions, Python code debugging, mathematical problem-solving, and strict RAG (Retrieval-Augmented Generation) pipelines.\n* **Not recommended for:** Creative writing, poetry, or highly imaginative storytelling.\n\n\n### Models Merged\n\nThe following models were included in the merge:\n* [khazarai/Qwen3-4B-Kimi2.5-Reasoning-Distilled](https://huggingface.co/khazarai/Qwen3-4B-Kimi2.5-Reasoning-Distilled)\n* [khazarai/Qwen3-4B-Qwen3.6-plus-Reasoning-Distilled](https://huggingface.co/khazarai/Qwen3-4B-Qwen3.6-plus-Reasoning-Distilled)\n\n\n### Configuration\n\nThe following YAML configuration was used to produce this model:\n\n```yaml\n\nmodels:\n  - model: khazarai/Qwen3-4B-Kimi2.5-Reasoning-Distilled\n  - model: khazarai/Qwen3-4B-Qwen3.6-plus-Reasoning-Distilled\nmerge_method: slerp\nbase_model: khazarai/Qwen3-4B-Kimi2.5-Reasoning-Distilled\nparameters:\n  t:\n    - filter: embed_tokens\n      value: 1\n      \n    - filter: lm_head\n      value: 1\n\n    - value: 1\n    \n    - filter: self\n      value: [0, 0.1, 0.2, 0.3, 0.5, 0.7, 0.8, 0.9, 1]\n      \ndtype: bfloat16\n```",
    "related_quantizations": []
  },
  "tags": [
    "gguf",
    "mergekit",
    "merge",
    "text-generation",
    "en",
    "dataset:khazarai/qwen3.6-plus-high-reasoning-500x",
    "dataset:khazarai/kimi-2.5-high-reasoning-250x",
    "base_model:khazarai/Qwen3-4B-Qwen3.6-plus-Reasoning-Slerp",
    "base_model:quantized:khazarai/Qwen3-4B-Qwen3.6-plus-Reasoning-Slerp",
    "license:apache-2.0",
    "endpoints_compatible",
    "region:us",
    "conversational"
  ],
  "likes": 3,
  "downloads": 1160,
  "gated": false,
  "private": false,
  "last_modified": "2026-04-13T18:25:58.000Z",
  "created_at": "2026-04-12T01:07:15.000Z",
  "pipeline_tag": "text-generation",
  "library_name": ""
}

{
  "_id": "69daf04371b9db9fb7315504",
  "id": "khazarai/Qwen3-4B-Qwen3.6-plus-Reasoning-Slerp-GGUF",
  "modelId": "khazarai/Qwen3-4B-Qwen3.6-plus-Reasoning-Slerp-GGUF",
  "sha": "7e07a9f8ace681d8fa32270a7537741af82a93c9",
  "createdAt": "2026-04-12T01:07:15.000Z",
  "lastModified": "2026-04-13T18:25:58.000Z",
  "author": "khazarai",
  "downloads": 1160,
  "likes": 3,
  "gated": false,
  "private": false,
  "pipeline_tag": "text-generation",
  "library_name": "",
  "siblings_count": 5
}