OliviaRossi/Qwopus-KAT-Coder-35B-Merged-GGUF overview

---

license: apache-2.0

language:

• en
• zh

pipeline_tag: text-generation

tags:

• code
• agent
• agentic-coding
• moe
• gguf
• llama-cpp
• apex-quality
• q4_k_m
• q8_0
• reasoning
• tool-calling
• software-engineering
• mtp
• multi-token-prediction
• vision
• multimodal
• image-to-code

base_model:

• OliviaRossi/Qwopus-KAT-Coder-35B-Merged
• Jackrong/Qwopus3.6-35B-A3B-Coder
• Kwaipilot/KAT-Coder-V2.5-Dev

---

<a id="readme-top"></a>

<div align="center">

!Qwopus-KAT-Coder-35B-Merged

!Qwopus-KAT-Coder-35B-Merged






Self-speculative Multi-Token Prediction, native vision input, and MoE Adaptive Precision (APEX) quantization — see capabilities and quantization details below.

</div>

<br>

Contents — At a Glance · Executive Summary · MTP & Vision · Fusion Architecture · Intended Use & Limitations · Quantization & Hardware Sizing · Quickstart · Sampling · Prompt & Chat Templates · Lineage · Citation · License

---

At a Glance

| | |

| :--- | :--- |

| Total parameters | ~35B |

| Active parameters / token | ~3B (8 of 256 routed experts + 1 shared expert) |

| Layers | 40 |

| Attention | Hybrid GatedDeltaNet linear attention with periodic full attention |

| Context window | 32,768+ tokens |

| Flagship Quantization | APEX Quality (~21.3 GB) · Q4_K_M (~20.4 GB) · Q8_0 (~35.2 GB) |

| Recommended Template | Improved Chat Template for Qwen 3.x |

| Ideal hardware | Single 24 GB GPU (RTX 3090/4090), or 32 GB+ unified memory (Apple Silicon) |

| Languages | English, Chinese |

| License | Apache 2.0 |

<p align="right"><a href="#readme-top">↑ back to top</a></p>

---

Executive Summary

Qwopus-KAT-Coder-35B is a Mixture-of-Experts (MoE) coding model built by merging two top-tier 35B fine-tunes:

1. Jackrong/Qwopus3.6-35B-A3B-Coder — deep algorithmic logic synthesis, complex mathematical reasoning, and MTP speculative decoding.
2. Kwaipilot/KAT-Coder-V2.5-Dev — state-of-the-art SWE-agent repository execution, native tool-calling, and automated bug resolution.

The result pairs multi-step reasoning depth with autonomous environment tool execution: a private, self-hosted coding agent that fits on a single 24 GB consumer GPU.

> Quantization Highlight: This repository features APEX Quality quantization — an adaptive mixed-precision MoE layout that preserves critical shared experts and attention blocks at high precision while applying layer-gradient compression across the 256 routed experts, delivering near-Q8 accuracy at a 21.3 GB footprint.

<p align="right"><a href="#readme-top">↑ back to top</a></p>

---

MTP & Vision Capabilities

Both parent lines trace back to Qwen3.6-35B-A3B (Alibaba Cloud), inheriting access to native vision encoding and a multi-token prediction (MTP) speculative head.

Multi-token prediction (MTP)

Self-speculative decoding allows the model to draft multiple tokens per forward pass using its own MTP head without requiring an external draft model. Enable it in llama.cpp via --spec-type draft-mtp.

• Realistic speedup: Community testing on Qwen3.6-35B-A3B MTP builds reports 1.4–2.2× faster generation with no accuracy loss, depending on prompt predictability and acceptance rates.
• Merge behavior note: The Qwopus3.6 parent ships an MTP head, while KAT-Coder-V2.5-Dev does not. The merged weights preserve the MTP projection layers; benchmark --spec-type draft-mtp against standard autoregressive decoding on your specific workload.

Vision / multimodal input

The underlying architecture supports image input for tasks like UI-screenshot-to-code, architecture diagram scaffolding, and GUI bug root-cause analysis.

> Requirement: Vision processing requires a matching multimodal projector (mmproj) file passed via --mmproj <path> (e.g. llama-mtmd-cli or llama-server). The text GGUF alone processes text and code tokens.

<p align="right"><a href="#readme-top">↑ back to top</a></p>

---

The Fusion Architecture

<div align="center">

!Fusion architecture

!Fusion architecture

</div>

| Highlight | Description |

| :--- | :--- |

| Active-parameter efficiency (A3B) | 35B total parameters with ~3B active per token (8 of 256 routed experts + 1 shared expert). Achieves 70B-tier coding performance at 3B-tier inference latency. |

| Hybrid GatedDeltaNet linear attention | Linear recurrence layers interleaved with periodic standard self-attention, keeping KV cache growth and memory scaling flat over long context (32k+ tokens). |

| Dual-format expert consolidation | Unified blending across Qwopus's fused 3D expert tensors and KAT-Coder's discrete expert matrices, combining logic reasoning and tool use into single cohesive expert representations. |

<p align="right"><a href="#readme-top">↑ back to top</a></p>

---

Intended Use & Limitations

Intended use

• Local, self-hosted coding agent: multi-file repository refactoring, bug fixing, test generation, and autonomous terminal tool execution.
• High-performance private code assistant running entirely on local consumer hardware (24 GB VRAM or 32 GB+ Apple Silicon).
• Multimodal UI-to-code workflows (when paired with a compatible mmproj vision file).

Out of scope

• Unreviewed execution in safety-critical production environments without automated validation/sandboxing.
• General-purpose casual chat (the model is strongly optimized for coding, reasoning, and agentic JSON/XML tool calling).

<p align="right"><a href="#readme-top">↑ back to top</a></p>

---

Quantization & Hardware Sizing

Available Quantizations

| Quant Type | File Size | Memory Required (VRAM / RAM) | Precision Strategy | Recommended Setup |

| :--- | :---: | :---: | :--- | :--- |

| APEX-Quality | ~21.3 GB | 24 GB VRAM / 32 GB RAM | Adaptive MoE Gradient (Q8/Q6/Q5/Q4) | ⭐ Recommended Flagship (RTX 3090/4090, Mac 32GB+) |

| Q4_K_M | ~20.4 GB | 24 GB VRAM / 32 GB RAM | Standard uniform K-quantization | 24 GB VRAM / Apple Silicon |

| Q8_0 | ~35.2 GB | 40 GB+ VRAM / 64 GB RAM | Full 8-bit uniform quantization | A100 / A6000 / Mac Studio (64GB+) |

Deep Dive: APEX Quality MoE Architecture

Standard uniform quantization treats all weights equally, which can degrade sparse MoE networks. APEX (Adaptive Precision for EXpert Models) applies mixed-precision quantization mapped to the structural importance of each layer group:

• Shared Experts (shexp) $\rightarrow$ Q8_0: Always active across every token; kept at 8-bit precision to eliminate cumulative outlier degradation.
• *Attention & SSM Projections (attn_qkv, ssm_) $\rightarrow$ Q6_K**: Preserves state-space recurrence and linear attention dynamics over long repositories.
• Embeddings & Output (token_embd, output) $\rightarrow$ Q8_0 / Q6_K: Maintains vocabulary coordinate stability and clean token sampling.
• *256 Routed Experts (ffn__exps) $\rightarrow$ 3-Tier Layer Gradient**:

- Edge Layers (L0–L4 & L35–L39): Q6_K (preserves initial feature extraction and final token routing).

- Near-Edge Layers (L5–L9 & L30–L34): Q5_K (smooth transition gradient).

- Middle Layers (L10–L29): Q4_K / IQ4_XS (aggressive compression across redundant sparse experts).

> Result: APEX Quality matches uncompressed Q8_0 coding benchmarks while fitting entirely within 24 GB VRAM at 30–60+ tokens/sec.

<p align="right"><a href="#readme-top">↑ back to top</a></p>

---

Quickstart Guide

1. Launch with llama.cpp

Interactive CLI (APEX Quality + MTP speculative decoding)

llama-cli \
  -hf OliviaRossi/Qwopus-KAT-Coder-35B-Merged-GGUF:Qwopus-KAT-Coder-35B-Merged-APEX-Quality.gguf \
  -ngl 99 \
  -c 32768 \
  --spec-type draft-mtp \
  --temp 0.7 --top-p 0.95 --top-k 20 \
  -p "You are an expert autonomous software engineer. Write a production-grade Async Task Queue in Python with Redis backend."

Local OpenAI-Compatible Server

llama-server \
  -hf OliviaRossi/Qwopus-KAT-Coder-35B-Merged-GGUF:Qwopus-KAT-Coder-35B-Merged-APEX-Quality.gguf \
  -ngl 99 \
  -c 32768 \
  --spec-type draft-mtp \
  --temp 0.7 --top-p 0.95 --top-k 20 \
  --port 8080 \
  --host 0.0.0.0

> Vision Support: To enable multimodal image input, append --mmproj <path-to-mmproj.gguf> to your llama-server command.

<details>

<summary><b>2. Ollama Integration (with Custom Modelfile)</b></summary>

<br>

Create a Modelfile:

FROM ./Qwopus-KAT-Coder-35B-Merged-APEX-Quality.gguf

TEMPLATE """{{- if .System }}<|im_start|>system
{{ .System }}<|im_end|>
{{ end }}{{- range .Messages }}<|im_start|>{{ .Role }}
{{ .Content }}<|im_end|>
{{ end }}<|im_start|>assistant
"""

PARAMETER stop "<|im_end|>"
PARAMETER stop "<|endoftext|>"
PARAMETER temperature 0.7
PARAMETER top_p 0.95
PARAMETER top_k 20
PARAMETER presence_penalty 1.5
PARAMETER repeat_penalty 1.0
PARAMETER num_ctx 32768

Build and run:

ollama create qwopus-kat-35b -f Modelfile
ollama run qwopus-kat-35b

</details>

<details>

<summary><b>3. Agentic IDE Setup (Cline / Roo-Code / Continue.dev)</b></summary>

<br>

Point your IDE agent extension directly to your local llama-server:

{
  "apiProvider": "openai",
  "apiBaseUrl": "http://localhost:8080/v1",
  "apiKey": "local",
  "modelId": "Qwopus-KAT-Coder-35B-Merged-APEX-Quality.gguf",
  "contextWindow": 32768,
  "maxTokens": 4096
}

</details>

<p align="right"><a href="#readme-top">↑ back to top</a></p>

---

Recommended Sampling Hyperparameters

> Important: Do not use generic ultra-low temperatures (0.1–0.2) or high repetition penalties (>1.05) on Qwen3.6-35B-A3B MoE models. Doing so causes expert activation collapse and repetitive looping. Use the tested settings below:

temperature: 0.7           # Optimal range is 0.6–0.9 for reasoning + code synthesis
top_p: 0.95
top_k: 20
min_p: 0.00                # Keep at 0.00 to avoid over-filtering sparse expert activations
presence_penalty: 1.5       # Keeps multi-step logic focused and avoids reasoning drift
repeat_penalty: 1.0         # Stay at 1.0 (do not exceed 1.05)
context_window: 32768

<p align="right"><a href="#readme-top">↑ back to top</a></p>

---

Prompt Template & Chat Template Options

⭐ Recommended: Improved Chat Template for Qwen 3.x

For optimal tool-calling accuracy, strict schema adherence, and multi-turn stability, use the standalone Improved Chat Template for Qwen 3.x:

• Repository: OliviaRossi/Improved-Chat-Template-for-Qwen-3.x
• Key Enhancements:

- Fixes tool-call XML/JSON schema formatting bugs in complex multi-file coding workflows.

- Eliminates reasoning/thinking tag leakage into assistant responses.

- Properly formats tool execution returns and error traces across long multi-turn sessions.

To use it with llama-server or vLLM:

# Download template
curl -LO https://huggingface.co/OliviaRossi/Improved-Chat-Template-for-Qwen-3.x/raw/main/chat_template.jinja

# Pass directly to llama-server
llama-server --chat-template-file chat_template.jinja ...

---

Default Template Structure (ChatML)

<|im_start|>system
You are an autonomous AI coding assistant. You analyze complex codebases, write clean and efficient code, debug issues systematically, and produce complete, working implementations.<|im_end|>
<|im_start|>user
{prompt}<|im_end|>
<|im_start|>assistant

<p align="right"><a href="#readme-top">↑ back to top</a></p>

---

Acknowledgments & Lineage

| Role | Model / Resource | Link |

| :--- | :--- | :--- |

| Ultimate base architecture | Qwen3.6-35B-A3B (Alibaba Cloud) | — |

| Qwopus base fine-tune | Jackrong/Qwopus3.6-35B-A3B-Coder | huggingface.co |

| KAT-Coder base fine-tune | Kwaipilot/KAT-Coder-V2.5-Dev | huggingface.co |

| Merged base weights | OliviaRossi/Qwopus-KAT-Coder-35B-Merged | huggingface.co |

| GGUF Quantizations | OliviaRossi/Qwopus-KAT-Coder-35B-Merged-GGUF | huggingface.co |

| Recommended Chat Template | OliviaRossi/Improved-Chat-Template-for-Qwen-3.x | huggingface.co |

| Quantization Toolchain | llama.cpp & apex-quant | llama.cpp · apex-quant |

<p align="right"><a href="#readme-top">↑ back to top</a></p>

---

Citation

@misc{qwopus-kat-coder-35b-merged,
  title  = {Qwopus-KAT-Coder-35B-Merged},
  author = {OliviaRossi},
  year   = {2026},
  note   = {SLERP merge of Jackrong/Qwopus3.6-35B-A3B-Coder and Kwaipilot/KAT-Coder-V2.5-Dev with APEX-Quality MoE quantization},
  url    = {https://huggingface.co/OliviaRossi/Qwopus-KAT-Coder-35B-Merged-GGUF}
}

<p align="right"><a href="#readme-top">↑ back to top</a></p>

---

License

Released under the Apache 2.0 license. Usage is also subject to the licenses of the underlying parent models.

<div align="center">

Built for developers who want a private, autonomous coding agent running locally on consumer hardware.

</div>