---

license: apache-2.0

language:

• en
• zh

pipeline_tag: text-generation

library_name: gguf

tags:

• code
• agent
• agentic-coding
• moe
• llama-cpp
• apex-quality
• q4_k_m
• q8_0
• reasoning
• tool-calling
• software-engineering
• mtp
• multi-token-prediction
• conversational

base_model:

• OliviaRossi/Ornith-Qwopus-KAT-Coder-35B-Merged

---

Ornith-Qwopus-KAT-Coder-35B-Merged-GGUF

!Ornith-Qwopus-KAT-Coder-35B merge lineage

!Ornith-Qwopus-KAT-Coder-35B merge lineage

What This Is

Ornith-Qwopus-KAT-Coder-35B is a 35B-parameter Mixture-of-Experts (MoE) coding and agentic-reasoning model, produced by merging two independently strong 35B systems into a single checkpoint and shipping it as quantized GGUF files for local inference with llama.cpp and compatible runtimes (Ollama, LM Studio, Jan, vLLM, SGLang, etc.).

What went into it:

1. OliviaRossi/Qwopus-KAT-Coder-35B-Merged — the first-generation merge of Jackrong/Qwopus3.6-35B-A3B-Coder (multi-step reasoning and logic synthesis) with Kwaipilot/KAT-Coder-V2.5-Dev (autonomous, repository-level tool calling and agentic SWE workflows). This is where the model's MTP speculative-decoding head and its strong terminal/agent tool-use behavior come from.
2. ornith-ai/Ornith-1.5-35B-A3B — a self-improving foundation model trained through a joint task-generation, scaffold-construction, and rollout-optimization loop, contributing markedly stronger long-horizon reasoning, planning, and general instruction-following on top of the same Qwen3.6-35B-A3B backbone.

How it was made: the two parent checkpoints were combined with a tensor-streamed SLERP merge (α = 0.5) — attention, MoE expert, and MTP-head weights are interpolated spherically to preserve the geometry of each parent's learned representations, while normalization and embedding layers use simple LERP to keep the shared vocabulary space stable. The resulting BF16 weights (safetensors repo) were then quantized to GGUF using llama.cpp and a custom mixed-precision APEX scheme (see below).

Why merge these two: Qwopus-KAT-Coder is already tuned for autonomous, tool-calling SWE agents, but its reasoning depth is bounded by its Qwopus/KAT-Coder training data. Ornith-1.5 was trained specifically to push long-horizon reasoning and agentic planning further through self-improvement, and outperforms the base Qwen3.6-35B-A3B it shares a backbone with across coding, reasoning, and agentic benchmarks. Merging the two aims to combine Qwopus-KAT-Coder's proven tool-execution fluency with Ornith-1.5's stronger planning and reasoning, in a single checkpoint that still runs on a single 24 GB consumer GPU.

Self-speculative Multi-Token Prediction and MoE Adaptive Precision (APEX) quantization — see capabilities and quantization details below.

Contents — At a Glance · Executive Summary · MTP Capabilities · The Fusion Architecture · Intended Use & Limitations · Quantization & Hardware Sizing · Quickstart · Sampling · Prompt & Chat Templates · Lineage · Citation · License

---

At a Glance

| | |

| ------------------------------ | ------------------------------------------------------------------------------------------------------------------- |

| Total parameters | ~35B |

| Active parameters / token | ~3B (8 of 256 routed experts + 1 shared expert) |

| Layers | 40 |

| Attention | Hybrid GatedDeltaNet linear attention with periodic full attention |

| Context window | 32,768 tokens native (extendable via YaRN RoPE scaling) |

| Available quantizations | APEX Quality (~21.6 GB) · Q4_K_M (~20.7 GB) · Q8_0 (~35.6 GB) |

| Recommended template | Improved Chat Template for Qwen 3.x |

| Ideal hardware | Single 24 GB GPU (RTX 3090/4090), or 32 GB+ unified memory (Apple Silicon) |

| Languages | English, Chinese |

| License | Apache 2.0 |

↑ back to top

---

Executive Summary

Ornith-Qwopus-KAT-Coder-35B is a Mixture-of-Experts (MoE) coding model built by merging two top-tier 35B systems:

1. OliviaRossi/Qwopus-KAT-Coder-35B-Merged — the first-generation fusion of Qwopus 3.6's deep reasoning/logic synthesis with KAT-Coder V2.5's autonomous SWE tool-calling, carrying an MTP speculative-decoding head.
2. ornith-ai/Ornith-1.5-35B-A3B — a self-improving foundation model trained through joint task-generation, scaffold-construction, and rollout-optimization, delivering strong long-horizon reasoning, agentic tool use, and instruction-following.

The result stacks a third source of reasoning and planning strength on top of the existing Qwopus + KAT-Coder agentic-coding fusion: a private, self-hosted coding agent that still fits on a single 24 GB consumer GPU.

> Quantization Highlight: This repository features APEX Quality quantization — an adaptive mixed-precision MoE layout that preserves critical shared experts and attention blocks at high precision while applying layer-gradient compression across the 256 routed experts, delivering near-Q8 accuracy at roughly a 21.6 GB footprint.

↑ back to top

---

MTP Capabilities

The Qwopus-KAT-Coder parent line ships a Multi-Token Prediction (MTP) speculative head, inherited from Qwen3.6-35B-A3B. This merge preserves the MTP projection layers.

Self-speculative decoding allows the model to draft multiple tokens per forward pass using its own MTP head without requiring an external draft model. Enable it in llama.cpp via --spec-type draft-mtp.

• Realistic speedup: community testing on comparable Qwen3.6-35B-A3B MTP builds reports 1.4–2.2× faster generation with no accuracy loss, depending on prompt predictability and acceptance rates.
• Merge behavior note: ornith-ai/Ornith-1.5-35B-A3B does not ship its own MTP head. Benchmark --spec-type draft-mtp against standard autoregressive decoding on your specific workload before relying on it in production.

↑ back to top

---

The Fusion Architecture

| Highlight | Description |

| ------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |

| Active-parameter efficiency (A3B) | 35B total parameters with ~3B active per token (8 of 256 routed experts + 1 shared expert). Targets 70B-tier coding performance at 3B-tier inference latency. |

| Hybrid GatedDeltaNet linear attention | Linear recurrence layers interleaved with periodic standard self-attention, keeping KV cache growth and memory scaling flat over long context. |

| Triple-lineage expert consolidation | Blends Qwopus's fused expert tensors, KAT-Coder's discrete expert matrices, and Ornith-1.5's reasoning-optimized routed experts into unified expert representations. |

| Reasoning-augmented tool calling | Inherits Ornith-1.5's <think>...</think> reasoning-trace behavior alongside the base model's structured XML/JSON tool-call formatting. |

↑ back to top

---

Intended Use & Limitations

Intended use

• Local, self-hosted coding agent: multi-file repository refactoring, bug fixing, test generation, and autonomous terminal tool execution.
• Long-horizon agentic planning tasks that benefit from Ornith-1.5's reasoning strength combined with KAT-Coder's SWE tool-calling.
• High-performance private code assistant running entirely on local consumer hardware (24 GB VRAM or 32 GB+ Apple Silicon).

Out of scope

• Unreviewed execution in safety-critical production environments without automated validation/sandboxing.
• General-purpose casual chat (the model is strongly optimized for coding, reasoning, and agentic JSON/XML tool calling).
• Vision / multimodal input — this merge line is text-only; no mmproj projector is provided.

↑ back to top

---

Quantization & Hardware Sizing

Available Quantizations

This repository ships three GGUF builds — a flagship mixed-precision build plus two standard uniform quantizations for compatibility and maximum-accuracy use cases:

| Quant Type | File Size | Memory Required (VRAM / RAM) | Precision Strategy | Recommended Setup |

| -------------------- | ------------ | -------------------------------- | ----------------------------------------- | ------------------------------------------------------- |

| APEX-Quality | ~21.6 GB | 24 GB VRAM / 32 GB RAM | Adaptive MoE Gradient (Q8/Q6/Q5/Q4) | ⭐ Recommended (RTX 3090/4090, Mac 32GB+) |

| Q4_K_M | ~20.7 GB | 24 GB VRAM / 32 GB RAM | Standard uniform K-quantization | 24 GB VRAM / Apple Silicon — best for max compatibility |

| Q8_0 | ~35.6 GB | 40 GB+ VRAM / 64 GB RAM | Full 8-bit uniform quantization | A100 / A6000 / Mac Studio (64GB+) — near-lossless |

APEX-Quality is the recommended default: it targets near-Q8_0 accuracy at roughly the same footprint as Q4_K_M by spending precision selectively (see the breakdown below) instead of uniformly. Use plain Q4_K_M if you need a standard, broadly-compatible quant for tooling that doesn't understand the APEX layer scheme; use Q8_0 if you have the VRAM/RAM headroom and want the closest match to the unquantized BF16 weights.

Deep Dive: APEX Quality MoE Architecture

Standard uniform quantization treats all weights equally, which can degrade sparse MoE networks. APEX (Adaptive Precision for EXpert Models) applies mixed-precision quantization mapped to the structural importance of each layer group:

• Shared Experts (shexp) → Q8_0: always active across every token; kept at 8-bit precision to eliminate cumulative outlier degradation.
• *Attention & SSM Projections (attn_qkv, ssm_) → Q6_K**: preserves state-space recurrence and linear attention dynamics over long repositories.
• Embeddings & Output (token_embd, output) → Q8_0 / Q6_K: maintains vocabulary coordinate stability and clean token sampling.
• *256 Routed Experts (ffn__exps) → 3-Tier Layer Gradient**:

- Edge layers (L0–L4 & L35–L39): Q6_K (preserves initial feature extraction and final token routing).

- Near-edge layers (L5–L9 & L30–L34): Q5_K (smooth transition gradient).

- Middle layers (L10–L29): Q4_K / IQ4_XS (aggressive compression across redundant sparse experts).

> Result: APEX Quality tracks uncompressed Q8_0 coding benchmarks closely while fitting entirely within 24 GB VRAM at 30–60+ tokens/sec.

↑ back to top

---

Quickstart Guide

1. Launch with llama.cpp

Interactive CLI (APEX Quality + MTP speculative decoding)

llama-cli \
  -hf OliviaRossi/Ornith-Qwopus-KAT-Coder-35B-Merged-GGUF:Ornith-Qwopus-KAT-Coder-35B-Merged-APEX-Quality.gguf \
  -ngl 99 \
  -c 32768 \
  --spec-type draft-mtp \
  --temp 0.7 --top-p 0.95 --top-k 20 \
  -p "You are an expert autonomous software engineer. Write a production-grade Async Task Queue in Python with a Redis backend."

Local OpenAI-compatible server

llama-server \
  -hf OliviaRossi/Ornith-Qwopus-KAT-Coder-35B-Merged-GGUF:Ornith-Qwopus-KAT-Coder-35B-Merged-APEX-Quality.gguf \
  -ngl 99 \
  -c 32768 \
  --spec-type draft-mtp \
  --temp 0.7 --top-p 0.95 --top-k 20 \
  --port 8080 \
  --host 0.0.0.0

Quick pull with a specific quant tag

llama serve -hf OliviaRossi/Ornith-Qwopus-KAT-Coder-35B-Merged-GGUF:Q4_K_M
llama cli -hf OliviaRossi/Ornith-Qwopus-KAT-Coder-35B-Merged-GGUF:Q4_K_M

2. Ollama Integration (with Custom Modelfile)

ollama run hf.co/OliviaRossi/Ornith-Qwopus-KAT-Coder-35B-Merged-GGUF:Q4_K_M

Or create a Modelfile for finer control:

FROM ./Ornith-Qwopus-KAT-Coder-35B-Merged-APEX-Quality.gguf

TEMPLATE """{{- if .System }}<|im_start|>system
{{ .System }}<|im_end|>
{{ end }}{{- range .Messages }}<|im_start|>{{ .Role }}
{{ .Content }}<|im_end|>
{{ end }}<|im_start|>assistant
"""

PARAMETER stop "<|im_end|>"
PARAMETER stop "<|endoftext|>"
PARAMETER temperature 0.7
PARAMETER top_p 0.95
PARAMETER top_k 20
PARAMETER presence_penalty 1.5
PARAMETER repeat_penalty 1.0
PARAMETER num_ctx 32768

Build and run:

ollama create ornith-qwopus-kat-35b -f Modelfile
ollama run ornith-qwopus-kat-35b

3. Agentic IDE Setup (Cline / Roo-Code / Continue.dev)

{
  "apiProvider": "openai",
  "apiBaseUrl": "http://localhost:8080/v1",
  "apiKey": "local",
  "modelId": "Ornith-Qwopus-KAT-Coder-35B-Merged-APEX-Quality.gguf",
  "contextWindow": 32768,
  "maxTokens": 4096
}

↑ back to top

---

Recommended Sampling Hyperparameters

> Important: Do not use generic ultra-low temperatures (0.1–0.2) or high repetition penalties (>1.05) on this A3B MoE lineage. Doing so causes expert activation collapse and repetitive looping. Use the tested settings below:

temperature: 0.7           # Optimal range is 0.6–0.9 for reasoning + code synthesis
top_p: 0.95
top_k: 20
min_p: 0.00                # Keep at 0.00 to avoid over-filtering sparse expert activations
presence_penalty: 1.5      # Keeps multi-step logic focused and avoids reasoning drift
repeat_penalty: 1.0        # Stay at 1.0 (do not exceed 1.05)
context_window: 32768

For reproducing reasoning-heavy benchmarks inherited from the Ornith-1.5 parent, temperature: 1.0 with top_p: 1.0 may better match the reported eval settings.

↑ back to top

---

Prompt Template & Chat Template Options

⭐ Recommended: Improved Chat Template for Qwen 3.x

For optimal tool-calling accuracy, strict schema adherence, and multi-turn stability, use the standalone Improved Chat Template for Qwen 3.x:

• Repository: OliviaRossi/Improved-Chat-Template-for-Qwen-3.x
• Key Enhancements:

- Fixes tool-call XML/JSON schema formatting bugs in complex multi-file coding workflows.

- Eliminates reasoning/thinking tag leakage into assistant responses.

- Properly formats tool execution returns and error traces across long multi-turn sessions.

# Download template
curl -LO https://huggingface.co/OliviaRossi/Improved-Chat-Template-for-Qwen-3.x/raw/main/chat_template.jinja

# Pass directly to llama-server
llama-server --chat-template-file chat_template.jinja ...

Default Template Structure (ChatML)

<|im_start|>system
You are an autonomous AI coding assistant. You analyze complex codebases, write clean and efficient code, debug issues systematically, and produce complete, working implementations.<|im_end|>
<|im_start|>user
{prompt}<|im_end|>
<|im_start|>assistant

> Note: Because the ornith-ai/Ornith-1.5-35B-A3B parent is a reasoning model, some assistant turns may open with a <think>...</think> block before the final answer. Enable a reasoning parser (e.g. --reasoning-parser qwen3 in vLLM/SGLang, or the equivalent llama.cpp flag) if you want the trace separated from the final response.

↑ back to top

---

Acknowledgments & Lineage

| Role | Model / Resource | Link |

| ---------------------------- | ---------------------------------------------------------| ------------------------------------------------------------------------------------------------------------- |

| Ultimate base architecture | Qwen3.6-35B-A3B (Alibaba Cloud) | — |

| Qwopus base fine-tune | Jackrong/Qwopus3.6-35B-A3B-Coder | huggingface.co |

| KAT-Coder base fine-tune | Kwaipilot/KAT-Coder-V2.5-Dev | huggingface.co |

| First-generation merge | OliviaRossi/Qwopus-KAT-Coder-35B-Merged | huggingface.co |

| Reasoning parent | ornith-ai/Ornith-1.5-35B-A3B | huggingface.co |

| Merged base weights | OliviaRossi/Ornith-Qwopus-KAT-Coder-35B-Merged | huggingface.co |

| GGUF quantizations (this repo)| OliviaRossi/Ornith-Qwopus-KAT-Coder-35B-Merged-GGUF | huggingface.co |

| Recommended chat template | OliviaRossi/Improved-Chat-Template-for-Qwen-3.x | huggingface.co |

| Quantization toolchain | llama.cpp & apex-quant | llama.cpp · apex-quant |

↑ back to top

---

Citation

@misc{ornith-qwopus-kat-coder-35b-merged-gguf,
  title  = {Ornith-Qwopus-KAT-Coder-35B-Merged-GGUF},
  author = {OliviaRossi},
  year   = {2026},
  note   = {SLERP merge of OliviaRossi/Qwopus-KAT-Coder-35B-Merged and ornith-ai/Ornith-1.5-35B-A3B with APEX-Quality MoE quantization},
  url    = {https://huggingface.co/OliviaRossi/Ornith-Qwopus-KAT-Coder-35B-Merged-GGUF}
}

↑ back to top

---

License

Released under the Apache 2.0 license. Usage is also subject to the licenses of the underlying parent models (Jackrong/Qwopus3.6-35B-A3B-Coder, Kwaipilot/KAT-Coder-V2.5-Dev, and ornith-ai/Ornith-1.5-35B-A3B).

Built for developers who want a private, autonomous coding agent running locally on consumer hardware.