---

license: mit

license_link: https://huggingface.co/deepreinforce-ai/Ornith-1.0-35B/blob/main/LICENSE

language:

• en

library_name: transformers

base_model_relation: quantized

pipeline_tag: text-generation

base_model:

• deepreinforce-ai/Ornith-1.0-35B

tags:

• qwen
• qwen3
• qwen3.5
• moe
• distillation
• chain-of-thought
• agentic
• tool-use
• chained-distill
• imatrix
• GGUF
• mxfp4
• quantized
• conversational

---

💎 Ornith-1.0-35B-A3B - Custom Mixed Precision GGUFs with Imatrix

> Ornith-1.0, a self-improving family of open-source models specially for agentic coding tasks.\

> Built on top of pretrained Gemma 4 and Qwen 3.5, it achieves state-of-the-art performance among open-source models of comparable size on coding benchmarks.




This repository contains custom, highly optimized, multi-tier mixed precision GGUF weights for deepreinforce-ai/Ornith-1.0-35B.

Ornith-1.0 35B achieves state-of-the-art performance among open-source models of comparable size across a broad range of agentic coding benchmarks.

> [!TIP]

> Highly Recommended: Always keep reasoning/thinking enabled.\

> Ornith thoroughly plans and reasons through code edits before execution, ensuring an efficient and clean output.\

> Unlike baseline Qwen models, which frequently execute blindly and backtrack after generating broken code.

<img style="width: 100%; max-width: 900px;" src="https://deep-reinforce.com/ornith/assets/ornith_35b_eval_202606252204.png" alt="Ornith 1.0 35B Benchmark Results" title="Ornith 1.0 35B Benchmark Results">

To learn more about Ornith 1.0, read their blog post.\

To learn more about how to use Ornith 1.0 35B, view the base model.\

A smaller variant is also available: Ornith-1.0-9B

These quants were generated using manual layer targeting to maximize quality while shrinking the massive VRAM footprint of the Mixture of Experts layers.

📄 GGUF Files

In order of quality:

| Filename | Size | Quants |

| :--- | :--- | :--- |

| Ornith-1.0-35B-A3B-MXFP4_MOE_Q8_0_F16-Imatrix.gguf | 20.7 GB | MXFP4_MOE + Q8_0 + F16 |

| Ornith-1.0-35B-A3B-MXFP4_MOE_Q8_0-Imatrix.gguf | 19.8 GB | MXFP4_MOE + Q8_0 |

| Ornith-1.0-35B-MXFP4_MOE-Only-Imatrix.gguf | 18.5 GB | *MXFP4_MOE Only*** |

Updated 2026-07-08:

- Added Ornith-1.0-35B-MXFP4_MOE-Only-Imatrix.gguf

Updated 2026-07-04:

- Created a bigger imatrix and requantized the models

📊 Importance Matrix (Imatrix)

<details>

<summary>Expand to view datasets & details</summary>

The following datasets were used for the imatrix:

• I am currently remaking this imatrix with 2048 ctx, 800 chunks, with smaller sets of text, and:

- Ornith-generated agent skills markdown text & project documentation markdown text

- Ornith-generated sub-agent prompt+result markdown text

• Old/Current: Custom Target Matrix (2048 ctx with 600 chunks), up to a max of 100MB of each:

- eaddario/imatrix-calibration - tools_huge

- Glint-Research/Fable-5-traces

- lordx64/fable-sft-combined-v2

- angrygiraffe/claude-opus-4.6-4.7-reasoning-8.7k

- nohurry/Opus-4.6-Reasoning-3000x-filtered

- eaddario/imatrix-calibration - code_huge, math_huge

- osunlp/QUEST-RL-Data

- osunlp/QUEST-SFT-Data-Open-ended

- m-a-p/CodeFeedback-Filtered-Instruction

- TuringEnterprises/Rubric-Graded-Reasoning) - Rubric-CS, Rubric-DS

</details>

---

🔍 Precision Matrix & Flavor Variations

Standard global quantization presets (like stock MXFP4_MOE) compress the backbone layers uniformly, which degrades the delicate reasoning capabilities of advanced agent models.\

This repository provides multiple distinct manual configuration layouts to balance precision and memory constraints:

1. The Tri-Quant Hybrid Flavor (MXFP4 + Q8_0 + F16)

Ornith-1.0-35B-A3B-MXFP4_MOE_Q8_0_F16-Imatrix.gguf - Designed for maximum quality preservation, this layout implements a strict 3-Tier Precision Matrix:

• Tier 1 (Core & Mamba Gating - F16 Precision):

- token_embd.weight, output.weight - Protects the critical input/output vocabulary mappings. Dramatically prevents text degradation.

- ssm_alpha, ssm_beta - Protects the integrity of the Mamba state-space calculations across long-range context tokens.

• Tier 2 (Backbone & Shared - Q8_0 Precision): ssm_out, *._shexp - Keeps the attention mechanics, and all trailing shared experts at high quality, to protect the logical research loops.
• Tier 3 (Routed Experts - MXFP4 Precision): ffn_down_exps, ffn_gate_exps, ffn_up_exps - Shrink the massive background expert parameters directly to MXFP4.

2. The Dual-Quant Hybrid Flavor (MXFP4 + Q8_0)

Ornith-1.0-35B-A3B-MXFP4_MOE_Q8_0-Imatrix.gguf - Designed for a slightly leaner memory profile, this layout utilizes 2-Tier Precision:

• Tier 1 (Backbone - Q8_0 Precision): All attention blocks, Mamba structures, vocabulary embeddings, and internal routers use the universal Q8_0 format.
• Tier 2 (Experts - MXFP4 Precision): The heavy sparse expert blocks are target-quantized directly to MXFP4.

3. Bonus Single-Quant (MXFP4)

Ornith-1.0-35B-MXFP4_MOE-Only-Imatrix.gguf - Using only MXFP4, this shrinks the model down to 18.5 GB. The quality is not the best, but it can still do decent work.

• Single Tier (All Layers - MXFP4 Precision): All layers are target-quantized directly to MXFP4, for speed and a low VRAM footprint.

---

📝 Exact Conversion Details

These files were converted via llama-quantize utilizing the following manual recipe parameters:

Convert SafeTensors to GGUF:

# Requires python3.12, with `pip install --upgrade transformers`
python convert_hf_to_gguf.py "Ornith-1.0-35B/" --outtype f16 --outfile "Ornith-1.0-35B_F16.gguf"

Generate Tri-Quant MXFP4_MOE + Q8_0 + F16:

llama-quantize \
  --tensor-type ".*_shexp\.weight=Q8_0" \
  --tensor-type "token_embd\.weight=F16" \
  --tensor-type "^output\.weight=F16" \
  --tensor-type "blk\..*\.(ssm_alpha|ssm_beta)\.weight=F16" \
  --tensor-type "blk\..*\.(ffn_down_exps|ffn_gate_exps|ffn_up_exps)\.weight=MXFP4" \
  --imatrix "imatrix.gguf" \
  "Ornith-1.0-35B_F16.gguf" \
  "Ornith-1.0-35B-A3B-MXFP4_MOE_Q8_0_F16-Imatrix.gguf" \
  Q8_0

Generate Dual-Quant MXFP4_MOE + Q8_0:

llama-quantize \
  --tensor-type ".*_shexp\.weight=Q8_0" \
  --tensor-type "blk\..*\.(ffn_down_exps|ffn_gate_exps|ffn_up_exps)\.weight=MXFP4" \
  --imatrix "imatrix.gguf" \
  "Ornith-1.0-35B_F16.gguf" \
  "Ornith-1.0-35B-A3B-MXFP4_MOE_Q8_0-Imatrix.gguf" \
  Q8_0

Generate Single-Quant MXFP4_MOE:

llama-quantize \
  --tensor-type ".*_shexp\.weight=MXFP4" \
  --tensor-type "token_embd\.weight=MXFP4" \
  --tensor-type "^output\.weight=MXFP4" \
  --tensor-type "blk\..*\.(ssm_alpha|ssm_beta|ssm_out|attn_gate|attn_qkv|ffn_down|ffn_gate|ffn_up|attn_k|attn_q|attn_v|attn_output)\.weight=MXFP4" \
  --tensor-type "blk\..*\.(ffn_down_exps|ffn_gate_exps|ffn_up_exps)\.weight=MXFP4" \
  --imatrix "imatrix.gguf" \
  "Ornith-1.0-35B_F16.gguf" \
  "Ornith-1.0-35B-MXFP4_MOE-Only-Imatrix.gguf" \
  MXFP4_MOE

---

📝 Local Deployment & llama-server Configuration (config.ini)

To maintain the rock-solid reasoning loop and prevent agents from falling into repetitive tool-calling deadlocks, use the following server parameter recommendations (Similar to other Qwen 3.5+ configurations).

# --- Samplers (Dynamic & Expressive) ---
temperature = 0.55
top-k = 15
top-p = 0.90
min-p = 0.12
# --- Penalties (Prevent Syntax & Reasoner Corruption) ---
repeat-penalty = 1.08
presence-penalty = 0.00
# --- DRY Sampler (Protects Indentation & Structural Boilerplate) ---
dry-multiplier = 0.8
dry-base = 1.75
dry-allowed-length = 3
dry-penalty-last-n = 1024
dry-sequence-breaker = ["\n", "```\n", ":", "\t", "\"", "|", "-", "}", "]"]
# --- Enforced Execution Graph ---
samplers = top_k;top_p;min_p;temp;dry

Highly Recommended: Always keep reasoning/thinking enabled, for better quality results.

# --- Reasoning ---
chat-template-kwargs = { "enable_thinking":true }
reasoning = on
reasoning-budget = 20480
reasoning-format = auto

This works well with 256k context window.

> [!TIP]

> For Maximum Quality at 100k+ Context: \

> Use the MXFP4_MOE + Q8_0 + F16 split-quantized version.

> - Preserved at F16: token_embd.weight, output.weight, .ssm_alpha.weight, and .ssm_beta.weight.

> - Why this matters: Keeping these critical layers at full precision prevents the model from dropping fine details during extreme "needle-in-a-haystack" retrieval tasks (large context windows).

> - What to avoid: If output.weight or the embedding layers are quantized to Q8_0 or lower, logit precision rounds off, causing the model to lose accuracy and forget specific details in long-context scenarios.

Updated 2026-07-17:

- Improve sampling settings again (no more looping for me)

Updated 2026-07-08:

- Improve sampling settings to lessen reasoning loops (also faster token-generation due to temp & min-p changes)

---

ℹ️ Misc Details

I'm doing this as a side hobby, with my AMD 5900X, 64GB DDR4, RTX 3060 12GB & RTX 5060 Ti 16GB.

In addition to the above configuration, I also use:

slots = 1
parallel = 1
no-warmup = true

flash-attn = on
mlock = false
no-mmap = false
context-shift = false

batch-size = 2048
ubatch-size = 256

fit = on
fit-target = 768
main-gpu = 0
split-mode = layer
n-gpu-layers = 999
n-cpu-moe = 0
tensor-split = 16,12
override-tensor = (token_embd)=CUDA0,(vision|vpm|nextn)=CPU

fit-ctx = 262144
cache-type-k = q8_0
cache-type-v = q8_0

jinja = true
chat-template = jinja
chat-template-file = chat_template.jinja

For further quality and better ssm behaviour, this configuration can help:

context-shift = false
cache-type-k = f16
cache-type-v = f16

---

🤝 Support the Journey

As a passionate developer, I'm always programming, automating, or experimenting with new ideas.\

I love building open-source tools, trying out new web tech, and creating things that don't yet exist, including local AI & quantizing models.

I love sharing these creations to give back to the community.\

If my projects have saved you time or helped you out, consider supporting my work below!

👉 Support me on Ko-fi

---

✨ Acknowledgments

• deepreinforce-ai for the exceptional Ornith-1.0 base model.

📜 License

Released under MIT.

🔗 Citation

@misc{ornith-35b,
    title = {{Ornith-1.0-35B}: Agentic Coding, Open to All},
    url = {https://deep-reinforce.com/ornith_1_0.html},
    author = {{DeepReinforce Team}},
    year = {2026}
}