---

license: mit

license_link: https://huggingface.co/deepreinforce-ai/Ornith-1.0-9B/blob/main/LICENSE

base_model: deepreinforce-ai/Ornith-1.0-9B

base_model_relation: quantized

pipeline_tag: text-generation

library_name: gguf

tags:

- gguf

- autoround

- intel-autoround

- quantized

- llama.cpp

- code

- agent

---

Ornith-1.0-9B AutoRound GGUF

GGUF quantizations of deepreinforce-ai/Ornith-1.0-9B,

produced with Intel AutoRound using its alg_ext

iterative calibration (~200 optimization steps per block) rather than plain round-to-nearest.

At a given size this recovers noticeably more quality than vanilla GGUF quants, with the largest

benefit at the low bit widths.

Ornith-1.0 is a self-improving, MIT-licensed family of open-source agentic-coding models from

Deep Reinforce; the 9B is the lightweight, single-GPU member (dense, 32 layers, 256K context). See

the original model card for capabilities

and coding benchmarks. This repo only re-packages those weights as GGUF.

Quality vs. size

Every quant was measured against the BF16 reference on wikitext-2 test: PPL and, more

importantly, KL-divergence of the output distribution (how far the quant drifts from full

precision; lower is better). Top-1 is the share of positions whose argmax token still matches

BF16. Reference BF16 PPL = 8.13.

!Quality vs. size on wikitext-2: Mean KLD (top) and PPL (bottom) against bits-per-weight

| Scheme | Size | bpw | PPL | ΔPPL | Mean KLD | Top-1 agree |

|--------|-----:|----:|----:|-----:|---------:|------------:|

| Q2_K_S | 3.7 GB | 3.30 | 9.89 | +21.7% | 0.2960 | 76.8% |

| Q3_K_S | 4.3 GB | 3.81 | 8.53 | +4.9% | 0.1216 | 85.0% |

| Q3_K_M | 4.5 GB | 4.04 | 8.57 | +5.4% | 0.1214 | 84.9% |

| Q3_K_L | 4.9 GB | 4.40 | 8.60 | +5.8% | 0.1201 | 84.9% |

| Q4_K_S | 5.4 GB | 4.78 | 8.05 | -1.0% | 0.0425 | 91.0% |

| Q4_K_M ⭐ | 5.6 GB | 5.03 | 8.08 | -0.6% | 0.0406 | 91.2% |

| Q6_K | 7.4 GB | 6.58 | 8.15 | +0.3% | 0.0040 | 96.6% |

| Q8_0 | 9.5 GB | 8.51 | 8.14 | +0.1% | 0.0020 | 97.5% |

Recommendations

• Q4_K_M ⭐ is the best all-round pick: smaller than the Q5 tier yet closest of the mid-size

quants to BF16 (KLD 0.04, Top-1 91%). Start here.

• Q6_K / Q8_0 are effectively lossless (KLD under 0.004) when you have the VRAM.
• Q3_K_* are usable on tight memory. Reach for Q2_K_S only when you must, since 2-bit shows

noticeable drift.

Why no Q5 or Q2_K_MIXED? On the KLD benchmark the Q5_K quants were not closer to BF16 than

Q4_K_M despite being larger, so they add size without quality for this model and were omitted. The

Q2_K_MIXED variant was defective here (broken output distribution) and was also excluded. (This

base checkpoint ships no MTP/NextN head, so there is no speculative-decoding block to preserve.)

Multimodal

Ornith inherits a vision tower. Multimodal projectors are included for use via llama-mtmd-cli

(pair any LM quant above with one of these). For text-only / coding use you can ignore them.

| Projector | Size | Notes |

|-----------|-----:|-------|

| mmproj-Ornith-1.0-9B-f16.gguf | 0.9 GB | recommended (half size, no visible quality loss) |

| mmproj-Ornith-1.0-9B-bf16.gguf | 0.9 GB | bf16 alternative |

| mmproj-Ornith-1.0-9B-f32.gguf | 1.8 GB | full precision |

Usage (llama.cpp)

# chat / completion (the model's chat template, incl. <think> reasoning, is embedded)
llama-cli   -m Ornith-1.0-9B-Q4_K_M.gguf -ngl 999 -c 8192 -p "Write a Python LRU cache."

# OpenAI-compatible server
llama-server -m Ornith-1.0-9B-Q4_K_M.gguf -ngl 999 -c 8192

# multimodal (image + text)
llama-mtmd-cli -m Ornith-1.0-9B-Q4_K_M.gguf \
               --mmproj mmproj-Ornith-1.0-9B-f32.gguf \
               --image picture.jpg -p "Describe this screenshot."

How these were made

auto-round --model deepreinforce-ai/Ornith-1.0-9B \
           --scheme gguf:q4_k_m --enable_alg_ext --enable_torch_compile \
           --output_dir ./quantized

All outputs were NaN-scanned and inference-checked. The comparison table and the Pareto plot above

(pareto.png, comparison.csv in this repo) come from llama-perplexity --kl-divergence against

the BF16 reference.

Support

These quantized models are made in my spare time using expensive hardware such as DGX Spark systems for quantization and validation. If you find these GGUFs useful for your projects, consider buying me a coffee to help cover hardware and compute costs. Every bit of support helps me keep producing high-quality quantized models for the community!

☕ Support me on Ko-fi

Credits & license

• Base model © Deep Reinforce, MIT (LICENSE).
• Quantization: Intel AutoRound (Apache-2.0) and llama.cpp.