---

base_model: ornith-ai/Ornith-1.5-35B-A3B

quantized_by: vcruz305

tags:

- gguf

- llama.cpp

- moe

- imatrix

---

Ornith-1.5-35B-A3B — GGUF (1-3 bit IQ ladder)

GGUF quantizations of ornith-ai/Ornith-1.5-35B-A3B, a

sparse MoE model (256 experts, 8 active per token) with hybrid linear/full attention and an MTP

(multi-token-prediction / NextN) head. This repo covers the low-bit IQ ladder from IQ1_S through IQ3_M,

quantized with a mixed-corpus importance matrix for better accuracy at the smallest sizes.

Imatrix

Built with a mixed calibration corpus rather than plain wikitext, to better match this model's actual

use cases and its MoE expert-coverage needs:

• ~45% agentic tool-use data (chat-template-rendered)
• ~35% code data (HumanEval excluded, to avoid contaminating any code-eval benchmarks run against these

quants)

• ~20% wikitext-103 (general language coverage)

~11M characters, tokenized with the model's own chat template, run through llama-imatrix

(-c 1024 -b 1024 -ub 1024 -np 2). Final calibration PPL: 3.9309 +/- 0.00885.

The blk.40 (MTP/NextN) pin

blk.40 is the model's MTP/NextN head. It sits structurally outside the normal forward pass, so no

calibration corpus — however good — can ever give it real importance-matrix coverage (llama-imatrix

never activates it during a forward pass, confirmed in the load logs of every serving attempt).

Quantizing it at the same bit-width as the rest of the ladder fails outright at IQ1_S with

"Missing importance matrix for tensor blk.40.attn_k.weight in a very low-bit quantization".

Every quant in this repo pins blk.40.* to Q8_0 via --tensor-type blk.40=q8_0 regardless of the overall

quant level, and this is verified programmatically after each quantize (every blk.40.* tensor confirmed

Q8_0 or F32 before upload). This adds a small, fixed amount of size to every file but keeps the MTP head

numerically sound at every bit-width in the ladder.

Files

| Quant | Size | Notes |

|-----------|---------|-------|

| IQ1_S | 8.38 GB | Smallest, most aggressive. Expect noticeable quality loss vs FP. |

| IQ1_M | 9.14 GB | |

| IQ2_XXS | 10.4 GB | |

| IQ2_XS | 11.41 GB | |

| IQ2_S | 11.55 GB | |

| IQ2_M | 12.56 GB | |

| IQ3_XXS | 14.52 GB | |

| IQ3_XS | pending | uploading as part of the same ladder run |

| IQ3_S | pending | |

| IQ3_M | pending | Best quality in this ladder, closest to the source BF16. |

(Sizes for IQ3_XS/IQ3_S/IQ3_M will be filled in as each finishes quantizing and uploading.)

Usage

llama-server -m Ornith-1.5-35B-IQ2_M.gguf -ngl 99 -c 8192

Standard llama.cpp builds will load and run these normally; the MTP/NextN tensors are inert unless served

with an MTP-aware fork (--spec-type draft-mtp), in which case they can be used for self-speculative

decoding.

Source

• Base model: ornith-ai/Ornith-1.5-35B-A3B
• Quantized with a custom CUDA build of llama.cpp targeting GB10 (Grace Blackwell, sm_121a)