---

license: mit

language:

- en

base_model: deepreinforce-ai/Ornith-1.0-35B

base_model_relation: quantized

tags:

- gguf

- llama.cpp

- mtp

- speculative-decoding

- multi-token-prediction

- draft-mtp

- qwen35moe

- unsloth

pipeline_tag: text-generation

library_name: gguf.llama.cpp

---

Ornith-1.0-35B UD-Q4_K_XL + MTP (GGUF)

This repository contains an experimental MTP-enabled build of Ornith-1.0-35B based on Unsloth's UD-Q4_K_XL GGUF.

The normal Ornith model weights and Unsloth quantization map are preserved. The only model-weight addition is a set of *20 MTP tensors (blk.40.) grafted from the Q4_K_M donor-head package published by skinnyctax. The resulting model can use llama.cpp's draft-mtp self-speculative decoding** without loading a separate draft model.

File

| File | Size | SHA-256 |

|---|---:|---|

| Ornith-1.0-35B-UD-Q4_K_XL-MTP.gguf | 21.29 GiB (~22.87 GB) | 366d7ee1e2322fba81308f7116cd0263e4acde855224d215149cf24970009e65 |

Sources

Base GGUF

• Repository: https://huggingface.co/unsloth/Ornith-1.0-35B-GGUF
• File: Ornith-1.0-35B-UD-Q4_K_XL.gguf
• Direct model page: https://huggingface.co/unsloth/Ornith-1.0-35B-GGUF/blob/main/Ornith-1.0-35B-UD-Q4_K_XL.gguf

The base GGUF contains the complete Ornith model with Unsloth's UD-Q4_K_XL mixed quantization layout and tokenizer metadata.

MTP donor heads

• Repository: https://huggingface.co/skinnyctax/Ornith-1.0-35B-Q6_K-Frankenstein-MTP-GGUF
• Donor file: ornith-1.0-35b-Q4_K_M-MTP-donor-heads.gguf
• Graft script: gguf_mtp_graft.py

The donor file is not a standalone draft model. It contains only the MTP head tensors intended to be grafted into a compatible 40-block qwen35moe GGUF.

How this build was made

The build started from:

Ornith-1.0-35B-UD-Q4_K_XL.gguf

and grafted the Q4_K_M MTP donor heads:

ornith-1.0-35b-Q4_K_M-MTP-donor-heads.gguf

The transformation was performed with the donor repository's gguf_mtp_graft.py logic.

Structural changes

Before grafting, the Unsloth target contained:

53 metadata KV pairs
733 tensors
qwen35moe.block_count = 40

The donor contributed:

20 MTP tensors
blk.40.*

The final GGUF contains:

54 metadata KV pairs
753 tensors
qwen35moe.block_count = 41
qwen35moe.nextn_predict_layers = 1

llama.cpp interprets this as:

n_layer     = 40
n_layer_all = 41

In other words, the original 40 model layers remain the target model and the appended layer is used as the MTP prediction head.

Tokenizer preservation

The tokenizer from the Unsloth base GGUF was preserved intact. Validation showed:

vocab type = BPE
n_vocab    = 248320
n_merges   = 247587

This matters because the standalone donor-head GGUF intentionally does not contain the full tokenizer arrays and therefore cannot be loaded directly as --model-draft by current llama.cpp builds.

Important graft-script fix used for this build

The version of gguf_mtp_graft.py retrieved during this build required a small local correction when re-serializing general.architecture.

The published script stored that value with:

kv_pairs.append((key, 8, arch.encode("utf-8")))

For the GGUF string value to retain its length-prefixed encoding, this build used:

kv_pairs.append((key, 8, enc_str(arch)))

Without that correction, the generated file failed GGUF header parsing in the llama.cpp build used for validation. No model tensor data was changed by this fix; it only corrected serialization of that metadata string during header reconstruction.

Quantization details

This file should be understood as Unsloth UD-Q4_K_XL plus Q4_K_M MTP heads, not as a globally uniform Q4_K_M model.

• The 733 original target tensors retain the quantization types from Unsloth's UD-Q4_K_XL GGUF.
• The 20 appended MTP tensors retain their donor quantization.
• No requantization of the base model was performed during the graft.

The validated final tensor-type summary was:

f32  : 368 tensors
f16  :   2 tensors
q8_0 : 252 tensors
q4_K :  86 tensors
q5_K :  39 tensors
q6_K :   6 tensors

Validation

The final GGUF was validated with:

llama.cpp build: b10331
commit: 7ba604f1c
platform: Windows AMD64
GPU: NVIDIA GeForce RTX 4070

The model successfully loaded with:

• 131072-token context
• --spec-type draft-mtp
• --spec-draft-n-max 3
• Q8_0 target KV cache
• Q8_0 speculative-draft KV cache
• Flash Attention
• partial MoE CPU offload
• the Unsloth mmproj-F16.gguf

llama.cpp successfully created the MTP draft context against the target model, initialized speculative decoding, loaded the multimodal projector, and reached model loaded / listening state.

How to run with llama.cpp

Do not use the donor-head GGUF as --model-draft. The MTP heads are already embedded in this model.

A minimal invocation is:

llama-server \
  --model Ornith-1.0-35B-UD-Q4_K_XL-MTP.gguf \
  --spec-type draft-mtp \
  --spec-draft-n-max 3

A configuration close to the one used during validation is:

llama-server \
  --model Ornith-1.0-35B-UD-Q4_K_XL-MTP.gguf \
  --mmproj mmproj-F16.gguf \
  --spec-type draft-mtp \
  --spec-draft-n-max 3 \
  --host 127.0.0.1 \
  --port 8080 \
  --threads-http 4 \
  --device CUDA0 \
  --n-cpu-moe 36 \
  --gpu-layers 999 \
  --ctx-size 131072 \
  --parallel 1 \
  --flash-attn on \
  --cache-type-k q8_0 \
  --cache-type-v q8_0 \
  --spec-draft-type-k q8_0 \
  --spec-draft-type-v q8_0 \
  --batch-size 4096 \
  --ubatch-size 512 \
  --threads 8 \
  --threads-batch 16 \
  --jinja \
  --load-mode mlock \
  --fit off

For multimodal/vision use, obtain a compatible projector such as Unsloth's mmproj-F16.gguf from the original GGUF repository. The projector is not embedded in this model file.

Performance note

The donor repository reports substantial MTP acceptance and inference-speed improvements for its own Q4_K_M and Q6_K grafted builds. Those measurements should not be assumed to apply directly to this UD-Q4_K_XL build.

This specific file has been functionally validated for correct loading and MTP initialization, but no controlled acceptance-rate or tokens-per-second benchmark is claimed here yet.

Credits

• DeepReinforce AI — Ornith-1.0-35B base model
• Unsloth — GGUF conversion and UD-Q4_K_XL quantization
• skinnyctax — MTP donor-head extraction, grafting approach, and graft script
• llama.cpp — GGUF runtime and draft-mtp speculative decoding support

This repository does not claim authorship of the original model, quantization method, or donor MTP work. It packages the specific UD-Q4_K_XL + MTP graft described above for convenient llama.cpp use.