---

license: mit

license_link: https://huggingface.co/deepreinforce-ai/Ornith-1.0-9B/blob/main/LICENSE

library_name: llama.cpp

pipeline_tag: text-generation

base_model:

• deepreinforce-ai/Ornith-1.0-9B

base_model_relation: quantized

tags:

• gguf
• llama.cpp
• quantized
• nvfp4
• mtp
• speculative-decoding
• qwen3.5
• qwen
• dense
• agentic-coding
• reasoning
• deepreinforce
• orn-1
• ornith
• blackwell
• sm_120
• sm_121
• rtx-6000-pro
• dgx-spark

---

s-batman/Ornith-1.0-9B-NVFP4-MTP-GGUF

NVFP4 quantization of deepreinforce-ai/Ornith-1.0-9B with Multi-Token Prediction (MTP) heads grafted from Qwen3.5-9B-MTP, packaged as a single GGUF for llama.cpp.

Designed for NVIDIA Blackwell GPUs (sm_120 / sm_121) including the RTX PRO 6000 and the DGX Spark (GB10). NVFP4 is dequantized natively by Blackwell tensor cores, and the grafted MTP heads enable draft-mtp speculative decoding for ~5x decode throughput versus the body-only NVFP4 quant on this dense 9B.

Original Model

Ornith-1.0-9B is a self-improving agentic-coding model released by the DeepReinforce team, post-trained via RL on top of Qwen3.5-9B. It is the dense sibling of Ornith-1.0-35B (which is MoE). Both share the Qwen3.5 hybrid-attention trunk (linear SSM + full-attention every 4th layer) and the same reasoning_content block-before-answer output format. Ornith-9B is competitive with Qwen3.5-9B and Gemma 4-9B on Terminal-Bench 2.1, SWE-bench Verified, and Claw-eval.

• Architecture: Qwen3.5 dense (qwen3_5_text), 32 trunk layers + 1 embedded MTP layer (counted as blk.32), hidden_size 4096
• Parameters: 9.2B total / ~9.2B active (dense - all params active per token)
• Vocabulary: 248,320 tokens (multimodal vocab preserved; vision tower not included in this GGUF)
• License: MIT (inherited from upstream)
• Citation: see Citation below

Quantization Details

| Property | Value |

|----------|-------|

| Trunk weights (32 layers) | NVFP4 (GGML type 40) - E4M3, block size 16 |

| MTP heads (1 grafted layer) | NVFP4 (GGML type 40) - 15 tensors total (11 trunk-style + 4 MTP glue), ~125 MB |

| Sensitive tensors | Q8_0 for token_embd / output; F32 for all norms and biases |

| Total size | 9.32 GB (8.50 BPW) |

| Total tensors | 442 (includes 15 blk.32.* tensors that fuse trunk + MTP into one logical block) |

| Source quantization | llama-quantize MXFP4_MOE applied to the BF16 trunk grafted with MTP heads (MXFP4_MOE is the Blackwell-native NVFP4 path; works on dense models despite the name) |

| MTP source | MTP block grafted byte-for-byte from unsloth/Qwen3.5-9B-MTP-GGUF (4 MTP-specific tensors: eh_proj + 3 norms) |

| Conversion tool | llama.cpp build with sm_120 Blackwell support |

What is NVFP4?

NVFP4 is NVIDIA native 4-bit block-floating-point format with E4M3 scale factors, dequantized directly by Blackwell tensor cores. Compared to standard 4-bit integer quants (Q4_K):

• Higher throughput on Blackwell: hardware-native dequantization eliminates integer-to-float conversion overhead
• Lower memory bandwidth: 8.50 BPW overall (heavier than the 35B 4.57 because dense models have many more active parameters per token and benefit less from aggressive quant) vs ~10-12 BPW for an equivalent Q8_0 dense quant
• Preserved quality: 16-element sub-block scaling keeps more information than uniform 4-bit quant

What is MTP and why graft it?

Multi-Token Prediction uses a small auxiliary head to predict multiple tokens ahead. At inference, those predictions become a draft that the main model verifies in a single forward pass - speculative decoding with zero quality loss (output distribution is identical to non-MTP) and significant decode speedup when drafts are accepted.

Ornith-1.0-9B is a Qwen3.5-9B post-trained variant. Its trunk shares the same parameter shapes and base tensor layout as the MTP-enabled Qwen3.5-9B, including the same qwen3_5_text MTP block position (15 tensors at blk.32.*, ~125 MB total). Because the MTP head operates on the same hidden state and embedding space as the trunk, the grafted heads work without any additional training - the MTP drafts the next token given the trunk hidden state, the trunk then verifies. No re-tuning required.

Why 5x speedup on a 9B but only ~1.7x on the 35B MoE? The MTP head is a small extra forward pass over the trunk hidden state - it adds a fixed ~30% decoding cost per token. On the 35B MoE, the trunk is bandwidth-bound and very fast per token (only ~3B active params), so the MTP overhead dominates and limits speedup. On the dense 9B, the trunk is heavier per token (~9.2B active) and bandwidth-limited, so the MTP "drafts" almost pay for themselves - most of the speculative work fills the bandwidth pipe that already saturated by the trunk own decode. The result is a much larger relative win on the dense model.

Acceptance rates measured on RTX PRO 6000:

| Sampling | Accept rate | Speedup vs body-only |

|----------|-------------|----------------------|

| Greedy (temp=0) | ~100% | ~5-6x |

| temp=0.6, top-p=0.95 | 87-93% | ~5x |

| temp=0.9 (production) | 75-86% | ~5x |

Performance

RTX PRO 6000 (Blackwell, sm_120, 97 GB GDDR7, ~1.79 TB/s)

Single-slot, 8k context, temp=0.9, top_p=0.95, top_k=20, min_p=0.01, KV cache q8_0, b=ub=2048:

| Mode | Decode (tok/s) | Prefill (tok/s) | Notes |

|------|----------------|-----------------|-------|

| NVFP4 (body only, no MTP) | ~52 | ~190 | baseline |

| NVFP4 + MTP (n-max=3) ⭐ | 263.6 | 219 | production config |

Smoke test details: 181 generated tokens, 25 input tokens, 85.6% MTP accept rate (131/153 drafted tokens accepted). Reasoning trace verified - model correctly answered "17 x 23?" with 391 and produced a coherent chain-of-thought before the final number.

The 9B at c=8k with 5 slots (c=524288, parallel=5) fits comfortably in the 6000 Pro 97 GB VRAM (~18 GB model + ~10 GB KV/q8_0 + ~5 GB MTP working set), leaving headroom for long-context workloads.

DGX Spark (Blackwell GB10, sm_121, 121 GB unified LPDDR5X, ~273 GB/s)

The DGX Spark unified memory and slower bandwidth make it ideal for memory-bandwidth-bound dense models. Expect ~150-170 tok/s decode at single-slot 8k context with MTP enabled (proportional to the bandwidth ratio vs the 6000 Pro).

Provided Files

| File | Size | Notes |

|------|------|-------|

| ornith-1.0-9b-NVFP4-MTP.gguf | 9.32 GB | The recommended file. Body in NVFP4, MTP head grafted in NVFP4. |

| chat_template.jinja | 7.6 KB | Jinja chat template (inherited from upstream; emits reasoning_content block) |

| config.json | 2.9 KB | Original HF model config (text_config subset) |

| tokenizer.json | 19 MB | HF fast tokenizer |

| tokenizer_config.json | 1.2 KB | Tokenizer metadata |

| vocab.json | 6.7 MB | BPE vocab |

| preprocessor_config.json | 390 B | Image preprocessor (inherited; unused by text-only inference) |

| processor_config.json | 1.2 KB | Multimodal processor (inherited; unused by text-only inference) |

| generation_config.json | 137 B | Generation defaults |

| LICENSE | 1 KB | MIT license text |

| README.md | - | This file |

The video_preprocessor_config.json file is also present but is informational only.

Usage with llama.cpp

Requirements

• llama.cpp build with NVFP4 support, sm_120 / sm_121 CUDA (Blackwell)
• CUDA toolkit with Blackwell support
• Build with -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES="120;121" (set to your GPU compute capability)

Server (recommended for production)

RTX PRO 6000 / single-GPU Blackwell, 5 slots, 512k context:

llama-server \
  -m ornith-1.0-9b-NVFP4-MTP.gguf \
  --host 0.0.0.0 --port 8080 --slots --metrics \
  -t 64 -cb --no-warmup --no-mmap --mlock \
  --jinja -fa on -ctk q8_0 -ctv q8_0 \
  -b 2048 -ub 2048 -c 524288 -np 5 -ngl 99 \
  --chat-template-file chat_template.jinja \
  --spec-type draft-mtp \
  --spec-draft-n-max 3 \
  --temp 0.9 --top-p 0.95 --top-k 20 --min-p 0.01 --repeat-penalty 1.1

CLI

llama-cli \
  -m ornith-1.0-9b-NVFP4-MTP.gguf \
  --chat-template-file chat_template.jinja \
  -p "What is 17 times 23?" \
  -ngl 99 \
  --temp 0.9 --top-p 0.95 --top-k 20 --min-p 0.01 \
  --spec-type draft-mtp --spec-draft-n-max 3

Direct download with llama.cpp

llama-cli \
  --hf-repo s-batman/Ornith-1.0-9B-NVFP4-MTP-GGUF \
  --hf-file ornith-1.0-9b-NVFP4-MTP.gguf \
  -p "What is 17 times 23?"

Important Notes

• Blackwell only. NVFP4 is a hardware-specific format. It will not run efficiently on non-Blackwell GPUs. For AMD, Intel, or pre-Blackwell NVIDIA GPUs, use the upstream deepreinforce-ai/Ornith-1.0-9B-GGUF (Q4_K_M / Q8_0 variants).
• --no-mmap recommended on unified-memory machines (DGX Spark) to avoid mmap-related slowdowns.
• KV cache type q8_0 is the production setting. F16 KV uses ~2x more memory for negligible quality change on long context.
• --reasoning-budget is recommended on production deployments to cap the model tendency to ramble in its reasoning_content block before producing the final answer.
• 9B vs 35B: if your workload is light (single-turn chat, short context) the 35B will give you higher quality. For high-throughput multi-slot workloads (5+ concurrent users, agentic loops) the 9B NVFP4+MTP is a better fit - lower VRAM, higher tokens-per-second per slot, and the MTP speedup is more pronounced on the dense model.

How the MTP graft was made

The MTP heads were transferred byte-for-byte from unsloth/Qwen3.5-9B-MTP-GGUF into the deepreinforce-ai Ornith-1.0-9B BF16 GGUF before quantization:

1. Source GGUF (Unsloth, MTP-enabled Qwen3.5-9B) has 442 tensors and a 4-tensor MTP block at blk.32.nextn.*:

- eh_proj.weight - [8192, 4096] F16 (~64 MB)

- shared_head_norm.weight - [4096] F32

- enorm.weight - [4096] F32

- hnorm.weight - [4096] F32

The MTP block is embedded at blk.32 (the trunk last layer slot) - not appended as a virtual layer like in the 35B MoE case. This means qwen35.block_count=33 (32 trunk layers + 1 fused MTP layer).

1. Destination GGUF (Ornith-1.0-9B BF16) has 427 tensors, qwen35.block_count=32, no MTP tensors. Its last tensor is output_norm.weight.

1. Graft procedure (per-tensor copy):

- Parse source GGUF: discover all blk.32.nextn.* tensors, record each one (name, dims, type, file-relative offset, byte_count).

- Parse destination GGUF: get data-region start, data size, tensor count.

- Build new KV block: copy dst KV verbatim except bump qwen35.block_count 32->33 and append qwen35.nextn_predict_layers=1 (U32). The KV key prefix (qwen35. here, vs qwen35moe. in the 35B) is discovered dynamically by suffix matching.

- Build new tensor index: 427 dst entries (offsets preserved, since dst tensor index is data-region-relative) *plus 15 new blk.32. entries** (the 11 trunk-style tensors at blk.32 plus the 4 MTP glue tensors) with sequential 32-byte-aligned offsets starting after the dst data block.

- Write output: header (24 B) | new KV | new tensor index | padding to 32 B align | dst data block (verbatim) | 15 blk.32 tensors.

Why 15 tensors instead of 4? Initial attempts grafted only the 4 MTP-specific tensors (eh_proj + 3 norms). The loader rejected this with missing tensor blk.32.attn_norm.weight - the MTP head in this dense architecture reuses the trunk blk.32 attention/FFN weights, so the entire blk.32 block (trunk-style tensors + MTP glue) must be present together. This is in contrast to the 35B MoE, where the MTP block is appended after blk.40 as an independent set of 20 tensors.

1. Critical offset gotcha (worth knowing if you re-graft): GGUF tensor offsets are data-region-relative, not file-absolute. To read a tensor payload you MUST seek to (data_start + offset). A bare seek(offset) lands near byte 0 and silently copies trunk tensors - producing a GGUF that loads but has garbage in the MTP slot. Both dst and MTP offsets in the output must be data-region-relative: MTP offsets = dst_data_size + (src_tensor_offset - src_min_offset). The 35B graft script worked by accident because dst_data_start == out_data_start (same alignment); the 9B exposes this with dst_data_start = 10,967,232 vs out_data_start = 10,967,520 (288-byte difference).

1. Byte-level verification: all 15 blk.32 tensors were SHA256-verified after the graft against the source before quantization began. Byte-identical copies.

1. Quantize the merged BF16 GGUF with llama-quantize MXFP4_MOE (= NVFP4 path on Blackwell, despite the name; the underlying GGML type 40 is the same as NVFP4).

Acceptance at ~100% on greedy decoding confirms the graft is functionally correct: when the trunk deterministically predicts token N, the MTP head prediction of token N+1 matches exactly.

The graft script is available on request.

Licensing

This model is licensed under MIT, the same license as the original deepreinforce-ai/Ornith-1.0-9B. See LICENSE for the full text.

The MTP heads originate from Qwen/Qwen3.5-9B (Apache 2.0). The MTP block was sourced via the unsloth/Qwen3.5-9B-MTP-GGUF redistribution, which is permitted under the original Apache 2.0 terms.

Citation

@misc{ornith-9b,
    title = {{Ornith-1.0-9B}: Agentic Coding, Open to All},
    url = {https://deep-reinforce.com/ornith_1_0.html},
    author = {{DeepReinforce Team}},
    year = {2026}
}

@misc{qwen3.5-9b,
    title = {{Qwen3.5-9B}: Hybrid-Attention Dense Language Model},
    url = {https://huggingface.co/Qwen/Qwen3.5-9B},
    author = {{Qwen Team}},
    year = {2026}
}

Acknowledgements

• DeepReinforce Team for training and releasing Ornith-1.0-9B
• Alibaba Qwen Team for the Qwen3.5 base models
• Unsloth for the MTP-enabled GGUF redistribution that made the graft possible
• llama.cpp for NVFP4 support and the draft-mtp speculative decoding implementation