---

license: mit

base_model: ornith-ai/Ornith-1.5-9B

base_model_relation: quantized

quantized_by: Avifenesh

pipeline_tag: text-generation

tags:

- nvfp4

- gguf

- memra

- speculative-decoding

- mtp

- conversational

- blackwell

- qwen3_5

model-index:

- name: Ornith-1.5-9B-NVFP4-MTP-GGUF

results:

- task:

type: text-generation

name: MTP speculative decode, greedy K=3, chat-templated probes (exactness harness; spec stream gated token-identical to plain decode)

dataset:

name: memra run-spec probes (deterministic, greedy)

type: memra-probe

metrics:

- name: draft acceptance, embedded grafted MTP head, K=3 (chat-templated code probe)

type: acceptance-rate

value: 0.594

- name: draft acceptance, FR-Spec masked head (top-32768 own-gen SXC ranks), K=3

type: acceptance-rate

value: 0.556

source:

name: memra v0.104.1 run-spec, RTX 5090 Laptop (exactness gates; no throughput claims from this card class)

url: https://github.com/avifenesh/memra

---

Ornith-1.5-9B — NVFP4 GGUF with MTP head (memra serving artifact)

NVFP4 (4-bit e2m1, per-16 FP8-e4m3 scales) GGUF of

ornith-ai/Ornith-1.5-9B, quantized from the

official BF16 safetensors release (token embeddings and output head Q5_K, norms F32).

18.8 GB BF16 → 5.4 GB. Built on request

(discussion)

as the 9B sibling of

Ornith-1.5-35B-A3B-NVFP4-MTP-GGUF.

The MTP head in this file is a graft, and the card says so. The official Ornith-1.5-9B

checkpoint declares an MTP head in its config (mtp_num_hidden_layers: 1) but ships **no

mtp.* tensors — 760 tensors where its base has 775. This artifact takes the trained

MTP head from Qwen/Qwen3.5-9B** (Apache-2.0, the

base model this fine-tune derives from — the two checkpoints are tensor-census-identical

except for those 15 head tensors) and grafts it onto the unchanged official Ornith trunk at

blk.32 (nextn_predict_layers=1). A draft head can never change output — the target

verifies every drafted token — it only moves acceptance and speculative speed. Measured

acceptance with the grafted head is in the tables below; the speculative stream is gated

token-identical to plain greedy decode at every K.

Built as a serving artifact for memra, a

from-scratch Rust + CUDA inference engine for RTX Blackwell (sm_120a) with per-request

exactness gates: speculative, graphed, and batched serving are gated byte-identical to

plain decode.

NVFP4 is not an upstream llama.cpp tensor type: this file runs on memra and on the

NVFP4 branch of avifenesh/llama.cpp.

For upstream-llama.cpp/Ollama use, the official

BF16–Q8_0 GGUFs are the right pick

(they carry no MTP head).

• Engine: https://github.com/avifenesh/memra (MIT, crates.io: memra-server)
• Hosted inference: https://inference.tiyuvta.ai
• Context: 262,144 native; chat template embedded — **byte-identical to the repo's

chat_template.jinja** (sha256 9dd2fbd270fe…44e8fd235; XML tool calling + <think>

reasoning)

• Vocabulary: 248,320 (text→text serving; the upstream vision tower is not in this artifact)

Provenance and verification

| | |

|---|---|

| Trunk | ornith-ai/Ornith-1.5-9B official BF16 safetensors (MIT), unchanged bytes; per-shard sha256 receipts banked at mint |

| MTP head | the 15 mtp. tensors of Qwen/Qwen3.5-9B (Apache-2.0, trained by Qwen for this backbone), grafted at blk.32 through the standard converter mapping; per-tensor sha256 receipts banked at mint. Ornith-1.5-9B itself ships no mtp. weights |

| Conversion | convert_hf_to_gguf.py (avifenesh/llama.cpp nvfp4-imatrix-scale-search), BF16, MTP bundled (qwen35, 33 blocks incl. NextN, rope.dimension_count 64) |

| Quantization | llama-quantize NVFP4 ftype (same branch), --output-tensor-type q5_k --token-embedding-type q5_k: blocks + MTP NVFP4, embeddings/output Q5_K, norms F32 — the same recipe as the 35B sibling |

| Main file | Ornith-1.5-9B-NVFP4-Q5K-mtp.gguf, 5,442,334,016 B, sha256 7a8631ecdffe51f90533f4fa053a6dfb96528d4240a9ed81abd1c0c087c47502 |

| Exactness | memra v0.104.1 batteries, 2026-08-22, RTX 5090 Laptop: kernel-check green · run-spec K=1..8 self-consistency PASS every K (spec ≡ plain greedy, token-identical), raw and chat-templated, embedded and masked head · prefill ≡ decode argmax and batched-prime ≡ tokenwise argmax MATCH on all probes · memra chat render ≡ transformers.apply_chat_template prompt ids on all probes · 48-tok greedy vs BF16 CPU oracle forks only at logit near-ties (top-2 margin ≤ 0.5, rig token = oracle rank-2; an official-Q8_0 control forks from the same oracle within 30–40 tokens on the same probes) · serve surface: greedy chat determinism ×2 byte-identical, tools round-trip emits OpenAI tool_calls with schema-typed arguments ("celsius":true) and zero content leak, per-request usage.spec accounting |

Measured acceptance (exactness harness, not a throughput bank)

Greedy run-spec, memra v0.104.1, RTX 5090 Laptop. Acceptance = accepted/drafted over the

run; the speculative stream is token-identical to plain decode in every row (that is the

gate). This laptop card is thermally limited, so no tok/s numbers are published here —

measure on your own hardware (run-spec, or frspec-owngen --validate).

| head | probe | K=1 | K=2 | K=3 |

|---|---|---|---|---|

| embedded grafted head | chat-templated code prompt, 64 tok | 0.80 | 0.71 | 0.59 |

| embedded grafted head | raw continuation probe, 32 tok | 0.72 | 0.57 | 0.45 |

| masked head (frspec-sxc32768) | chat-templated code prompt, 64 tok | 0.70 | 0.71 | 0.56 |

What ships here, file by file

| file | size | what it is |

|---|---|---|

| Ornith-1.5-9B-NVFP4-Q5K-mtp.gguf | 5.4 GB | The trunk. NVFP4 weights, Q5_K embeddings + full 248,320-row LM head, grafted MTP (NextN) block at blk.32. Serves on its own; every other file in this repo is optional speed. |

| mtp-Ornith-1.5-9B-NVFP4-frspec-sxc32768.gguf | 0.94 GB | Standalone masked MTP draft head, top-32,768 own-generation ranks (agentic-session prompt corpus). |

| ornith15-9b-ranks-sxc-32768.gguf.txt | ~186 KB | Ranks list, plain text: one token id per line, most-frequent first, 32,768 lines. Drives memra's load-time trim (MEMRA_FRSPEC_TRIM) — this is the file that pairs with a trunk that already carries a head. |

| ornith15-9b-ranks-sxc-32768.gguf | ~131 KB | The same ranks as a one-tensor GGUF container (d2t, i32 [32768]). Interchangeable with the .txt anywhere a ranks file is accepted. |

Where the masked head lives, and what the mask does

mtp-Ornith-1.5-9B-NVFP4-frspec-sxc32768.gguf contains:

| tensor | shape / type | role |

|---|---|---|

| output.weight | [4096, 32768] NVFP4 | the masked LM head — 32,768 rows gathered from the trunk's 248,320-row head, in rank order |

| d2t | i32 [32768] | the mask itself: d2t[i] = full-vocab token id of masked row i |

| blk.32.nextn.{eh_proj,enorm,hnorm,shared_head_norm}.weight + blk.32.attn_ / blk.32.ffn_ | — | the MTP (NextN) block, extracted byte-verbatim from the trunk, block requantized Q4_K_M |

| token_embd.weight | [4096, 248320] Q5_K | full-vocab embeddings (drafting reads the trunk's ids) |

"Masked head" means: the draft proposes tokens only from the top-32,768 ids ranked by how

often this model itself emits them (FR-Spec-style d2t trim). The ranked distribution

is 100% model-generated — 138,065 own-generated tokens over real agentic-session

prompts; external text was used as prompts only. topN = 32768 fixes the masked head's

shape — a different topN is a different artifact.

The mask can never change output. Verification runs on the target's full vocabulary, so

a trim moves draft acceptance (speed) and nothing else — run-spec asserts the masked-head

speculative stream token-identical to plain decode, and that gate is PASS on this artifact

(file sha256 6b027b05ba92dfda2f9786a1a970b028b8475345223b9dc85b25d1e51224775b). On the

build's --validate A/B the trim cost 0.5 acceptance pts against the untrimmed embedded

head (0.429 vs 0.434 over the held-out sweep) for a 7.6× smaller draft-head read — verdict

GOOD on the measurement card; end-to-end tok/s decides it on yours.

How to run it (memra)

Install memra per its README (release

installer or cargo build --release). Three ways to draft, all gated byte-identical to

plain decode:

1. Trunk alone — the grafted head is embedded

MEMRA_COMPAT=openai \
MEMRA_MODELS="ornith9b=Ornith-1.5-9B-NVFP4-Q5K-mtp.gguf" \
memra-server

Paths accept hf:owner/repo[:file-substring] specs, which download on first use:

MEMRA_MODELS="ornith9b=hf:Avifenesh/Ornith-1.5-9B-NVFP4-MTP-GGUF:Q5K-mtp" memra-server

2. Trunk + pre-trimmed masked head

MEMRA_MODELS="ornith9b=Ornith-1.5-9B-NVFP4-Q5K-mtp.gguf+mtp-Ornith-1.5-9B-NVFP4-frspec-sxc32768.gguf" \
memra-server

One-off CLI (no server) — run-spec sweeps K and **asserts the speculative stream is

token-identical to plain decode**:

MEMRA_MTP_DRAFT=mtp-Ornith-1.5-9B-NVFP4-frspec-sxc32768.gguf \
run-spec Ornith-1.5-9B-NVFP4-Q5K-mtp.gguf

3. Ranks file only — the trim happens at load

MEMRA_FRSPEC_TRIM=<ranks> takes either the .txt or the ranks .gguf and self-trims at

load: the engine gathers the 32,768 ranked rows from the trunk's own output.weight

bytes (byte-level row gather, zero requant).

MEMRA_FRSPEC_TRIM=ornith15-9b-ranks-sxc-32768.gguf.txt \
MEMRA_MODELS="ornith9b=Ornith-1.5-9B-NVFP4-Q5K-mtp.gguf" \
memra-server

Note on safetensors trunks: the **official Ornith-1.5-9B safetensors checkpoint ships no

MTP head**, so there is nothing there for a ranks file to trim — speculative drafting for

this model comes from this repo's GGUFs.

The attach is a log line, not the absence of an error

A wrong path or flag does not fail — the trunk's embedded full head drafts instead. Check

the boot log:

| you used | the line that proves it |

|---|---|

| +draft in MEMRA_MODELS | [worker] ornith9b: regime draft attached (…frspec-sxc32768.gguf) |

| MEMRA_MTP_DRAFT | [mtp-draft] external draft head: blk.32, source=output.weight, head_vocab=32768 (trimmed, d2t map) |

| MEMRA_FRSPEC_TRIM | [frspec-trim] self-trimmed head: 32768 rows of main output.weight (Q5_K) |

Other runtimes

The trunk is a standard GGUF layout for the qwen35 architecture with a NextN block;

engines that read NextN heads see the grafted MTP out of the box. The masked draft files

carry the d2t tensor layout (see

ggml-org/llama.cpp#25187); outside

memra, draft from the trunk's embedded full head instead of the masked files.

Build your own ranks (and your own masked head)

Rank files are vocab + distribution artifacts of the exact serving model: derive fresh

ranks from the model's own generations for every model and every requant. The tools ship in

the memra repo:

# 1. Ranks from the model's OWN generations (corpus text is prompts only).
frspec-owngen Ornith-1.5-9B-NVFP4-Q5K-mtp.gguf myranks.gguf 32768 my-prompts/
# -> myranks.gguf (d2t container) + myranks.gguf.txt (one id per line)

# 2. Extract the MTP block byte-verbatim, trim the head to your ranks, requantize
#    (NVFP4 head + Q4_K_M block — the measured-best order). Needs python3 + numpy,
#    llama.cpp's gguf-py (MEMRA_GGUFPY) and llama-quantize (MEMRA_QUANTIZE).
tools/make-trimmed-draft.sh Ornith-1.5-9B-NVFP4-Q5K-mtp.gguf myranks.gguf.txt my-draft.gguf 32768

The measured laws

(docs/DRAFT-REGIME.md):

per model, per requant (foreign ranks measured −12 acceptance pts on an identical

tokenizer); chat template ON if you serve chat (a raw-derived rank set once cost −15

acceptance pts); corpus floor ≥ 4× topN own-generated tokens (131,072 for a 32,768

head); validate before trusting (frspec-owngen … --validate A/Bs trimmed vs untrimmed

end-to-end).

Built with convert_hf_to_gguf.py + llama-quantize (NVFP4 branch) + frspec-owngen +

tools/make-trimmed-draft.sh — the same recipe documented above.