---

base_model: deepreinforce-ai/Ornith-1.0-35B

library_name: gguf

pipeline_tag: text-generation

license: other

license_name: mit-apache-2.0-composite

tags:

- gguf

- llama.cpp

- ornith

- mixture-of-experts

- speculative-decoding

- mtp

- vulkan

- amd

- strix-halo

- cuda

- nvidia

- windows

---

Ornith-1.0-35B MTP Strix Halo Hybrid GGUF

Available optimized builds

| Build | Runtime target | Integrated MTP |

|---|---|---:|

| Full MTP build (this repository) | Current llama.cpp with draft-mtp; patched Vulkan/CUDA deployments | Yes |

| LM Studio compatible build | LM Studio runtimes that reject integrated Qwen3.6/Ornith MTP metadata | No |

The two repositories contain the same selectively optimized 40-layer base.

Choose this repository when the runtime supports integrated MTP. Choose the

second repository for LM Studio compatible, no integrated MTP loading.

Fast Windows local endpoint

For the tested RTX 4060 Laptop 8 GiB + 64 GiB RAM layout, use the complete

Windows endpoint recipe or download and run

run-ornith-mtp-windows.ps1. It starts a

localhost-only OpenAI-compatible API with the exact llama.cpp b10066 image and

the measured CPU-MoE/GPU-dense MTP profile.

.\run-ornith-mtp-windows.ps1 -ModelDir 'C:\Models\Ornith-MTP' -Port 18081

Endpoint: http://127.0.0.1:18081/v1. Do not substitute the no-MTP LM Studio

file if speculative acceleration is required.

The documented flow was re-tested from a fresh C:\Models\Ornith-MTP

directory using a real Hugging Face download (no hard links), SHA-256

verification, Docker startup, /health, /v1/models, and

/v1/chat/completions.

This repository contains a single, deployment-ready GGUF optimized and tested

for batch-1 agent workloads on AMD Ryzen AI MAX+ 395 / Radeon 8060S (gfx1151,

128 GiB UMA) with the Vulkan backend of llama.cpp b9994. The same artifact was

also A/B-tested on a Windows 11 laptop with an 8 GiB NVIDIA RTX 4060 Laptop GPU

using the official CUDA llama.cpp container. The two platforms and runtimes are

reported separately below; server Vulkan results are not mixed with laptop

CUDA results.

File

| File | Size | Quantization | SHA-256 |

|---|---:|---|---|

| ornith-1.0-35b-MTP-graft-down-Q4_0.gguf | 19,377.09 MiB | Q4_K_M with 20 hot ffn_down_exps tensors overridden to Q4_0; integrated MTP layer | 365a7c02dfd320b9696f189d6dc12bd2b0eabb9f8e58ba9fc8cab3af93c0234b |

The base model is deepreinforce-ai/Ornith-1.0-35B.

The compatible MTP-only tensors originate from

a4lg/Qwen3.6-35B-A3B-MTP-ONLY-GGUF.

The combined artifact therefore retains the base model's MIT terms and the

MTP donor's Apache-2.0 terms; review both linked source repositories before

redistribution or commercial use.

What was changed

• grafted the compatible Qwen3.6 35B A3B MTP layer into the Ornith GGUF;
• selectively requantized 20 frequently streamed MoE down-projection tensors

from Q6_K to Q4_0 while retaining the surrounding Q4_K_M plan;

• selected MTP depth 2 and probability floor 0.20 by measured sweep;
• combined verified MTP with exact n-gram drafting for repeated agent/code

traffic;

• retained Q8_0/Q8_0 KV cache after a 64K boundary test showed Q4_0 KV slower.

No weights were trained or fine-tuned. The target model still verifies every

speculative token. Selective requantization does change numerical weights, so

quality was checked end-to-end on the same Orion benchmark slice.

Benchmarks

AMD Strix Halo server (Vulkan)

Hardware: Ryzen AI MAX+ 395, Radeon 8060S Vulkan (gfx1151), 128 GiB UMA,

batch/parallelism 1. Full machine-readable results are in

benchmark-results.json.

| Metric | Archived Ornith Q4_K_M | This hybrid | Change |

|---|---:|---:|---:|

| Weighted decode, identical 43-task Orion run | 61.446 tok/s | 79.661 tok/s | +29.6% |

| Mean request decode | 65.681 tok/s | 83.944 tok/s | +27.8% |

| P50 request decode | 68.45 tok/s | 84.84 tok/s | +23.9% |

| Maximum request decode | 75.03 tok/s | 177.48 tok/s | +136.5% |

| End-to-end wall time | 72m 42s | 36m 11s | -50.2% |

| Aggregate quality | 40.8333 / 43 | 40.8333 / 43 | identical |

On a five-run repeated code stream, the combined MTP + n-gram profile averaged

99.088 tok/s (90.367 minimum, 116.945 maximum). This is workload-dependent,

not a universal 96 tok/s floor. At 62,630 prompt tokens plus 512 generated

tokens, Q8 KV decoded at 70.29 tok/s.

Windows 11 laptop (CUDA): baseline vs this hybrid

This is a same-machine comparison, not a comparison against the server. The

test machine was a MACHENIKE L16P running Windows 11 Pro 24H2

(26100.4652), with an Intel Core i7-13650HX, 63.74 GiB RAM and an NVIDIA

GeForce RTX 4060 Laptop GPU (8,188 MiB, driver 591.74). Docker Desktop ran the

official ghcr.io/ggml-org/llama.cpp:server-cuda image, llama.cpp b10066

(86a9c79f8).

Both GGUF files used the same 16,384-token context, batch 2048, ubatch 512,

parallelism 1, Flash Attention, Q8_0 K/V cache, --n-gpu-layers all,

--cpu-moe and --no-mmap. This 8 GiB layout keeps dense/attention tensors

on the RTX GPU and MoE experts in host RAM. The baseline was

ornith-1.0-35b-Q4_K_M.gguf (SHA-256

ff25291b2599fb927a835e624d2b3540106af61761c3fa57ac4264046dbec002);

the hybrid was the file published in this repository. A naive 15-layer split

was rejected because a realistic 1K prompt fell to about 1.2 tok/s when MoE

expert traffic crossed the CPU/GPU boundary.

Raw /completion runs generated 128 tokens. Prompt rows were repeated twice;

the repeated-code row was repeated three times.

| Laptop profile | 1K decode | 8K decode | Repeated-code decode | 8K prompt processing |

|---|---:|---:|---:|---:|

| Baseline Q4_K_M, no speculation | 5.57 tok/s | 23.90 tok/s | 21.95 tok/s | 341.48 tok/s |

| This hybrid, speculation off | 29.70 tok/s | 29.59 tok/s | 26.75 tok/s | 365.26 tok/s |

| This hybrid, MTP + n-gram | 28.74 tok/s | 66.88 tok/s | 67.23 tok/s | 354.19 tok/s |

The baseline 1K row includes cold expert-cache/thrashing behavior and should

not be read as a universal 5.3x gain. The stable non-speculative hybrid gain

was +23.8% at 8K and +21.8% on repeated code. MTP + n-gram reached 67.23 tok/s

when draft acceptance was high; on less predictable short output it was 3.2%

slower than the hybrid without speculation. Observed draft acceptance was

about 45.5-51.9% for non-repetitive generations and 100% for repeated code.

| Laptop resource peak | Baseline Q4_K_M | Hybrid, no spec | Hybrid, MTP + n-gram |

|---|---:|---:|---:|

| GPU temperature | 71 C | 69 C | 69 C |

| GPU utilization | 98% | 98% | 98% |

| VRAM used | 2,342 MiB | 2,234 MiB | 2,672 MiB |

| GPU power | 49.34 W | 45.80 W | 46.47 W |

| Whole-system RAM used | 44.48 GiB | 42.64 GiB | 42.77 GiB |

The same four seeded Orion coding workspaces were then executed with

approval=auto, a 24-step cap and a 900-second per-task timeout. Deterministic

local verifiers evaluated the resulting files.

| Orion coding scenario | Baseline quality | Hybrid quality | Baseline time | Hybrid time | Hybrid time reduction |

|---|---:|---:|---:|---:|---:|

| Java service coverage | 80%, 8/10 | 80%, 8/10 | 323.9 s | 322.4 s | 0.4% |

| Kafka Node order pipeline | 75%, 6/8 | 75%, 6/8 | 282.6 s | 185.7 s | 34.3% |

| Rabbit retry / DLQ | 75%, 6/8 | 75%, 6/8 | 466.0 s | 324.7 s | 30.3% |

| Kafka Java outbox / idempotency | 80%, 8/10 | 80%, 8/10 | 382.4 s | 313.4 s | 18.1% |

| Total / average | 77.5%, 28/36 | 77.5%, 28/36 | 1,454.9 s | 1,146.2 s | 21.2% |

The Windows laptop A/B therefore measured identical verifier quality and a

1.269x end-to-end speedup (308.7 seconds saved) for the hybrid. Both artifacts

missed the same Orion cli_exit_zero and final_contract checks: their

workspace changes passed identically, but neither produced the harness's

required final changed / verified / not verified / risks contract. This was

not an API, context or infrastructure failure and does not indicate a hybrid

quality regression.

These laptop numbers are intentionally modest in scope: one Windows machine,

16K context, 2/2/3 raw repetitions, 128 generated tokens, and four coding

scenarios. The 60-67 tok/s range is workload-dependent; a realistic production

expectation for ordinary agent traffic is roughly a 20-30% improvement.

Recommended llama-server profile

Use llama.cpp b9994 or a compatible build with MTP support:

GGML_VK_ALLOW_GRAPHICS_QUEUE=1 \
GGML_VK_DISABLE_COOPMAT=1 \
GGML_VK_MAX_NODES_PER_SUBMIT=50 \
llama-server \
  --model ornith-1.0-35b-MTP-graft-down-Q4_0.gguf \
  --alias ornith-1.0-35b \
  --ctx-size 131072 --parallel 1 \
  --batch-size 2048 --ubatch-size 1024 \
  --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 \
  --n-gpu-layers all --device Vulkan0 --no-mmap \
  --spec-type ngram-mod,draft-mtp \
  --spec-draft-n-max 2 --spec-draft-n-min 1 --spec-draft-p-min 0.20 \
  --spec-ngram-mod-n-min 48 --spec-ngram-mod-n-max 64 \
  --spec-ngram-mod-n-match 24

These environment variables and throughput figures are specific to the tested

AMD Strix Halo Vulkan system. Other backends and GPUs require their own sweep.

Validation

• 790 Vulkan backend operation tests passed for every tested K-quant row-merge

value (1, 2, 4, 8; 3,160 test cases total);

• identical aggregate quality on the same 43 tasks;
• Windows 11 / RTX 4060 Laptop A/B: identical 28/36 Orion checks with 21.2%

lower hybrid wall time across the same four coding scenarios;

• GGUF checksum verified before publication;
• 131,072-token server context and OpenAI-compatible API verified in production.

Reproducibility

The source fork, tensor manifest, production Compose profile, benchmark JSON,

and detailed research materials are maintained separately in the author's

private source repository. This Hugging Face repository intentionally contains

only the model, benchmark summary, and this concise deployment card.