---

license: other

license_name: aeon-ultimate

license_link: https://huggingface.co/AEON-7/Ornith-1.0-35B-AEON-Ultimate-Uncensored-BF16

base_model: AEON-7/Ornith-1.0-35B-AEON-Ultimate-Uncensored-BF16

tags:

- gguf

- llama-cpp

- quantized

- qwen3

- moe

- mixture-of-experts

- coding

- reasoning

- mtp

- speculative-decoding

model_type: qwen3_moe

quantized_by: vcruz305

pipeline_tag: text-generation

---

Ornith 1.0 35B AEON Ultimate Uncensored - GGUF

GGUF quantizations of AEON-7/Ornith-1.0-35B-AEON-Ultimate-Uncensored-BF16, produced from the BF16 source weights using importance-matrix calibration.

Two variants are provided: standard trunk (no MTP) and MTP-grafted (with Multi-Token Prediction block for speculative decoding).

Files

Standard Trunk (no MTP)

For standard autoregressive inference. Smaller files, no speculative decoding overhead.

| File | Quant | Size | BPW | Target GPU |

|------|-------|------|-----|------------|

| ornith-aeon-35b-Q8_0.gguf | Q8_0 | 35 GB | ~8.5 | 48GB+ (near-lossless) |

| ornith-aeon-35b-Q6_K.gguf | Q6_K | 27 GB | ~6.6 | 32GB+ (high quality) |

| ornith-aeon-35b-Q5_K_M.gguf | Q5_K_M | 24 GB | ~5.7 | 24GB (quality-first) |

| ornith-aeon-35b-Q4_K_M.gguf | Q4_K_M | 21.2 GB | ~4.8 | 24GB (balanced) |

| ornith-aeon-35b-Q4_K_S.gguf | Q4_K_S | 19.9 GB | ~4.6 | 24GB fallback |

| ornith-aeon-35b-IQ4_XS.gguf | IQ4_XS | 18.7 GB | ~4.3 | 20GB (RTX 4000 Ada) |

| ornith-aeon-35b-Q3_K_M.gguf | Q3_K_M | 16.8 GB | ~3.9 | 16GB GPUs |

| ornith-aeon-35b-Q2_K.gguf | Q2_K | 12.9 GB | ~3.0 | 12GB GPUs |

| ornith-aeon-35b-IQ1_M.gguf | IQ1_M | 8.2 GB | ~1.8 | 8GB GPUs |

| ornith-aeon-35b.imatrix | - | 184 MB | - | Importance matrix |

MTP-Grafted (with Multi-Token Prediction)

These GGUFs contain all 785 MTP tensors grafted from the base Qwen/Qwen3.5-35B-A3B model, adding a full MTP prediction block (blk.40) with 256 MoE experts. Use with --spec-type draft-mtp for speculative decoding. MTP is bundled in the GGUF -- no separate draft file needed.

| File | Quant | Size | BPW | Target GPU |

|------|-------|------|-----|------------|

| ornith-aeon-35b-MTP-Q8_0.gguf | Q8_0 | 36 GB | ~8.5 | 48GB+ (near-lossless) |

| ornith-aeon-35b-MTP-Q6_K.gguf | Q6_K | 28 GB | ~6.6 | 32GB+ (high quality) |

| ornith-aeon-35b-MTP-Q5_K_M.gguf | Q5_K_M | 24 GB | ~5.7 | 24GB (quality-first) |

| ornith-aeon-35b-MTP-Q4_K_M.gguf | Q4_K_M | 21.7 GB | ~4.8 | 24GB (balanced) |

| ornith-aeon-35b-MTP-Q4_K_S.gguf | Q4_K_S | 20.4 GB | ~4.6 | 24GB fallback |

| ornith-aeon-35b-MTP-IQ4_XS.gguf | IQ4_XS | 19.2 GB | ~4.3 | 20-24GB |

| ornith-aeon-35b-MTP-Q3_K_M.gguf | Q3_K_M | 17.2 GB | ~3.9 | 16GB GPUs |

| ornith-aeon-35b-MTP-Q2_K.gguf | Q2_K | 13.3 GB | ~3.0 | 12GB GPUs |

| ornith-aeon-35b-MTP.imatrix | - | 184 MB | - | Importance matrix |

Why Two Variants?

The original AEON fine-tune lost its 785 MTP weight tensors. HuggingFace Transformers' AutoModelForCausalLM silently drops all mtp. tensors during loading (_keys_to_ignore_on_load_unexpected = [r"^mtp."]). The mtp_num_hidden_layers: 1 in config.json is orphaned metadata from the base Qwen3.5-35B-A3B model.

The MTP-grafted variants restore all 785 MTP tensors (~488 MB BF16) by copying them from the base Qwen3.5-35B-A3B model. This works because Ornith shares identical architecture, hidden dimensions, expert count, and embedding space with its base model. Community-measured acceptance rates for grafted MTP: 58-100%.

MTP Benchmark Results

Tested: MTP-Q4_K_M on DGX Spark (GB10)

| Mode | Prompt | Generation |

|------|--------|------------|

| Standard (no MTP) | 47.4 t/s | 34.9 t/s |

| Embedded --spec-type draft-mtp | 46.2 t/s | 62.8 t/s |

1.8x generation speedup with the bundled MTP head on a single GPU.

Embedded MTP vs separate draft model

An important distinction for MoE speculative decoding:

• Embedded MTP head (bundled in the GGUF, --spec-type draft-mtp): Net positive. The MTP head shares the model's KV cache and embedding space. Measured +27-80% generation speedup depending on workload and hardware.
• Separate draft model (-md with a standalone GGUF): Net negative for MoE. A separate draft model triggers expert-union overhead during batch verification -- more expert weight blocks must be read from VRAM, exceeding the savings. Benchmarks show -18% to -52% regression.

The MTP-grafted GGUFs in this repo use the embedded approach.

How to Run

Standard inference (recommended for most users)

llama-server \
  -m ornith-aeon-35b-Q4_K_M.gguf \
  -a ornith-aeon-35b \
  --host 0.0.0.0 --port 8083 \
  -ngl 99 \
  --n-cpu-moe 3 \
  -fa on \
  -ctk q4_0 -ctv q4_0 \
  -c 32768 \
  -b 2048 -ub 768 \
  -np 1 -cb \
  -n 16384 \
  --temp 0.6 --top-k 20 --top-p 0.95 \
  --repeat-penalty 1.1 \
  --jinja \
  --reasoning-format deepseek \
  --reasoning-budget 1024

With MTP speculative decoding (faster generation)

Requires llama.cpp built from latest master (b9606+). MTP is bundled in the GGUF -- no separate draft file needed.

llama-server \
  -m ornith-aeon-35b-MTP-Q4_K_M.gguf \
  -a ornith-aeon-35b \
  --spec-type draft-mtp \
  --host 0.0.0.0 --port 8083 \
  -ngl 99 \
  --n-cpu-moe 3 \
  -fa on \
  -ctk q4_0 -ctv q4_0 \
  -c 32768 \
  -b 2048 -ub 768 \
  --parallel 1 \
  --temp 0.6 --top-k 20 --top-p 0.95 \
  --repeat-penalty 1.1 \
  --jinja \
  --reasoning-format deepseek \
  --reasoning-budget 1024

RTX 4000 Ada 20GB

llama-server \
  -m ornith-aeon-35b-MTP-IQ4_XS.gguf \
  -a ornith-aeon-35b \
  --spec-type draft-mtp \
  --host 0.0.0.0 --port 8083 \
  -ngl 99 \
  --n-cpu-moe 5 \
  -fa on \
  -ctk q4_0 -ctv q4_0 \
  -c 8192 \
  -b 1024 -ub 512 \
  --parallel 1 \
  --temp 0.6 --top-k 20 --top-p 0.95 \
  --repeat-penalty 1.1 \
  --jinja \
  --reasoning-format deepseek \
  --reasoning-budget 1024

High quality on 32GB+ GPU

llama-server \
  -m ornith-aeon-35b-MTP-Q6_K.gguf \
  -a ornith-aeon-35b \
  --spec-type draft-mtp \
  --host 0.0.0.0 --port 8083 \
  -ngl 99 \
  -fa on \
  -ctk q8_0 -ctv q8_0 \
  -c 32768 \
  -b 2048 -ub 768 \
  --parallel 1 \
  --temp 0.6 --top-k 20 --top-p 0.95 \
  --repeat-penalty 1.1 \
  --jinja \
  --reasoning-format deepseek \
  --reasoning-budget 1024

Speculative Decoding Options

All available in llama.cpp latest master. Work on any CUDA GPU (Ada Lovelace, Ampere, Blackwell):

| Strategy | Flag | Draft Model? | Notes |

|----------|------|-------------|-------|

| MTP (embedded) | --spec-type draft-mtp | Bundled in MTP GGUFs | Requires --parallel 1. Measured +80% on Spark. |

| Eagle3 | --spec-type draft-eagle3 | Separate GGUF | PR #18039. No pre-trained drafter for this model yet. |

| DFlash | --spec-type draft-dflash | Separate GGUF | z-lab/Qwen3.5-35B-A3B-DFlash. Best results on vLLM/SGLang at BF16/FP8. |

| N-gram | --spec-default | None | Zero VRAM cost. Marginal benefit on structured output. |

Key Parameters

| Parameter | Value | Why |

|-----------|-------|-----|

| -ngl 99 | Offload all layers to GPU | Maximizes speed |

| --spec-type draft-mtp | MTP speculative decoding | Only with MTP-grafted GGUFs |

| --parallel 1 | Single slot | Required for MTP |

| --n-cpu-moe N | Offload N MoE layers to CPU | Saves ~0.8GB per layer |

| -fa on | Flash attention | Reduces KV cache memory |

| -ctk q4_0 -ctv q4_0 | Quantize KV cache | Saves ~60% KV cache VRAM |

| --jinja | Jinja2 chat template | Required for this model |

| --reasoning-format deepseek | Thinking mode | Model uses think tags |

If It OOMs

1. Increase --n-cpu-moe (trades speed for VRAM)
2. Lower context: -c 8192 or -c 4096
3. Lower batch: -b 512 -ub 256
4. Use a trunk (non-MTP) file to save ~0.5-1GB
5. Use a smaller quant

Model Details

• Architecture: Qwen3.5-MoE (Mixture of Experts)
• Total Parameters: ~35B
• Active Parameters per Token: ~9B (4 of 64 experts active)
• MTP Tensors: 785 (grafted from base model in MTP variants)
• Chat Template: ChatML
• Thinking/Reasoning: DeepSeek-style think tags
• Context: Up to 131K tokens (limited by available VRAM)

Quantization Details

• Source: BF16 safetensors from AEON-7/Ornith-1.0-35B-AEON-Ultimate-Uncensored-BF16
• MTP Graft Source: 785 mtp.* tensors from Qwen/Qwen3.5-35B-A3B (~488 MB BF16)
• Importance Matrix: Calibrated on diverse coding, debugging, system design, and reasoning prompts
• llama.cpp: Built from latest master (post-b9606, with MTP/Eagle3/DFlash support merged)
• Platform: DGX Spark (aarch64, CUDA 13.0)

Credits

• Base model by AEON-7
• MTP weights from Qwen/Qwen3.5-35B-A3B
• Quantization by vcruz305