GraySoft
Download guIDE →
Projects Models Compare Cloud benchmarks FAQ Download guIDE →
Model Intelligence Sheet

wepiqx/Ornith-1.0-9B-ASHQ1-MTP-GGUF overview

Ornith 1.0 9B — ASHQ1 Quantization MTP Aware ASHQ1 quantization of Ornith 1.0 9B https://huggingface.co/deepreinforce ai/Ornith 1.0 9B with explicit MTP Multi …

ggufqwenqwen3.59bornithcodingagentdeepreinforcequantizedquantizationllama-cppimatrixhybrid-quantizationASHQ1priority-queuemsemtpmulti-token-predictiontext-generationenbase_model:deepreinforce-ai/Ornith-1.0-9Bbase_model:quantized:deepreinforce-ai/Ornith-1.0-9Blicense:mitendpoints_compatible

Runs locally from ~4.75 GB disk (8 GB VRAM class GPUs with llama.cpp / guIDE).

Downloads
0
Likes
1
Pipeline
text-generation
Author
wepiqx

Repository Files & Downloads

4 GGUF files detected
Direct downloads for local inference
FileTypeQuantizationSizeLink
Ornith-1.0-9B-MTP-BF16-ASHQ1-4850.ggufGGUFBF164.75 GBDownload
Ornith-1.0-9B-MTP-BF16-ASHQ1-5500.ggufGGUFBF165.39 GBDownload
Ornith-1.0-9B-MTP-BF16-ASHQ1-6500.ggufGGUFBF166.36 GBDownload
Ornith-1.0-9B-MTP-BF16-ASHQ1-7300.ggufGGUFBF167.14 GBDownload

Model Details

Model IDwepiqx/Ornith-1.0-9B-ASHQ1-MTP-GGUF
Authorwepiqx
Pipelinetext-generation
Licensemit
Base modeldeepreinforce-ai/Ornith-1.0-9B
Last modified2026-07-01T10:34:19.000Z

Model README

---

license: mit

language:

  • en

base_model: deepreinforce-ai/Ornith-1.0-9B

pipeline_tag: text-generation

library_name: gguf

tags:

  • qwen
  • qwen3.5
  • 9b
  • ornith
  • coding
  • agent
  • deepreinforce
  • quantized
  • quantization
  • gguf
  • llama-cpp
  • imatrix
  • hybrid-quantization
  • ASHQ1
  • priority-queue
  • mse
  • mtp
  • multi-token-prediction

---

Ornith-1.0-9B — ASHQ1 Quantization (MTP-Aware)

ASHQ1 quantization of Ornith-1.0-9B with explicit MTP (Multi-Token Prediction) handling — MTP heads are kept at Q8_0 and excluded from the classifier budget, preserving their quality while the priority queue optimises the main transformer layers.

Huge thanks to protoLabsAI/Ornith-1.0-9B-MTP-GGUF for providing the BF16 source with MTP heads — we adapted it with ASHQ1 to deliver both speed and quality.

> Note: File names contain "BF16" for HuggingFace parser compatibility — these are ASHQ1 quants, not BF16.

Quants

| File | Size | Method | Description |

|:-----|:----:|:------:|:------------|

| Ornith-1.0-9B-MTP-BF16-ASHQ1-4850.gguf | 4.7 GB | ASHQ1 (v6, --allow-q3-or-lower) | Tiny — uses IQ tiers to fit 4.7 GB, MTP heads at Q8_0 |

| Ornith-1.0-9B-MTP-BF16-ASHQ1-5500.gguf | 5.4 GB | ASHQ1 (v6) | Compact — beats i1-Q6_K (7.0 GB), 23% smaller |

| Ornith-1.0-9B-MTP-BF16-ASHQ1-6500.gguf | 6.4 GB | ASHQ1 (v6) | Balanced — near i1-Q8_0 quality at 33% smaller |

| Ornith-1.0-9B-MTP-BF16-ASHQ1-7300.gguf | 7.2 GB | ASHQ1 (v6) | Best — surpasses i1-Q8_0 quality, 25% smaller |

All comparisons are against i1 quants (uniform quantization with bartowski's Ornith imatrix). ASHQ1 consistently beats them at smaller sizes: the priority queue allocates bits where they matter most. See wepiqx/ASHQ1 for full benchmarks across architectures. MTP heads are deployed at Q8_0 and excluded from the classifier budget.

> Note: The 4850 variant uses IQ tiers via --allow-q3-or-lower to fit the tight budget — quality is roughly on par with Q4_K_S, while 5500+ delivers a significant step up. It's best suited for lighter tasks, while coding and agentic workloads may prefer 5500+. Feel free to try it either way.

How ASHQ1 Works

See wepiqx/ASHQ1 for the full method, source code, and benchmarks.

Usage

llama.cpp

# Chat mode with built-in Jinja template (recommended)
llama-cli \
  -m Ornith-1.0-9B-MTP-BF16-ASHQ1-6500.gguf \
  --jinja \
  -ngl 99 \
  -c 8192

Recommended sampling: temperature 0.6–1.0, top_p 0.95, top_k 20.

Ollama

The model's built-in Jinja template is auto-detected by Ollama — no need to specify TEMPLATE manually.

  1. Create a Modelfile:
FROM ./Ornith-1.0-9B-MTP-BF16-ASHQ1-6500.gguf

PARAMETER num_ctx 8192
PARAMETER temperature 0.6
PARAMETER top_k 20
PARAMETER top_p 0.95
  1. Build and run:
ollama create ornith-ashq1-mtp-6500 -f Modelfile
ollama run ornith-ashq1-mtp-6500

LM Studio

  1. Open LM Studio
  2. Drag the desired GGUF file (4850, 5500, 6500, or 7300) into the app
  3. Set GPU Offload to 99 layers
  4. Set context length to 8192+
  5. Set preset sampling: temperature 0.6, top_p 0.95, top_k 20
  6. Start chatting

Quantization Configs

Generated by ASHQ1 priority queue (v6) — files in this repo were produced with these exact llama-quantize arguments.

4850 Config (--allow-q3-or-lower)

--output-tensor-type Q5_K
--token-embedding-type Q5_K
--tensor-type "(blk|BLK)\.(32)\.nextn=Q8_0"
--tensor-type "(blk|BLK)\.(32)\.ffn_down=Q4_K"
--tensor-type "(blk|BLK)\.(31)\.ffn_down=IQ4_XS"
--tensor-type "(blk|BLK)\.(30)\.ffn_down=IQ3_S"
--tensor-type "(blk|BLK)\.(32)\.attn_v=Q4_K"
--tensor-type "(blk|BLK)\.(32)\.attn_q=Q4_K"
--tensor-type "(blk|BLK)\.(32)\.attn_k=Q4_K"
--tensor-type "(blk|BLK)\.((?:26|27|28|29))\.ffn_down=IQ3_XXS"
--tensor-type "(blk|BLK)\.((?:22|23|24|25))\.ffn_down=IQ2_S"
--tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21))\.ffn_down=IQ2_XXS"
--tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27|(?:31|32))\.attn_k_norm=F16"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_norm=F16"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_a=F16"
--tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31|32))\.attn_norm=F16"
--tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27|(?:31|32))\.attn_q_norm=F16"
--tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31|32))\.post_attention_norm=F16"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_conv1d=F16"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_dt=F16"
--tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27|31)\.attn_q=Q5_K"
--tensor-type "(blk|BLK)\.(0|6|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.attn_qkv=Q5_K"
--tensor-type "(blk|BLK)\.(0|6|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.attn_gate=Q5_K"
--tensor-type "(blk|BLK)\.(0|6|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_alpha=Q5_K"
--tensor-type "(blk|BLK)\.(0|6|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_beta=Q5_K"
--tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27|31)\.attn_k=Q5_K"
--tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27|31)\.attn_v=Q5_K"
--tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27|(?:31|32))\.attn_output=Q4_K"
--tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31|32))\.ffn_gate=Q4_K"
--tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31|32))\.ffn_up=Q4_K"
--tensor-type "(blk|BLK)\.([1-2]|[4-5])\.attn_gate=Q4_K"
--tensor-type "(blk|BLK)\.([1-2]|[4-5])\.ssm_beta=Q4_K"
--tensor-type "(blk|BLK)\.([1-2]|[4-5])\.ssm_alpha=Q4_K"
--tensor-type "(blk|BLK)\.([1-2]|[4-5])\.attn_qkv=Q4_K"
--tensor-type "(blk|BLK)\.(28|30)\.ssm_out=IQ4_XS"
--tensor-type "(blk|BLK)\.((?:24|25)|29)\.ssm_out=IQ3_S"
--tensor-type "(blk|BLK)\.((?:21|22)|26)\.ssm_out=IQ3_XXS"
--tensor-type "(blk|BLK)\.(18|20)\.ssm_out=IQ2_S"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17))\.ssm_out=IQ2_XXS"
--tensor-type ".*output_norm.*=F16"

5500 Config (base=Q5_K_M)

--output-tensor-type Q5_K
--token-embedding-type Q5_K
--tensor-type "(blk|BLK)\.(32)\.nextn=Q8_0"
--tensor-type "(blk|BLK)\.(3)\.attn_q=Q6_K"
--tensor-type "(blk|BLK)\.(3)\.attn_k=Q6_K"
--tensor-type "(blk|BLK)\.(31)\.ffn_down=Q5_K"
--tensor-type "(blk|BLK)\.((?:20|21)|(?:24|25)|28|30)\.attn_qkv=Q6_K"
--tensor-type "(blk|BLK)\.(3|32)\.attn_v=Q6_K"
--tensor-type "(blk|BLK)\.((?:20|21)|(?:24|25)|28|30)\.ssm_beta=Q6_K"
--tensor-type "(blk|BLK)\.((?:20|21)|(?:24|25)|28|30)\.ssm_alpha=Q6_K"
--tensor-type "(blk|BLK)\.((?:20|21)|(?:24|25)|28|30)\.attn_gate=Q6_K"
--tensor-type "(blk|BLK)\.(31)\.attn_output=Q5_K"
--tensor-type "(blk|BLK)\.(30)\.ssm_out=Q5_K"
--tensor-type "(blk|BLK)\.(1)\.attn_gate=Q4_K"
--tensor-type "(blk|BLK)\.(1)\.ssm_alpha=Q4_K"
--tensor-type "(blk|BLK)\.(32)\.attn_q=Q4_K"
--tensor-type "(blk|BLK)\.(1)\.ssm_beta=Q4_K"
--tensor-type "(blk|BLK)\.(1)\.attn_qkv=Q4_K"
--tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30)|32)\.ffn_down=Q4_K"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_conv1d=F16"
--tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31|32))\.post_attention_norm=F16"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_norm=F16"
--tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31|32))\.attn_norm=F16"
--tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27|(?:31|32))\.attn_k_norm=F16"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_dt=F16"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_a=F16"
--tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27|(?:31|32))\.attn_q_norm=F16"
--tensor-type "(blk|BLK)\.((?:26|27|28|29|30|31))\.ffn_up=Q5_K"
--tensor-type "(blk|BLK)\.(0|2|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|22|26|29)\.attn_qkv=Q5_K"
--tensor-type "(blk|BLK)\.(0|2|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|22|26|29)\.ssm_alpha=Q5_K"
--tensor-type "(blk|BLK)\.(7|11|15|19|23|27|(?:31|32))\.attn_k=Q5_K"
--tensor-type "(blk|BLK)\.(7|11|15|19|23|27|31)\.attn_v=Q5_K"
--tensor-type "(blk|BLK)\.(0|2|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|22|26|29)\.attn_gate=Q5_K"
--tensor-type "(blk|BLK)\.(7|11|15|19|23|27|31)\.attn_q=Q5_K"
--tensor-type "(blk|BLK)\.(0|2|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|22|26|29)\.ssm_beta=Q5_K"
--tensor-type "(blk|BLK)\.((?:26|27|28|29|30|31))\.ffn_gate=Q5_K"
--tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25)|32)\.ffn_up=Q4_K"
--tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25)|32)\.ffn_gate=Q4_K"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29))\.ssm_out=Q4_K"
--tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27|32)\.attn_output=Q4_K"
--tensor-type ".*output_norm.*=F16"

6500 Config (base=Q5_K_M)

--output-tensor-type Q5_K
--token-embedding-type Q5_K
--tensor-type "(blk|BLK)\.(32)\.nextn=Q8_0"
--tensor-type "(blk|BLK)\.(31)\.ffn_down=Q6_K"
--tensor-type "(blk|BLK)\.(0|(?:9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_beta=Q8_0"
--tensor-type "(blk|BLK)\.(0|(?:9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.attn_gate=Q8_0"
--tensor-type "(blk|BLK)\.(3|7|15|19|23|27|31)\.attn_v=Q8_0"
--tensor-type "(blk|BLK)\.(0|(?:9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.attn_qkv=Q8_0"
--tensor-type "(blk|BLK)\.(0|(?:9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_alpha=Q8_0"
--tensor-type "(blk|BLK)\.(3|7|15|19|23|27|31)\.attn_k=Q8_0"
--tensor-type "(blk|BLK)\.(3|7|15|19|23|27|31)\.attn_q=Q8_0"
--tensor-type "(blk|BLK)\.(11)\.attn_q=Q6_K"
--tensor-type "(blk|BLK)\.(11)\.attn_k=Q6_K"
--tensor-type "(blk|BLK)\.(11)\.attn_v=Q6_K"
--tensor-type "(blk|BLK)\.([1-2]|[4-6]|8)\.ssm_beta=Q6_K"
--tensor-type "(blk|BLK)\.([1-2]|[4-6]|8)\.attn_qkv=Q6_K"
--tensor-type "(blk|BLK)\.((?:14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31))\.ffn_gate=Q6_K"
--tensor-type "(blk|BLK)\.([1-2]|[4-6]|8)\.ssm_alpha=Q6_K"
--tensor-type "(blk|BLK)\.((?:28|29|30))\.ssm_out=Q6_K"
--tensor-type "(blk|BLK)\.((?:14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31))\.ffn_up=Q6_K"
--tensor-type "(blk|BLK)\.([1-2]|[4-6]|8)\.attn_gate=Q6_K"
--tensor-type "(blk|BLK)\.(27|31)\.attn_output=Q6_K"
--tensor-type "(blk|BLK)\.(32)\.attn_v=Q5_K"
--tensor-type "(blk|BLK)\.(32)\.attn_k=Q4_K"
--tensor-type "(blk|BLK)\.(32)\.attn_q=Q4_K"
--tensor-type "(blk|BLK)\.((?:26|27|28|29|30))\.ffn_down=Q5_K"
--tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25)|32)\.ffn_down=Q4_K"
--tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27|(?:31|32))\.attn_q_norm=F16"
--tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31|32))\.post_attention_norm=F16"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_norm=F16"
--tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31|32))\.attn_norm=F16"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_a=F16"
--tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27|(?:31|32))\.attn_k_norm=F16"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_conv1d=F16"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_dt=F16"
--tensor-type "(blk|BLK)\.(18|(?:20|21|22)|(?:24|25|26))\.ssm_out=Q5_K"
--tensor-type "(blk|BLK)\.((?:4|5|6|7|8|9|10|11|12|13))\.ffn_gate=Q5_K"
--tensor-type "(blk|BLK)\.((?:4|5|6|7|8|9|10|11|12|13))\.ffn_up=Q5_K"
--tensor-type "(blk|BLK)\.(19|23)\.attn_output=Q5_K"
--tensor-type "(blk|BLK)\.([0-3]|32)\.ffn_up=Q4_K"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17))\.ssm_out=Q4_K"
--tensor-type "(blk|BLK)\.([0-3]|32)\.ffn_gate=Q4_K"
--tensor-type "(blk|BLK)\.(3|7|11|15|32)\.attn_output=Q4_K"
--tensor-type ".*output_norm.*=F16"

7300 Config (base=Q5_K_M)

--output-tensor-type Q5_K
--token-embedding-type Q5_K
--tensor-type "(blk|BLK)\.(32)\.nextn=Q8_0"
--tensor-type "(blk|BLK)\.(31)\.ffn_down=Q8_0"
--tensor-type "(blk|BLK)\.(27|31)\.attn_output=Q8_0"
--tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27|31)\.attn_v=Q8_0"
--tensor-type "(blk|BLK)\.(28|30)\.ssm_out=Q8_0"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.attn_qkv=Q8_0"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.attn_gate=Q8_0"
--tensor-type "(blk|BLK)\.((?:16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31))\.ffn_up=Q8_0"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_beta=Q8_0"
--tensor-type "(blk|BLK)\.((?:16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31))\.ffn_gate=Q8_0"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_alpha=Q8_0"
--tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27|31)\.attn_q=Q8_0"
--tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27|31)\.attn_k=Q8_0"
--tensor-type "(blk|BLK)\.(32)\.attn_k=Q6_K"
--tensor-type "(blk|BLK)\.((?:22|23|24|25|26|27|28|29|30))\.ffn_down=Q6_K"
--tensor-type "(blk|BLK)\.((?:4|5|6|7|8|9|10|11|12|13|14|15))\.ffn_up=Q6_K"
--tensor-type "(blk|BLK)\.((?:4|5|6|7|8|9|10|11|12|13|14|15))\.ffn_gate=Q6_K"
--tensor-type "(blk|BLK)\.(18|(?:20|21|22)|(?:24|25|26)|29)\.ssm_out=Q6_K"
--tensor-type "(blk|BLK)\.(19|23)\.attn_output=Q6_K"
--tensor-type "(blk|BLK)\.(32)\.attn_v=Q5_K"
--tensor-type "(blk|BLK)\.(32)\.ffn_gate=Q4_K"
--tensor-type "(blk|BLK)\.(32)\.ffn_up=Q4_K"
--tensor-type "(blk|BLK)\.(32)\.attn_q=Q4_K"
--tensor-type "(blk|BLK)\.((?:18|19|20|21))\.ffn_down=Q5_K"
--tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17)|32)\.ffn_down=Q4_K"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_dt=F16"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_norm=F16"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_a=F16"
--tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31|32))\.attn_norm=F16"
--tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31|32))\.post_attention_norm=F16"
--tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27|(?:31|32))\.attn_k_norm=F16"
--tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27|(?:31|32))\.attn_q_norm=F16"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_conv1d=F16"
--tensor-type "(blk|BLK)\.([0-3])\.ffn_gate=Q5_K"
--tensor-type "(blk|BLK)\.([0-3])\.ffn_up=Q5_K"
--tensor-type "(blk|BLK)\.((?:9|10)|(?:12|13|14)|(?:16|17))\.ssm_out=Q5_K"
--tensor-type "(blk|BLK)\.(7|11|15)\.attn_output=Q5_K"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|8)\.ssm_out=Q4_K"
--tensor-type "(blk|BLK)\.(3|32)\.attn_output=Q4_K"
--tensor-type ".*output_norm.*=F16"

Hardware Requirements

| Variant | Size (GGUF) | VRAM (ctx 8192) | Minimum GPU |

|:--------|:-----------:|:----------------:|:-----------:|

| 4850 | 4.7 GB | ~5.8 GB | 8 GB |

| 5500 | 5.4 GB | ~6.5 GB | 8 GB |

| 6500 | 6.4 GB | ~7.5 GB | 8 GB |

| 7300 | 7.2 GB | ~8.3 GB | 12 GB |

Notes

Run wepiqx/Ornith-1.0-9B-ASHQ1-MTP-GGUF with guIDE

Download guIDE — the AI-native code editor with local LLM inference and 69 built-in tools.

Download guIDE → · Browse 524k+ models · Compare models

Source: Hugging Face · Compare models