---

base_model: TheDrummer/Orion-26B-A4B-v1.1

tags:

• gemma4
• gguf
• moe
• uncensored
• creative

inference: false

---

🌌 Orion-26B-A4B-v1.1 - GGUF

This repository contains custom GGUF format quantized weights for TheDrummer/Orion-26B-A4B-v1.1.

🧠 About the Base Model

Orion-26B-A4B-v1.1 is an incredible creative and uncensored fine-tune built on top of the Gemma 4 26B A4B MoE architecture.

• Mixture of Experts (MoE): While the model contains 25.2 billion total parameters, it only activates 3.8 billion parameters during inference. This gives you the deep reasoning of a massive 30B class model at the speed and memory footprint of a tiny 4B model!
• Dual Thinking Mode: Natively supports both standard generation and "thinking" mode for complex reasoning tasks.
• Context Window: Up to 256K tokens via hybrid global/sliding-window attention.
• Focus: Geared specifically toward deep creativity, nuanced roleplay, and uncensored, highly steerable outputs.

---

📦 Available Quants & Hardware Requirements

Choose the quant that best fits your system's RAM/VRAM. For most users with standard hardware, Q4_K_M or Q5_K_M provides the best balance of speed and intelligence.

| Filename | Quant Type | File Size | Recommended Use |

|---|---|---|---|

| Orion-26B-A4B-v1.1-Q3_K_M.gguf | Q3_K_M | 13.3 GB | Max speed, fits in 16GB RAM. Noticeable perplexity loss. |

| Orion-26B-A4B-v1.1-Q4_K_S.gguf | Q4_K_S | 15.5 GB | Good for constrained VRAM. |

| Orion-26B-A4B-v1.1-Q4_K_M.gguf | Q4_K_M | 16.8 GB | ⭐ Recommended: Perfect balance of quality and size. |

| Orion-26B-A4B-v1.1-Q5_K_S.gguf | Q5_K_S | 18.0 GB | High quality, slightly faster than K_M. |

| Orion-26B-A4B-v1.1-Q5_K_M.gguf | Q5_K_M | 19.1 GB | Excellent quality, minimal degradation from base. |

| Orion-26B-A4B-v1.1-Q6_K.gguf | Q6_K | 22.6 GB | Near-perfect retention of the FP16 base. |

| Orion-26B-A4B-v1.1-Q8_0.gguf | Q8_0 | 26.9 GB | Maximum quality, virtually identical to FP16. |

---

🚀 How to Run

1. LM Studio / Faraday / GPT4All

1. Search for Abiray/Orion-26B-A4B-v1.1-GGUF in the app's download manager.
2. Download your preferred quant (e.g., Q4_K_M).
3. Load it and ensure the prompt template is set to Gemma 4 Chat.

2. llama.cpp (Command Line)

If you compiled llama.cpp from source, run the model using the built-in CLI. Adjust -t (threads) and -c (context size) to fit your hardware:

./llama-cli -m Orion-26B-A4B-v1.1-Q4_K_M.gguf -p "<start_of_turn>user\nWrite a creative story about a sentient AI.<end_of_turn>\n<start_of_turn>model\n" -n 512 -c 8192