---

base_model: XYZAILab/XYZ-Aquila-mini

library_name: gguf

license: apache-2.0

pipeline_tag: text-generation

tags:

• gguf
• quantized
• llama.cpp
• qwen3.6
• agentic-search
• thinking-model

---

<div align="center">

<a href="https://xyz-lab.ai/">

<img src="https://huggingface.co/XYZAILab/XYZ-Aquila-mini/resolve/main/assets/xyz-ai-lab-slogan.svg" width="520" alt="XYZ AI Lab — We Build The Minds That Build" />

</a>

</div>

XYZ-Aquila-mini - 4-bit GGUF Quants

This repository contains highly optimized 4-bit GGUF quantizations (K-quants and imatrix IQ-quants) for XYZAILab/XYZ-Aquila-mini.

• Original Model: XYZAILab/XYZ-Aquila-mini
• Base Architecture: Qwen3.6-35B-A3B (MoE, ~3B active parameters per token)
• Model Type: Open-weight thinking model optimized for Deep Search and Agentic Workflows.
• Quantization Formats: GGUF (IQ4_XS, IQ4_NL, Q4_K_M, Q4_K_S)

---

Quantization Details

This repository specifically focuses on 4-bit quantizations to maximize the balance between VRAM footprint and reasoning capability.

| File Name | Size | Quant Method | Description |

|---|---|---|---|

| XYZ-Aquila-mini-IQ4_XS.gguf | ~18.8 GB | IQ4_XS | 4-bit importance matrix (extra small). Optimized for low VRAM while retaining reasoning quality. |

| XYZ-Aquila-mini-IQ4_NL.gguf | ~19.9 GB | IQ4_NL | 4-bit importance matrix (non-linear). Improved non-linear weight distribution for accuracy. |

| XYZ-Aquila-mini-Q4_K_M.gguf | ~21.4 GB | Q4_K_M | Standard 4-bit K-quantization (medium). Recommended general-purpose balance of speed and quality. |

| XYZ-Aquila-mini-Q4_K_S.gguf | ~20.6 GB | Q4_K_S | Standard 4-bit K-quantization (small). Slightly smaller file size. |

---

Usage Guide

XYZ-Aquila-mini is a thinking model with Qwen-compatible reasoning and tool-call formats. It performs best when allowed to output chain-of-thought reasoning before acting.

1. Running with llama.cpp

You can run this model natively using llama-server to expose an OpenAI-compatible API that supports tool calling.

# Clone and build llama.cpp
git clone [https://github.com/ggml-org/llama.cpp](https://github.com/ggml-org/llama.cpp)
cd llama.cpp && cmake -B build && cmake --build build -j

# Download the recommended model from this repository
huggingface-cli download Abiray/XYZ-Aquila-mini-IQ4-GGUF XYZ-Aquila-mini-Q4_K_M.gguf --local-dir .

# Serve the model (adjust context length `-c` based on your available RAM/VRAM)
./build/bin/llama-server \
  -m XYZ-Aquila-mini-Q4_K_M.gguf \
  --port 8000 \
  -c 32768 \
  -ngl 99