liodon-ai/Qwen3.6-35B-A3B-imatrix-GGUF overview
Qwen3.6 35B A3B — iMatrix GGUF Higher quality GGUF quantizations of Qwen/Qwen3.6 35B A3B https://huggingface.co/Qwen/Qwen3.6 35B A3B using importance matrix iM…
Runs locally from ~12.05 GB disk (16 GB VRAM class GPUs with llama.cpp / guIDE).
Repository Files & Downloads
| File | Type | Quantization | Size | Link |
|---|---|---|---|---|
| Qwen3.6-35B-A3B-Q2_K.gguf | GGUF | Q2_K | 12.05 GB | Download |
| Qwen3.6-35B-A3B-Q3_K_M.gguf | GGUF | Q3_K_M | 15.61 GB | Download |
| Qwen3.6-35B-A3B-Q4_K_M.gguf | GGUF | Q4_K_M | 19.71 GB | Download |
| Qwen3.6-35B-A3B-Q5_K_M.gguf | GGUF | Q5_K_M | 23.03 GB | Download |
| Qwen3.6-35B-A3B-Q6_K.gguf | GGUF | Q6_K | 26.56 GB | Download |
| Qwen3.6-35B-A3B-Q8_0.gguf | GGUF | Q8_0 | 34.37 GB | Download |
Model Details
| Model ID | liodon-ai/Qwen3.6-35B-A3B-imatrix-GGUF |
|---|---|
| Author | liodon-ai |
| Pipeline | text-generation |
| License | apache-2.0 |
| Base model | Qwen/Qwen3.6-35B-A3B |
| Last modified | 2026-07-13T04:54:33.000Z |
Model README
---
license: apache-2.0
base_model: Qwen/Qwen3.6-35B-A3B
base_model_relation: quantized
pipeline_tag: text-generation
library_name: gguf
tags:
- gguf
- imatrix
- qwen3
- qwen3.6
- local-llm
- llama.cpp
- ollama
- lm-studio
- quantized
quantized_by: liodon-ai
---
Qwen3.6-35B-A3B — iMatrix GGUF
Higher-quality GGUF quantizations of Qwen/Qwen3.6-35B-A3B using importance matrix (iMatrix) calibration.
What is iMatrix?
Standard quantization rounds all weights equally. iMatrix runs a calibration pass over real text to identify which weights matter most, then prioritizes precision where it counts. The result: noticeably better coherence and instruction-following at Q2/Q3/Q4 — same file size, better output.
The i-quants (IQ2_M, IQ3_M, IQ4_XS) are exclusively iMatrix-based and provide the best quality-per-GB available.
Quick Start
Ollama
ollama run hf.co/liodon-ai/Qwen3.6-35B-A3B-imatrix-GGUF:Q4_K_Mllama.cpp
llama-cli -hf liodon-ai/Qwen3.6-35B-A3B-imatrix-GGUF:Q4_K_MLM Studio / Jan
Search liodon-ai/Qwen3.6-35B-A3B-imatrix-GGUF and pick your quant.
Available Quants
| Quant | Size | Notes |
|-------|------|-------|
| Q2_K | 12.94 GB | tiniest standard — runs almost anywhere |
| Q3_K_M | 16.76 GB | great for 8GB VRAM |
| Q4_K_M | 21.17 GB | sweet spot (recommended) |
| Q5_K_M | 24.73 GB | high quality |
| Q6_K | 28.51 GB | near-lossless |
| Q8_0 | 36.90 GB | basically full quality |
VRAM Requirements
| VRAM | Recommended Quant |
|------|-------------------|
| 6 GB | IQ2_M |
| 8 GB | IQ3_M or Q3_K_M |
| 10 GB | IQ4_XS or Q4_K_M |
| 12 GB | Q4_K_M |
| 16 GB | Q5_K_M |
| 24 GB | Q6_K or Q8_0 |
iMatrix vs Standard — Why It Matters
At low bit widths (Q2/Q3/Q4), standard quantization loses coherence and starts producing
repetitive or broken output. iMatrix keeps the model sharp by protecting the most important
weights. If you're running at Q4 or below, prefer the iMatrix quants from this repo over
standard Q-series from other repos.
Base Model
- Model: Qwen/Qwen3.6-35B-A3B
- Params: 35B (3.5B active, MoE)
- Context: 128K tokens
- License: Apache 2.0
- Authors: Qwen Team (Alibaba Cloud)
Run liodon-ai/Qwen3.6-35B-A3B-imatrix-GGUF with guIDE
Download guIDE — the AI-native code editor with local LLM inference and 69 built-in tools.
Source: Hugging Face · Compare models