Qwen2.5-32B-Instruct-AdiTurbo-GGUF

Low-bit GGUF quantizations of Qwen/Qwen2.5-32B-Instruct,

produced with the AdiTurbo Engine (private llama.cpp fork, build 8661).

Quantized with a fresh 50-chunk wikitext-2 importance matrix. Intended for

offline/on-device inference (Eyla AIOS). See the AdiTurbo benchmark for

perplexity/throughput numbers.

Files

This is a derivative quantization; the original model's license (apache-2.0) applies.

Question 2

What license applies to Adiuk/Qwen2.5-32B-Instruct-AdiTurbo-GGUF?

Accepted Answer

License: apache-2.0. Verify terms on Hugging Face before commercial use.

Question 3

How do I run Adiuk/Qwen2.5-32B-Instruct-AdiTurbo-GGUF locally?

Accepted Answer

Download a GGUF file from this page and load it in guIDE or llama.cpp. Pipeline task: text-generation.

Question 4

How much VRAM or disk space does Adiuk/Qwen2.5-32B-Instruct-AdiTurbo-GGUF need?

Accepted Answer

Runs locally from ~8.41 GB disk (12 GB VRAM class GPUs with llama.cpp / guIDE).

File	Type	Quantization	Size	Link
Qwen2.5-32B-Instruct-IQ2_XXS.gguf	GGUF	IQ2_XXS	8.41 GB	Download
Qwen2.5-32B-Instruct-Q3_K_M.gguf	GGUF	Q3_K_M	14.84 GB	Download
Qwen2.5-32B-Instruct-TQ3_0.gguf	GGUF	GGUF	13.23 GB	Download
Qwen2.5-32B-Instruct-TQ4_0.gguf	GGUF	GGUF	16.96 GB	Download

Model ID	Adiuk/Qwen2.5-32B-Instruct-AdiTurbo-GGUF
Author	Adiuk
Pipeline	—
License	apache-2.0
Base model	Qwen/Qwen2.5-32B-Instruct
Last modified	2026-06-22T05:22:16.000Z