Model Intelligence Sheet
Arki05/Qwen3.5-4B-GGUF-shards overview
Qwen3.5 4B, every tensor at every type Qwen/Qwen3.5 4B https://huggingface.co/Qwen/Qwen3.5 4B quantized to 25 llama.cpp types, each tensor at each type, for as…
Runs locally from ~3.5 MB disk (4 GB VRAM class GPUs with llama.cpp / guIDE).
Repository Files & Downloads
28 GGUF files detected
Direct downloads for local inference
| File | Type | Quantization | Size | Link |
|---|---|---|---|---|
| imatrix/imatrix_unsloth.gguf | GGUF | GGUF | 3.5 MB | Download |
| skeleton.gguf | GGUF | GGUF | 10.5 MB | Download |
| source/Qwen3.5-4B-BF16.gguf | GGUF | BF16 | 7.85 GB | Download |
| types/IQ1_M.gguf | GGUF | GGUF | 891.3 MB | Download |
| types/IQ1_S.gguf | GGUF | GGUF | 1.85 GB | Download |
| types/IQ2_S.gguf | GGUF | GGUF | 1.27 GB | Download |
| types/IQ2_XS.gguf | GGUF | GGUF | 2.16 GB | Download |
| types/IQ2_XXS.gguf | GGUF | GGUF | 2.05 GB | Download |
| types/IQ3_S.gguf | GGUF | GGUF | 1.70 GB | Download |
| types/IQ3_XXS.gguf | GGUF | GGUF | 1.51 GB | Download |
| types/IQ4_NL.gguf | GGUF | GGUF | 2.22 GB | Download |
| types/IQ4_XS.gguf | GGUF | GGUF | 2.09 GB | Download |
| types/MXFP4.gguf | GGUF | GGUF | 2.09 GB | Download |
| types/NVFP4.gguf | GGUF | GGUF | 2.22 GB | Download |
| types/Q1_0.gguf | GGUF | GGUF | 578.0 MB | Download |
| types/Q2_0.gguf | GGUF | GGUF | 1.12 GB | Download |
| types/Q2_K.gguf | GGUF | GGUF | 1.30 GB | Download |
| types/Q3_K.gguf | GGUF | GGUF | 1.70 GB | Download |
| types/Q4_0.gguf | GGUF | GGUF | 2.22 GB | Download |
| types/Q4_1.gguf | GGUF | GGUF | 2.46 GB | Download |
| types/Q4_K.gguf | GGUF | GGUF | 2.22 GB | Download |
| types/Q5_0.gguf | GGUF | GGUF | 2.71 GB | Download |
| types/Q5_1.gguf | GGUF | GGUF | 2.95 GB | Download |
| types/Q5_K.gguf | GGUF | GGUF | 2.71 GB | Download |
| types/Q6_K.gguf | GGUF | GGUF | 3.23 GB | Download |
| types/Q8_0.gguf | GGUF | GGUF | 4.17 GB | Download |
| types/TQ1_0.gguf | GGUF | GGUF | 860.0 MB | Download |
| types/TQ2_0.gguf | GGUF | GGUF | 1.02 GB | Download |
Model Details
Model README
---
license: apache-2.0
base_model: Qwen/Qwen3.5-4B
tags:
- gguf
---
Qwen3.5-4B, every tensor at every type
Qwen/Qwen3.5-4B quantized to 25 llama.cpp types, each tensor at each type, for assembling GGUFs with a type per tensor.
- Source: unsloth/Qwen3.5-4B-GGUF's BF16 GGUF and imatrix, unchanged (
source/,imatrix/). types/: quantized with llama.cpp'sllama-quantize(commitd81235049); modified versions of the original weights.
Apache License 2.0 (LICENSE), as the original model.
Run Arki05/Qwen3.5-4B-GGUF-shards with guIDE
Download guIDE — the AI-native code editor with local LLM inference and 69 built-in tools.
Source: Hugging Face · Compare models