Model Intelligence Sheet
armandosds/granite-4.1-3b-GGUF overview
NOTE This repository contains models that have been converted to the GGUF format with various quantizations from an IBM Granite .safetensors model. Please refe…
Runs locally from ~1.28 GB disk (4 GB VRAM class GPUs with llama.cpp / guIDE).
Repository Files & Downloads
15 GGUF files detected
Direct downloads for local inference
| File | Type | Quantization | Size | Link |
|---|---|---|---|---|
| granite-4.1-3b-Q2_K.gguf | GGUF | Q2_K | 1.28 GB | Download |
| granite-4.1-3b-Q3_K_L.gguf | GGUF | Q3_K_L | 1.74 GB | Download |
| granite-4.1-3b-Q3_K_M.gguf | GGUF | Q3_K_M | 1.61 GB | Download |
| granite-4.1-3b-Q3_K_S.gguf | GGUF | Q3_K_S | 1.46 GB | Download |
| granite-4.1-3b-Q4_0.gguf | GGUF | Q4_0 | 1.85 GB | Download |
| granite-4.1-3b-Q4_1.gguf | GGUF | Q4_1 | 2.03 GB | Download |
| granite-4.1-3b-Q4_K_M.gguf | GGUF | Q4_K_M | 1.96 GB | Download |
| granite-4.1-3b-Q4_K_S.gguf | GGUF | Q4_K_S | 1.86 GB | Download |
| granite-4.1-3b-Q5_0.gguf | GGUF | Q5_0 | 2.21 GB | Download |
| granite-4.1-3b-Q5_1.gguf | GGUF | Q5_1 | 2.40 GB | Download |
| granite-4.1-3b-Q5_K_M.gguf | GGUF | Q5_K_M | 2.27 GB | Download |
| granite-4.1-3b-Q5_K_S.gguf | GGUF | Q5_K_S | 2.21 GB | Download |
| granite-4.1-3b-Q6_K.gguf | GGUF | Q6_K | 2.60 GB | Download |
| granite-4.1-3b-Q8_0.gguf | GGUF | Q8_0 | 3.37 GB | Download |
| granite-4.1-3b-bf16.gguf | GGUF | BF16 | 6.34 GB | Download |
Model Details
| Model ID | armandosds/granite-4.1-3b-GGUF |
|---|---|
| Author | armandosds |
| Pipeline | text-generation |
| License | apache-2.0 |
| Base model | ibm-granite/granite-4.1-3b |
| Last modified | 2026-07-10T18:29:37.000Z |
Model README
---
pipeline_tag: text-generation
inference: false
license: apache-2.0
library_name: transformers
tags:
- language
- granite-4.1
- gguf
base_model:
- ibm-granite/granite-4.1-3b
---
> [!NOTE]
> This repository contains models that have been converted to the GGUF format with various quantizations from an IBM Granite .safetensors model.
>
> Please reference the base model's full model card here:
> https://huggingface.co/ibm-granite/granite-4.1-3b
Run armandosds/granite-4.1-3b-GGUF with guIDE
Download guIDE — the AI-native code editor with local LLM inference and 69 built-in tools.
Source: Hugging Face · Compare models