Model Intelligence Sheet
mrutkows/granite-5.0-120b-sft-GGUF overview
granite 5.0 120b sft GGUF NOTE This repository contains models that have been converted to the GGUF format with various quantizations from an IBM Granite base …
Runs locally from ~7.00 GB disk (8 GB VRAM class GPUs with llama.cpp / guIDE).
Repository Files & Downloads
13 GGUF files detected
Direct downloads for local inference
| File | Type | Quantization | Size | Link |
|---|---|---|---|---|
| granite-5.0-120b-sft-Q4_K_M-00001-of-00002.gguf | GGUF | Q4_K_M | 41.52 GB | Download |
| granite-5.0-120b-sft-Q4_K_M-00002-of-00002.gguf | GGUF | Q4_K_M | 25.12 GB | Download |
| granite-5.0-120b-sft-Q5_K_M-00001-of-00002.gguf | GGUF | Q5_K_M | 41.62 GB | Download |
| granite-5.0-120b-sft-Q5_K_M-00002-of-00002.gguf | GGUF | Q5_K_M | 36.46 GB | Download |
| granite-5.0-120b-sft-Q6_K-00001-of-00003.gguf | GGUF | Q6_K | 41.63 GB | Download |
| granite-5.0-120b-sft-Q6_K-00002-of-00003.gguf | GGUF | Q6_K | 41.62 GB | Download |
| granite-5.0-120b-sft-Q6_K-00003-of-00003.gguf | GGUF | Q6_K | 7.00 GB | Download |
| granite-5.0-120b-sft-bf16-00001-of-00006.gguf | GGUF | BF16 | 40.99 GB | Download |
| granite-5.0-120b-sft-bf16-00002-of-00006.gguf | GGUF | BF16 | 41.01 GB | Download |
| granite-5.0-120b-sft-bf16-00003-of-00006.gguf | GGUF | BF16 | 41.01 GB | Download |
| granite-5.0-120b-sft-bf16-00004-of-00006.gguf | GGUF | BF16 | 41.01 GB | Download |
| granite-5.0-120b-sft-bf16-00005-of-00006.gguf | GGUF | BF16 | 41.01 GB | Download |
| granite-5.0-120b-sft-bf16-00006-of-00006.gguf | GGUF | BF16 | 14.84 GB | Download |
Model Details
Model README
---
license: apache-2.0
library_name: llama.cpp
tags:
- language
- granite-5.0
- gguf
base_model:
- ibm-research/granite-5.0-120b-sft
---
granite-5.0-120b-sft (GGUF)
> [!NOTE]
> This repository contains models that have been converted to the GGUF format with various quantizations from an IBM Granite base model.
>
> Please reference the base model's full model card here:
> https://huggingface.co/ibm-research/granite-5.0-120b-sft
<!-- #### Requirements
- llama.cpp build: $b9768$ -->
Run mrutkows/granite-5.0-120b-sft-GGUF with guIDE
Download guIDE — the AI-native code editor with local LLM inference and 69 built-in tools.
Source: Hugging Face · Compare models