---

license: apache-2.0

language:

- en

library_name: gguf

pipeline_tag: text-generation

tags:

- gguf

- quantized

- llama-cpp

- ollama

- lm-studio

- sft

- distillation

- fable

- creative-writing

base_model: ermiaazarkhalili/Granite-4.1-3B-SFT-Fable5

---

Granite-4.1-3B-SFT-Fable5 — GGUF

Quantized GGUF builds of Granite-4.1-3B-SFT-Fable5, a

ibm-granite/granite-4.1-3b model supervised-fine-tuned on the FABLE-5 Complete-2M trace

corpus. These files run locally with llama.cpp,

Ollama, LM Studio, and any GGUF-compatible

runtime — no GPU required for the smaller quants.

Overview

| | |

|---|---|

| Fine-tuned model | Granite-4.1-3B-SFT-Fable5 |

| Base model | ibm-granite/granite-4.1-3b |

| Parameter class | 3B |

| Model family | dense |

| Training method | LoRA SFT (distillation), assistant-only loss masking |

| Dataset | FABLE-5 Complete-2M traces (private) |

| Format | GGUF (this repo) · safetensors (merged repo) |

What is FABLE-5 Complete-2M?

This model was fine-tuned on FABLE-5 Complete-2M, the full ~2M-trace FABLE-5 corpus

(cleaned). Each target completion may include a <think>…</think> reasoning span followed

by the response; training used assistant-only loss masking so the model learns to

produce the response, not echo the prompt. The dataset is private; the fine-tuned weights

are public.

Available Quantizations

| File | Quant | Size | Notes |

|------|-------|------|-------|

| granite-4.1-3b-sft-fable5.q4_k_m.gguf | Q4_K_M | ~2.1 GB | Recommended — best quality/size balance |

| granite-4.1-3b-sft-fable5.q5_k_m.gguf | Q5_K_M | ~2.4 GB | Higher quality |

| granite-4.1-3b-sft-fable5.q8_0.gguf | Q8_0 | ~3.6 GB | Maximum quality (near-lossless) |

Which to pick: Q4_K_M is the best size/quality trade-off for most users. Use Q5_K_M

if you have spare RAM/VRAM and want a little more fidelity, or Q8_0 for near-lossless

output when size is not a concern.

Usage

Ollama

ollama run hf.co/ermiaazarkhalili/Granite-4.1-3B-SFT-Fable5-GGUF:Q4_K_M "Write a short story about a clockwork fox."

llama.cpp

# One-shot
llama-cli -hf ermiaazarkhalili/Granite-4.1-3B-SFT-Fable5-GGUF --jinja -p "Write a short fable about ambition." -n 512
# Interactive chat
llama-cli -hf ermiaazarkhalili/Granite-4.1-3B-SFT-Fable5-GGUF --jinja -cnv

llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
    repo_id="ermiaazarkhalili/Granite-4.1-3B-SFT-Fable5-GGUF",
    filename="*q4_k_m.gguf",
    n_ctx=4096,
)
out = llm.create_chat_completion(
    messages=[{"role": "user", "content": "Write a short fable about ambition."}],
    max_tokens=512,
)
print(out["choices"][0]["message"]["content"])

Intended use & limitations

Research and non-commercial experimentation with FABLE-5-style creative / agentic

generation. As GGUF quantizations these carry unavoidable quality loss versus the source

safetensors weights — prefer Q8_0 when fidelity matters. Inherits every limitation of the

base model ibm-granite/granite-4.1-3b and the source fine-tune Granite-4.1-3B-SFT-Fable5. Verify

outputs before any downstream use.

Citation

@misc{granite_4_1_3b_fable5_gguf,
  author       = {Ermia Azarkhalili},
  title        = {Granite-4.1-3B-SFT-Fable5 — GGUF quantized},
  year         = {2026},
  publisher    = {Hugging Face},
  howpublished = {\url{https://huggingface.co/ermiaazarkhalili/Granite-4.1-3B-SFT-Fable5-GGUF}}
}