---

license: apache-2.0

language:

- en

base_model: ['unsloth/Qwen3-4B']

datasets:

- ermiaazarkhalili/Fable-5-Complete-2M-Clean

library_name: transformers

pipeline_tag: text-generation

tags:

- gguf

- llama.cpp

- quantized

- ollama

- lm-studio

- text-generation

---

Qwen3-4B-SFT-Fable5-GGUF

GGUF quantizations of Qwen3-4B-SFT-Fable5, for local inference

with llama.cpp, Ollama, or LM Studio.

Overview

| Property | Value |

|---|---|

| Source model | Qwen3-4B-SFT-Fable5 |

| Base model | [['unsloth/Qwen3-4B']](https://huggingface.co/['unsloth/Qwen3-4B']) |

| Training dataset | ermiaazarkhalili/Fable-5-Complete-2M-Clean |

| Architecture | Qwen3ForCausalLM |

| Parameters | 4.02 B |

| Layers | 36 |

| Vocab size | 151,936 |

| License | APACHE-2.0 |

| Developed by | Behrooz Azarkhalili |

Available quantizations

| File | Quant | Size | Notes |

|---|---|---:|---|

| qwen3-4b-sft-fable5.q4_k_m.gguf | Q4_K_M | 2.33 GiB | Smallest here; the common choice for local inference on limited VRAM. |

| qwen3-4b-sft-fable5.q5_k_m.gguf | Q5_K_M | 2.69 GiB | Balanced. The usual default when Q4_K_M feels lossy. |

| qwen3-4b-sft-fable5.q8_0.gguf | Q8_0 | 3.99 GiB | Largest, closest to the merged weights. Use when disk is not the constraint. |

Sizes above are the actual file sizes in this repository, not estimates.

Quick start

Ollama

ollama pull hf.co/ermiaazarkhalili/Qwen3-4B-SFT-Fable5-GGUF:Q4_K_M

llama.cpp

llama-cli -hf ermiaazarkhalili/Qwen3-4B-SFT-Fable5-GGUF:Q4_K_M -p "Tell me a fable about a clever fox." -n 256

Download a single file

from huggingface_hub import hf_hub_download

path = hf_hub_download("ermiaazarkhalili/Qwen3-4B-SFT-Fable5-GGUF", "qwen3-4b-sft-fable5.q4_k_m.gguf")
print(path)

Limitations

• Quantization is lossy: outputs will not match the source model exactly, and the

effect grows as the quant level drops.

• Primarily English training data.
• Not safety-tuned. Use appropriate guardrails.
• Research and prototyping, not production without further evaluation.

Citation

@misc{azarkhalili2026_qwen3_4b_sft_fable5_gguf,
    author = {Azarkhalili, Behrooz},
    title = {Qwen3-4B-SFT-Fable5-GGUF},
    year = {2026},
    publisher = {Hugging Face},
    url = {https://huggingface.co/ermiaazarkhalili/Qwen3-4B-SFT-Fable5-GGUF}
}

Acknowledgments

• llama.cpp for the GGUF format and tooling
• Unsloth and Hugging Face TRL
• Compute Canada / DRAC for HPC resources