---

license: apache-2.0

language:

- en

base_model: unsloth/Qwen3.5-4B

datasets:

- ermiaazarkhalili/Fable-5-Complete-2M-Clean

library_name: transformers

pipeline_tag: text-generation

tags:

- gguf

- llama.cpp

- quantized

- ollama

- lm-studio

- text-generation

---

Qwen3.5-4B-SFT-Fable5-GGUF

GGUF quantizations of Qwen3.5-4B-SFT-Fable5, for local inference

with llama.cpp, Ollama, or LM Studio.

Overview

| Property | Value |

|---|---|

| Source model | Qwen3.5-4B-SFT-Fable5 |

| Base model | unsloth/Qwen3.5-4B |

| Training dataset | ermiaazarkhalili/Fable-5-Complete-2M-Clean |

| Architecture | Qwen3_5ForConditionalGeneration |

| Parameters | 4.66 B |

| Layers | 32 |

| Vocab size | 248,320 |

| License | APACHE-2.0 |

| Developed by | Behrooz Azarkhalili |

Available quantizations

| File | Quant | Size | Notes |

|---|---|---:|---|

| qwen3.5-4b-sft-fable5.q4_k_m.gguf | Q4_K_M | 2.59 GiB | Smallest here; the common choice for local inference on limited VRAM. |

| qwen3.5-4b-sft-fable5.q5_k_m.gguf | Q5_K_M | 2.94 GiB | Balanced. The usual default when Q4_K_M feels lossy. |

| qwen3.5-4b-sft-fable5.q8_0.gguf | Q8_0 | 4.29 GiB | Largest, closest to the merged weights. Use when disk is not the constraint. |

Sizes above are the actual file sizes in this repository, not estimates.

Quick start

Ollama

ollama pull hf.co/ermiaazarkhalili/Qwen3.5-4B-SFT-Fable5-GGUF:Q4_K_M

llama.cpp

llama-cli -hf ermiaazarkhalili/Qwen3.5-4B-SFT-Fable5-GGUF:Q4_K_M -p "Tell me a fable about a clever fox." -n 256

Download a single file

from huggingface_hub import hf_hub_download

path = hf_hub_download("ermiaazarkhalili/Qwen3.5-4B-SFT-Fable5-GGUF", "qwen3.5-4b-sft-fable5.q4_k_m.gguf")
print(path)

Limitations

• Quantization is lossy: outputs will not match the source model exactly, and the

effect grows as the quant level drops.

• Primarily English training data.
• Not safety-tuned. Use appropriate guardrails.
• Research and prototyping, not production without further evaluation.

Citation

@misc{azarkhalili2026_qwen35_4b_sft_fable5_gguf,
    author = {Azarkhalili, Behrooz},
    title = {Qwen3.5-4B-SFT-Fable5-GGUF},
    year = {2026},
    publisher = {Hugging Face},
    url = {https://huggingface.co/ermiaazarkhalili/Qwen3.5-4B-SFT-Fable5-GGUF}
}

Acknowledgments

• llama.cpp for the GGUF format and tooling
• Unsloth and Hugging Face TRL
• Compute Canada / DRAC for HPC resources