---

license: apache-2.0

language:

- en

base_model: kai-os/Carnice-9b

datasets:

- ermiaazarkhalili/Fable-5-Complete-2M-Clean

library_name: transformers

pipeline_tag: text-generation

tags:

- gguf

- llama.cpp

- quantized

- ollama

- lm-studio

- text-generation

---

Carnice-9B-SFT-Fable5-GGUF

GGUF quantizations of Carnice-9B-SFT-Fable5, for local inference

with llama.cpp, Ollama, or LM Studio.

Overview

| Property | Value |

|---|---|

| Source model | Carnice-9B-SFT-Fable5 |

| Base model | kai-os/Carnice-9b |

| Training dataset | ermiaazarkhalili/Fable-5-Complete-2M-Clean |

| Architecture | Qwen3_5ForCausalLM |

| Parameters | 8.95 B |

| Layers | 32 |

| Vocab size | 248,320 |

| License | APACHE-2.0 |

| Developed by | Behrooz Azarkhalili |

Available quantizations

| File | Quant | Size | Notes |

|---|---|---:|---|

| carnice-9b-sft-fable5.q4_k_m.gguf | Q4_K_M | 5.24 GiB | Smallest here; the common choice for local inference on limited VRAM. |

| carnice-9b-sft-fable5.q5_k_m.gguf | Q5_K_M | 6.02 GiB | Balanced. The usual default when Q4_K_M feels lossy. |

| carnice-9b-sft-fable5.q8_0.gguf | Q8_0 | 8.87 GiB | Largest, closest to the merged weights. Use when disk is not the constraint. |

Sizes above are the actual file sizes in this repository, not estimates.

Quick start

Ollama

ollama pull hf.co/ermiaazarkhalili/Carnice-9B-SFT-Fable5-GGUF:Q4_K_M

llama.cpp

llama-cli -hf ermiaazarkhalili/Carnice-9B-SFT-Fable5-GGUF:Q4_K_M -p "Tell me a fable about a clever fox." -n 256

Download a single file

from huggingface_hub import hf_hub_download

path = hf_hub_download("ermiaazarkhalili/Carnice-9B-SFT-Fable5-GGUF", "carnice-9b-sft-fable5.q4_k_m.gguf")
print(path)

Limitations

• Quantization is lossy: outputs will not match the source model exactly, and the

effect grows as the quant level drops.

• Primarily English training data.
• Not safety-tuned. Use appropriate guardrails.
• Research and prototyping, not production without further evaluation.

Citation

@misc{azarkhalili2026_carnice_9b_sft_fable5_gguf,
    author = {Azarkhalili, Behrooz},
    title = {Carnice-9B-SFT-Fable5-GGUF},
    year = {2026},
    publisher = {Hugging Face},
    url = {https://huggingface.co/ermiaazarkhalili/Carnice-9B-SFT-Fable5-GGUF}
}

Acknowledgments

• llama.cpp for the GGUF format and tooling
• Unsloth and Hugging Face TRL
• Compute Canada / DRAC for HPC resources