---

license: apache-2.0

language:

- ar

- en

base_model: NightPrince/Muslim-6B-PRO

pipeline_tag: text-generation

tags:

- text-generation

- causal-lm

- arabic

- islamic

- tool-calling

- gguf

- llama.cpp

- voice-assistant

- quantized

---

<p align="center">

<img src="https://huggingface.co/NightPrince/Muslim-6B-PRO-GGUF/resolve/main/muslim-6b-pro-banner-light.png" alt="Muslim-6B-PRO" width="100%" />

</p>

Muslim-6B-PRO — GGUF

GGUF quantizations of NightPrince/Muslim-6B-PRO

for local inference with llama.cpp and compatible

runtimes (LM Studio, Ollama, koboldcpp, etc.).

Files

| File | Quant | Size | Notes |

|---|---|---|---|

| Muslim-6B-PRO-Q2_K.gguf | Q2_K | 2.26 GB | Smallest, largest quality loss |

| Muslim-6B-PRO-Q3_K_S.gguf | Q3_K_S | 2.57 GB | |

| Muslim-6B-PRO-Q3_K_M.gguf | Q3_K_M | 2.83 GB | |

| Muslim-6B-PRO-Q3_K_L.gguf | Q3_K_L | 3.06 GB | |

| Muslim-6B-PRO-Q4_0.gguf | Q4_0 | 3.24 GB | Legacy 4-bit |

| Muslim-6B-PRO-Q4_1.gguf | Q4_1 | 3.56 GB | Legacy 4-bit |

| Muslim-6B-PRO-Q4_K_S.gguf | Q4_K_S | 3.26 GB | |

| Muslim-6B-PRO-Q4_K_M.gguf | Q4_K_M | 3.41 GB | Recommended default — best size/quality balance |

| Muslim-6B-PRO-Q5_0.gguf | Q5_0 | 3.88 GB | Legacy 5-bit |

| Muslim-6B-PRO-Q5_1.gguf | Q5_1 | 4.19 GB | Legacy 5-bit |

| Muslim-6B-PRO-Q5_K_S.gguf | Q5_K_S | 3.88 GB | |

| Muslim-6B-PRO-Q5_K_M.gguf | Q5_K_M | 3.96 GB | Near-lossless, good balance for more headroom |

| Muslim-6B-PRO-Q6_K.gguf | Q6_K | 4.55 GB | Very close to F16 quality |

| Muslim-6B-PRO-Q8_0.gguf | Q8_0 | 5.89 GB | Near-lossless |

| Muslim-6B-PRO-f16.gguf | F16 | 11.08 GB | Full precision, no quantization loss |

Usage

llama.cpp

llama-cli -hf NightPrince/Muslim-6B-PRO-GGUF:Q4_K_M -p "السلام عليكم"

or, with a locally downloaded file:

llama-cli -m Muslim-6B-PRO-Q4_K_M.gguf -p "السلام عليكم"

Server mode (OpenAI-compatible API)

llama-server -m Muslim-6B-PRO-Q4_K_M.gguf --port 8080

Tool calling

This model uses the Hermes-style <tool_call> format. llama-server supports this natively via

its --jinja flag (uses the model's built-in chat template) combined with the standard

OpenAI-style tools parameter in requests.

Choosing a quant

• Q4_K_M is the recommended default for most use — the standard "good enough for almost

everyone" tradeoff.

• Go Q5_K_M or Q6_K if you have the VRAM/RAM headroom and want output closer to the

original fp16 model, especially for tool-call argument precision.

• Go Q2_K/Q3_K only under tight memory constraints — expect noticeably more degradation on

precise tasks like tool-call JSON formatting and exact surah/ayah references.

• Q8_0 or F16 for maximum fidelity when memory isn't a constraint.

Related resources

• Full-precision model card, training details, and dataset: NightPrince/Muslim-6B-PRO
• Live demo with real tool-calling (ZeroGPU): NightPrince/muslim-6b-pro-demo
• Fine-tuning code: github.com/NightPrinceY/Karnak-6B-Finetuning