---

license: apache-2.0

base_model: inception42/Jais-3-7B

base_model_relation: quantized

library_name: gguf

pipeline_tag: text-generation

language:

- ar

- en

tags:

- sakura

- gguf

- imatrix

- jais-3

- k2-horizon

- arabic

---

Sakura — Jais-3-7B (GGUF)

GGUF quantizations of Jais-3-7B (Arabic and English, dense, K2 Horizon architecture, 36 layers, 4096 hidden size, context up to 524,288 tokens by its config). The official GGUF repository IFM/Jais-3-7B-GGUF already exists; this repository differs in that every file up to 5 bits was made with an importance matrix (calibration text below) and in what we measured. The version with edited weights is Sakura-Jais-3-7B-Uncensored-GGUF.

What was done

• We converted the original safetensors checkpoint (inception42/Jais-3-7B, revision 94d69f49) to a BF16 GGUF with convert_hf_to_gguf.py of a llama.cpp checkout of 2026-10-07 (commit 9c2e0e491), then quantized with llama-quantize of llama.cpp b11594.
• The importance matrix was computed on the BF16 file with llama-imatrix (96 chunks of 512 tokens of our calib_mix_v1 text mix, which contains English and German but no Arabic). Q8_0 and Q6_K were made without it.
• No weights were edited.

Files

| File | Size | What it is |

|---|---:|---|

| Sakura-Jais-3-7B-Q8_0-8.92GiB.gguf | 8.92 GiB | Q8_0 (no importance matrix) |

| Sakura-Jais-3-7B-Q6_K-6.89GiB.gguf | 6.89 GiB | Q6_K (no importance matrix) |

| Sakura-Jais-3-7B-Q5_K_M-6.02GiB.gguf | 6.02 GiB | Q5_K_M with importance matrix |

| Sakura-Jais-3-7B-Q4_K_M-5.21GiB.gguf | 5.21 GiB | Q4_K_M with importance matrix |

| Sakura-Jais-3-7B-IQ4_XS-4.76GiB.gguf | 4.76 GiB | IQ4_XS with importance matrix |

Run

llama-server -m Sakura-Jais-3-7B-Q5_K_M-6.02GiB.gguf --jinja -ngl 99 -c 8192

Jais-3 is a model that always thinks: its chat template opens <ifm|think>, <ifm|think_fast> or <ifm|think_faster> for reasoning_effort high (default), medium or low. Pass the effort per request as chat_template_kwargs: {"reasoning_effort": "low"}. The architecture (K2 Horizon) needs a llama.cpp build with the support merged on 2026-10-06 (PR 29535); we used b11594, older builds do not know it. enable_thinking is not used by this template.

What we measured

Distance to Q8_0 (KL divergence of the next-token distribution of each file against our Q8_0 of the same model, llama-perplexity b11594, 20 chunks of 2048 tokens of an English (en) and a web (web) evaluation text; mean ± standard error; the reference is Q8_0, not the BF16 file). The self-check (Q8_0 against itself) gave 99.995 % same top token and KLD 0.

| File | Size | KLD en | KLD web | same top token en / web |

|---|---:|---:|---:|---|

| Q6_K | 6.89 GiB | 0.0157 ± 0.0006 | 0.0072 ± 0.0002 | 94.9 / 95.3 % |

| Q5_K_M | 6.02 GiB | 0.0138 ± 0.0008 | 0.0073 ± 0.0001 | 95.1 / 95.1 % |

| Q4_K_M | 5.21 GiB | 0.0320 ± 0.0012 | 0.0168 ± 0.0002 | 92.6 / 92.6 % |

| IQ4_XS | 4.76 GiB | 0.0601 ± 0.0022 | 0.0234 ± 0.0003 | 90.6 / 91.4 % |

The perplexity of the Q8_0 file on the same texts is 8.51 ± 0.18 (en) and 12.69 ± 0.24 (web). Q5_K_M (with importance matrix) is not further from Q8_0 than Q6_K (without); the 4-bit files are roughly two to three times (Q4_K_M) and three to four times (IQ4_XS) as far from Q8_0 as Q5_K_M, so we would take Q5_K_M or larger when memory allows.

Smoke test (Q8_0 and the uncensored Q8_0 of the sister repository, llama-server b11594, reasoning_effort low, temperature 0): three short questions, one English and two Arabic; all answers were coherent and in the language of the question.

Not measured: anything in Arabic (quality of the quantizations, perplexity, benchmarks), any capability benchmark, speed, long contexts, tool calls, the high and medium reasoning efforts.

License

Apache-2.0 as the original model (LICENSE). See NOTICE.

Credits

• Model: inception42/Jais-3-7B and the official GGUF IFM/Jais-3-7B-GGUF.
• Quantization and measurement: ggml-org/llama.cpp (MIT).

---

中文版见 README_zh.md。