---

license: apache-2.0

base_model:

- amalia-llm/AMALIA-9B-0626-SFT

base_model_relation: quantized

library_name: llama.cpp

pipeline_tag: text-generation

tags:

- gguf

- quantized

- llama-cpp

- imatrix

- llama

- portuguese

- european-portuguese

- kld-measured

- conversational

- endpoints_compatible

language:

- pt

---

!banner

<div style="border:2px solid currentColor; font-family:ui-monospace,'SF Mono','Cascadia Mono',Consolas,monospace;">

<div style="border-bottom:1px solid currentColor; padding:6px 12px; font-size:11px; letter-spacing:3px; text-transform:uppercase; opacity:0.7; text-align:center;">KIKOCIS // EUROPEAN-PORTUGUESE LLM // IMATRIX GGUF + KLD</div>

<div style="padding:14px; display:flex; flex-wrap:wrap; align-items:center; justify-content:center; gap:18px;">

<pre style="margin:0; font-size:9px; line-height:1.15;">

╔═══════════════╗

║ A M A L I A ║ pt-PT

╚═══════════════╝

┌──┐ ┌──┐ ┌──┐ ┌──┐

│Q3│ │Q4│ │Q5│ │Q8│ ← ladder

└──┘ └──┘ └──┘ └──┘

●────●────●────●

KLD vs F16

</pre>

<div style="text-align:center;">

<div style="font-size:23px; font-weight:800; letter-spacing:1px;">AMALIA-9B · GGUF</div>

<div style="font-size:12.5px; letter-spacing:1px; opacity:0.8; margin-top:5px;"><span style="white-space:nowrap;">llama · 9B</span> · <span style="white-space:nowrap;">32K ctx</span> · <span style="white-space:nowrap;">imatrix (pt)</span> · <span style="white-space:nowrap;">KLD-measured</span></div>

</div>

</div>

<table style="display:table; table-layout:fixed; width:100%; margin:0; border-collapse:collapse; font-size:12px;">

<tr>

<td style="border-top:1px solid currentColor; border-right:1px solid currentColor; padding:8px 12px;"><div style="font-size:10px; letter-spacing:1px; opacity:0.6;">FORMAT</div><div style="font-weight:700;">GGUF (imatrix)</div></td>

<td style="border-top:1px solid currentColor; border-right:1px solid currentColor; padding:8px 12px;"><div style="font-size:10px; letter-spacing:1px; opacity:0.6;">SIZES</div><div style="font-weight:700;">3.1 – 9.1 GB</div></td>

<td style="border-top:1px solid currentColor; border-right:1px solid currentColor; padding:8px 12px;"><div style="font-size:10px; letter-spacing:1px; opacity:0.6;">ARCH</div><div style="font-weight:700;">Llama · 42L</div></td>

<td style="border-top:1px solid currentColor; padding:8px 12px;"><div style="font-size:10px; letter-spacing:1px; opacity:0.6;">CONTEXT</div><div style="font-weight:700;">32768</div></td>

</tr>

<tr>

<td style="border-top:1px solid currentColor; border-right:1px solid currentColor; padding:8px 12px;"><div style="font-size:10px; letter-spacing:1px; opacity:0.6;">IMATRIX</div><div style="font-weight:700;">pt-PT corpus</div></td>

<td style="border-top:1px solid currentColor; border-right:1px solid currentColor; padding:8px 12px;"><div style="font-size:10px; letter-spacing:1px; opacity:0.6;">VALIDATION</div><div style="font-weight:700;">KLD vs F16</div></td>

<td style="border-top:1px solid currentColor; border-right:1px solid currentColor; padding:8px 12px;"><div style="font-size:10px; letter-spacing:1px; opacity:0.6;">LANGUAGE</div><div style="font-weight:700;">Português (EU)</div></td>

<td style="border-top:1px solid currentColor; padding:8px 12px;"><div style="font-size:10px; letter-spacing:1px; opacity:0.6;">LICENSE</div><div style="font-weight:700;">Apache-2.0</div></td>

</tr>

</table>

</div>

AMALIA-9B-0626-SFT — GGUF (imatrix + KLD)

> imatrix-quantized GGUFs of AMALIA-9B, an open European-Portuguese instruction model — one of the first open LLMs developed in Portugal. Runs from ~3.1 GB (IQ2_M) to ~9.1 GB (Q8_0). Calibrated (imatrix) on Portuguese text, and every quant is measured against the full-precision model with KLD so you can pick a size with the fidelity cost in front of you. Credit: this is the AMALIA-LLM project's model — amalia-llm/AMALIA-9B-0626-SFT; ours is the quant ladder + metrics.

✅ Recommended files

| Use case | File | Notes |

|---|---|---|

| ⭐ Best value | amalia-9b-Q5_K_M.gguf | The pick — lower KLD than Q6_K at 1 GB less (0.051 vs 0.078). ~6.1 GB. |

| Safe default | amalia-9b-Q4_K_M.gguf | Standard K-quant, ~5.2 GB; noticeably less faithful than Q5 (KLD 0.22 vs 0.05). |

| Max fidelity | amalia-9b-Q8_0.gguf | Lowest KLD (0.006) — near-lossless. ~9.1 GB. |

| Tightest RAM | amalia-9b-Q3_K_M.gguf | ~4.2 GB — runs on 8 GB machines, still coherent. |

| Smallest (aggressive) | amalia-9b-IQ2_M.gguf | ~3.1 GB — fits very tight RAM, at a real quality cost (top-1 61%). |

📦 Files (the full ladder)

| Quant | Bits | File size | Notes |

|---|---|---:|---|

| IQ2_M | ~2.7 | ~3.1 GB | Smallest — aggressive i-quant, real quality trade-off. |

| Q3_K_M | 3 | ~4.2 GB | Smallest comfortable quant. |

| IQ4_XS | ~4.3 | ~4.7 GB | Compact i-quant (imatrix). |

| Q4_K_M | 4 | ~5.2 GB | Standard K-quant. |

| Q5_K_M | 5 | ~6.1 GB | ⭐ Best quality/size. |

| Q6_K | 6 | ~7.0 GB | Near-lossless. |

| Q8_0 | 8 | ~9.1 GB | Highest precision. |

<!-- K-quants (Q_K) = solid general quants. I-quants (IQ) = smaller, need an imatrix (shipped). Higher number = more bits = closer to the original + bigger. -->

📊 Metrics — objective quality vs the F16 reference

KLD (Kullback–Leibler divergence, nats) measures how far each quant's output distribution drifts from the full-precision model — the gold-standard fidelity metric (lower = closer). Top-1 match = how often the quant's top token agrees with F16. Measured with llama-perplexity --kl-divergence over a Portuguese corpus at ctx 2048, vs the F16 GGUF.

| Model | Size GB | PPL | ΔPPL vs F16 | KLD mean | KLD p95 | Top-1 match |

|---|---:|---:|---:|---:|---:|---:|

| F16 (reference) | 17.1 | 20.04 | 0.000 | 0.0000 | 0.0000 | 100.0% |

| Q8_0 | 9.06 | 20.20 | +0.177 | 0.0057 | 0.0198 | 96.38% |

| Q6_K | 7.00 | 19.93 | -0.091 | 0.0779 | 0.1740 | 89.06% |

| Q5_K_M | 6.08 | 19.80 | -0.227 | 0.0511 | 0.1747 | 88.89% |

| Q4_K_M | 5.20 | 20.23 | +0.203 | 0.2230 | 0.7429 | 81.39% |

| IQ4_XS | 4.70 | 20.78 | +0.757 | 0.2173 | 0.7600 | 80.55% |

| Q3_K_M | 4.25 | 22.24 | +2.215 | 0.3258 | 1.1486 | 73.58% |

| IQ2_M | 3.11 | 21.99 | +1.968 | 0.8211 | 2.9052 | 60.69% |

_{Full per-quant reports in reports/; machine-readable summary in metrics/quant-summary.csv; SHA-256 of every file in reports/artifact-sha256sums.txt. Note: this is an instruction-tuned model measured on raw text, so the absolute PPL is higher than a base model's — what matters here is the relative KLD/top-1 across quants.}

> Why does Q5_K_M beat Q6_K on KLD here? It's real and reproducible for this model: Q5_K_M's imatrix-weighted bit allocation is very effective, so it lands closer to F16 on mean-KLD than Q6_K (which stays marginally ahead on top-1). Both are near-lossless — so Q5_K_M is the better value, and Q8_0 is the true fidelity ceiling.

> A note on i-quants: we tested the i-quants too. For this model the K-quants win at low bit — e.g. IQ3_M came out worse than Q3_K_M on KLD, so it isn't shipped. IQ2_M is included as the smallest option for very tight RAM, but with an honest quality cost (top-1 61%); if you can spare the space, Q3_K_M is a big step up.

📈 Charts

!kld vs size

!top-1 match

🧮 Will it fit? (RAM/VRAM cheat-sheet)

Memory ≈ weights + KV-cache (KV grows with context). Rough guide:

| you have | comfortable quant | context |

|---|---|---|

| 6 GB | IQ2_M / Q3_K_M | ~8–16K |

| 8 GB | Q3_K_M / Q4_K_M | ~16–32K |

| 12 GB | Q5_K_M / Q6_K | ~32K |

| 16 GB+ | Q6_K / Q8_0 | ~32K (native) |

🚀 How to run it

# ollama (pulls straight from this repo)
ollama run hf.co/KikoCis/AMALIA-9B-0626-SFT-GGUF:Q5_K_M

# llama.cpp
llama-server -m amalia-9b-Q5_K_M.gguf -c 32768 --jinja
llama-cli   -m amalia-9b-Q5_K_M.gguf -p "Explica em português o que é a saudade." -ngl 99

Recommended sampling: temperature ~0.7, top_p ~0.9. It's a chat/instruct model (uses its built-in chat template) — prompt it in European Portuguese for best results.

⚠️ Good to know

• Strengths: European-Portuguese fluency and instruction-following; one of the first open LLMs built in Portugal.
• Limits: specialised for Portuguese — not a coding/agentic model, and not tuned for English. Low-bit quants (IQ2/Q3) trade real fidelity for size (see KLD).
• Absolute perplexity looks high only because it's measured on raw text with an instruct model; the KLD table is the fair quant-quality signal.

📊 Evaluation methodology (how the numbers were measured)

• What: quantization fidelity vs the F16 GGUF — not a task benchmark. llama-perplexity --kl-divergence (KLD mean/p95/max, ΔPPL, top-1 agreement).
• Corpus: European-Portuguese text (Wikipedia + diverse), ctx 2048, same corpus family used for the imatrix calibration.
• Reference: the model's own F16 GGUF (KLD = 0 by definition).
• Date: 2026-07. Caveat: relative fidelity ranking across quants of this model; not comparable across different models/corpora.

🔁 Provenance & reproducibility

• Scripts: scripts/ — exact convert → imatrix → quant → KLD commands.
• imatrix: amalia-9b-pt.imatrix — importance matrix (Portuguese calibration) used for the IQ/K quants.
• Checksums: reports/artifact-sha256sums.txt — shasum -a 256 -c to verify.
• Source: amalia-llm/AMALIA-9B-0626-SFT — weights unmodified (faithful quantization).

🗒️ Changelog

• 2026-07 v1: imatrix GGUF ladder (IQ2_M → Q8_0) + KLD/PPL metrics, Portuguese-calibrated.

📚 Credit & license

Model, weights, training data: © the AMALIA-LLM project — model · website · paper (PROPOR 2026). Quant ladder + imatrix (pt) + KLD/PPL metrics: KikoCis. Apache-2.0 (same as upstream). No weights modified.