---

license: apache-2.0

base_model:

- BSC-LT/ALIA-40b-instruct-2601

base_model_relation: quantized

library_name: llama.cpp

pipeline_tag: text-generation

tags:

- gguf

- quantized

- llama-cpp

- imatrix

- llama

- spanish

- multilingual

- european

- sovereign-ai

- kld-measured

- conversational

- endpoints_compatible

language:

- es

- ca

- gl

- eu

- en

---

!banner

<div style="border:2px solid currentColor; font-family:ui-monospace,'SF Mono','Cascadia Mono',Consolas,monospace;">

<div style="border-bottom:1px solid currentColor; padding:6px 12px; font-size:11px; letter-spacing:3px; text-transform:uppercase; opacity:0.7; text-align:center;">KIKOCIS // EU-SOVEREIGN LLM // IMATRIX GGUF + KLD</div>

<div style="padding:14px; display:flex; flex-wrap:wrap; align-items:center; justify-content:center; gap:18px;">

<pre style="margin:0; font-size:9px; line-height:1.15;">

╔═══════════════════╗

║ A L I A · 40B ║ ES·CA·GL·EU

╚═══════════════════╝

┌───┐ ┌───┐ ┌───┐ ┌───┐

│IQ2│ │Q3 │ │Q4 │ │Q8 │

└───┘ └───┘ └───┘ └───┘

●─────●─────●─────●

KLD vs Q8-ref

</pre>

<div style="text-align:center;">

<div style="font-size:23px; font-weight:800; letter-spacing:1px;">ALIA-40b · GGUF</div>

<div style="font-size:12.5px; letter-spacing:1px; opacity:0.8; margin-top:5px;"><span style="white-space:nowrap;">llama · 46B</span> · <span style="white-space:nowrap;">160K ctx</span> · <span style="white-space:nowrap;">imatrix (es)</span> · <span style="white-space:nowrap;">KLD-measured</span></div>

</div>

</div>

<table style="display:table; table-layout:fixed; width:100%; margin:0; border-collapse:collapse; font-size:12px;">

<tr>

<td style="border-top:1px solid currentColor; border-right:1px solid currentColor; padding:8px 12px;"><div style="font-size:10px; letter-spacing:1px; opacity:0.6;">FORMAT</div><div style="font-weight:700;">GGUF (imatrix)</div></td>

<td style="border-top:1px solid currentColor; border-right:1px solid currentColor; padding:8px 12px;"><div style="font-size:10px; letter-spacing:1px; opacity:0.6;">SIZES</div><div style="font-weight:700;">~13.5 – 40 GB</div></td>

<td style="border-top:1px solid currentColor; border-right:1px solid currentColor; padding:8px 12px;"><div style="font-size:10px; letter-spacing:1px; opacity:0.6;">ARCH</div><div style="font-weight:700;">Llama · 46B · 48L</div></td>

<td style="border-top:1px solid currentColor; padding:8px 12px;"><div style="font-size:10px; letter-spacing:1px; opacity:0.6;">CONTEXT</div><div style="font-weight:700;">163840</div></td>

</tr>

<tr>

<td style="border-top:1px solid currentColor; border-right:1px solid currentColor; padding:8px 12px;"><div style="font-size:10px; letter-spacing:1px; opacity:0.6;">IMATRIX</div><div style="font-weight:700;">es corpus</div></td>

<td style="border-top:1px solid currentColor; border-right:1px solid currentColor; padding:8px 12px;"><div style="font-size:10px; letter-spacing:1px; opacity:0.6;">VALIDATION</div><div style="font-weight:700;">KLD vs Q8</div></td>

<td style="border-top:1px solid currentColor; border-right:1px solid currentColor; padding:8px 12px;"><div style="font-size:10px; letter-spacing:1px; opacity:0.6;">LANGUAGES</div><div style="font-weight:700;">ES·CA·GL·EU·EN</div></td>

<td style="border-top:1px solid currentColor; padding:8px 12px;"><div style="font-size:10px; letter-spacing:1px; opacity:0.6;">LICENSE</div><div style="font-weight:700;">Apache-2.0</div></td>

</tr>

</table>

</div>

ALIA-40b-instruct-2601 — GGUF (imatrix + KLD)

> imatrix-quantized GGUFs of ALIA-40b, the Barcelona Supercomputing Center's ~46B sovereign LLM for Spain / the EU (Spanish + Catalan, Galician, Basque + European languages), with a 160K native context. Runs from ~13.5 GB (IQ2_M) to ~40 GB (Q8_0). Calibrated (imatrix) on Spanish text, with per-quant KLD fidelity vs the near-lossless Q8_0 reference. Credit: this is BSC-LT's model — BSC-LT/ALIA-40b-instruct-2601; ours is the quant ladder + metrics.

✅ Recommended files

| Use case | File | Notes |

|---|---|---|

| Safe default | ALIA-40b-instruct-2601-Q4_K_M.gguf | The one most people want — good quality, ~23 GB (48 GB RAM). |

| Strong quality/size | ALIA-40b-instruct-2601-Q5_K_M.gguf | Closer to the reference, ~27 GB. |

| Runs on 32 GB | ALIA-40b-instruct-2601-Q3_K_M.gguf | ~19 GB — fits a 32 GB machine. |

| Smallest (tight RAM) | ALIA-40b-instruct-2601-IQ2_M.gguf | ~13.5 GB i-quant — runs a 46B on 24–32 GB, at a real quality cost. |

| Reference / max fidelity | ALIA-40b-instruct-2601-Q8_0.gguf | ~40 GB, near-lossless (the KLD reference). |

📦 Files (the ladder)

| Quant | Bits | File size | RAM (approx) | Notes |

|---|---|---:|---:|---|

| IQ2_M | ~2.7 | ~13.5 GB | 24–32 GB | Smallest — aggressive i-quant. |

| Q3_K_M | 3 | ~18.7 GB | 32 GB | Fits a 32 GB machine. |

| Q4_K_M | 4 | ~22.9 GB | 48 GB | Safe default. |

| Q5_K_M | 5 | ~26.8 GB | 48–64 GB | Strong quality/size. |

| Q8_0 | 8 | ~40 GB | 64 GB+ | Reference, near-lossless. |

<!-- A 46B is big: even IQ2_M needs ~24 GB RAM. Higher number = more bits = closer to the original + bigger. -->

📊 Metrics — fidelity vs the Q8_0 reference

KLD (Kullback–Leibler divergence, nats) measures how far each quant's output distribution drifts from the reference — lower = closer. Top-1 match = how often the quant's top token agrees with the reference. Measured with llama-perplexity --kl-divergence over a Spanish corpus at ctx 512.

> Why the reference is Q8_0, not F16? ALIA-40b's F16 is ~81 GB and does not fit this machine's GPU (Metal). Q8_0 is near-lossless (its own KLD vs F16 would be ~0.005), so it's a faithful stand-in reference for measuring how much the smaller quants drift. KLD values here are therefore relative to Q8_0 (Q8_0 = 0 by definition).

| Model | Size GB | KLD mean | KLD p95 | KLD max | Top-1 match |

|---|---:|---:|---:|---:|---:|

| Q8_0 (reference) | 40.0 | 0.0000 | 0.0000 | 0.0000 | 100.0% |

| Q5_K_M | 26.78 | 0.0098 | 0.0408 | 4.893 | 96.61% |

| Q4_K_M | 22.90 | 0.0337 | 0.1505 | 4.105 | 93.94% |

| Q3_K_M | 18.67 | 0.1114 | 0.5419 | 7.904 | 88.89% |

| IQ2_M | 13.54 | 0.3799 | 1.9178 | 12.259 | 77.82% |

_{Full per-quant reports in reports/; machine-readable summary in metrics/quant-summary.csv; SHA-256 of every file in reports/artifact-sha256sums.txt.}

📈 Charts

!kld vs size

!top-1 match

🧮 Will it fit? (RAM cheat-sheet)

A 46B is memory-hungry; add KV-cache on top (it grows with context — 160K is a lot).

| you have | quant | context |

|---|---|---|

| 24 GB | IQ2_M | ~8–16K |

| 32 GB | Q3_K_M / IQ2_M | ~16–32K |

| 48 GB | Q4_K_M / Q5_K_M | ~32–64K |

| 64 GB+ | Q8_0 | large (up to 160K with room) |

🚀 How to run it

# ollama
ollama run hf.co/KikoCis/ALIA-40b-instruct-2601-GGUF:Q4_K_M

# llama.cpp
llama-server -m ALIA-40b-instruct-2601-Q4_K_M.gguf -c 32768 --jinja -ngl 99

Recommended sampling: temperature ~0.7, top_p ~0.9. Chat/instruct model (uses its built-in template) — great for Spanish and the co-official languages (Catalan, Galician, Basque) + European languages.

⚠️ Good to know

• Strengths: a genuinely sovereign, EU-built 46B — strong Spanish + co-official + European multilinguality, 160K context, permissive licence.
• Limits: it's a 46B — even the smallest quant needs ~24 GB RAM; IQ2_M trades real quality for size. Not a specialised coding model.
• KLD is measured vs Q8_0 (F16 doesn't fit this GPU) — see the note above.

📊 Evaluation methodology

• What: quantization fidelity vs the Q8_0 reference — llama-perplexity --kl-divergence (KLD mean/p95/max, ΔPPL, top-1 agreement).
• Corpus: Spanish text, ctx 512, same corpus used for imatrix calibration.
• Reference: Q8_0 (near-lossless stand-in for F16, which is too large for this GPU).
• Date: 2026-07. Caveat: relative fidelity ranking across quants of this model.

🔁 Provenance & reproducibility

• Scripts: scripts/ — exact convert → Q8 → imatrix → quant → KLD commands.
• imatrix: alia-40b-es.imatrix — importance matrix (Spanish calibration).
• Checksums: reports/artifact-sha256sums.txt.
• Source: BSC-LT/ALIA-40b-instruct-2601 — weights unmodified (faithful quantization).

🗒️ Changelog

• 2026-07 v1: initial imatrix GGUF ladder (IQ2_M → Q8_0) + KLD metrics, Spanish-calibrated.

📚 Credit & license

Model, weights, training: © Barcelona Supercomputing Center — the ALIA / langtech-bsc project (model · ALIA-kit). Quant ladder + imatrix (es) + KLD metrics: KikoCis. Apache-2.0 (same as upstream). No weights modified.