---

license: apache-2.0

base_model: poolside/Laguna-XS-2.1

base_model_relation: quantized

pipeline_tag: text-generation

library_name: gguf

tags:

- gguf

- moe

- apex

- quantized

- imatrix

- llama.cpp

- laguna

---

Laguna-XS-2.1 — APEX GGUF

APEX (dynamic, role/band-aware bit allocation) and standard GGUF quants of

poolside/Laguna-XS-2.1 — a 40-layer,

256-routed-expert (+ shared expert) MoE with alternating attention/dense blocks, real

BF16 total size 62.35 GiB. This is a genuine instruct/chat model (has a working chat

template with a <think> reasoning channel) — verified with a real chat completion, not

just inferred from metadata.

update:

-v2 files: identical bit allocation to the originals, except the 48 attn_gate coefficients are kept at F32 instead of the tier's attention band type. They parameterise a nonlinearity that multiplies the state path, so quantising them lets error compound along the sequence; F32 costs about 31 MB. Prefer -v2.

Sizes

| tier | precision | file size | bits/weight | PPL (wikitext, 100×512-tok chunks) | Δ vs bf16 |

|---|---|---|---|---:|---:|

| bf16 (reference) | full | 62.35 GiB | 16.01 | 13.9091 | — |

| i-quality | Q6_K/Q4_K/Q3_K by role+band (dynamic-APEX) | 19.18 GiB | 4.59 | 14.0440 | +0.97% |

| APEX-dynamic (16GB-card) | Q3_K/IQ3_XXS/IQ2_XXS by role+band | 11.25 GiB | ~2.87 | 15.7621 | +13.3% |

| IQ3_XXS (stock, imatrix-guided) | uniform | 12.06 GiB | ~3.06 | 15.5641 | +11.9% |

On the 16GB-card size point specifically: dynamic-APEX lands smaller than a stock

IQ3_XXS at the same rough budget (11.25 vs 12.06 GiB) but scores very slightly worse on

raw PPL (15.76 vs 15.56, +1.3% relative to each other) — a real, honestly-reported

result, not a clean win for either approach at this exact size/architecture point. The

value of the dynamic-APEX tier here is landing precisely on a chosen VRAM budget via

role/depth-aware allocation, not a free quality win over a well-tuned stock quant.

i-quality is the tier to reach for if you're not VRAM-constrained to 16GB — it

recovers to within 1% of full bf16 quality at under a third of the size.

Structural allocation (dynamic-APEX)

Both APEX tiers use dynamic_apex.py's structural mode: routed experts are bucketed into

edge / near / middle depth-bands (edge = first/last layers, generally most load-bearing)

and allocated decreasing precision toward the middle:

• i-quality (19.18 GiB, 4.59 bpw): routed edge → Q6_K, near → Q4_K, middle → Q3_K.
• APEX-dynamic (11.25 GiB, ~2.87 bpw): routed edge → Q3_K, near → IQ3_XXS, middle → IQ2_XXS.

Attention, shared-expert, and router tensors stay at higher fixed precision (Q6_K–Q8_0)

in both tiers.

Calibration

Single imatrix (laguna-xs.imatrix) generated with llama-imatrix over a diverse,

code-weighted local corpus, used identically for every quant in this repo — only the

bit-allocation differs between tiers, isolating that as the sole variable in the size/PPL

comparison above.

Files

• laguna-xs.imatrix — the imatrix used for every quant below.
• Laguna-XS-2.1-APEX-i-quality.gguf (19.18 GiB) — recommended default; best

quality-per-byte of the tiers here.

• Laguna-XS-2.1-APEX-dynamic.gguf (11.25 GiB) — role-aware allocation targeting a

16GB-card budget.

• Laguna-XS-2.1-IQ3_XXS.gguf (12.06 GiB) — stock imatrix-guided baseline, included for

the size/PPL comparison above (not the recommended pick — i-quality or APEX-dynamic

both do better per byte).

Attribution

• Base model: Poolside — poolside/Laguna-XS-2.1.
• APEX recipe & toolkit: LocalAI — localai-org/apex-quant.
• Quantization engine: llama.cpp (ggml-org).

Unofficial community quantization; not affiliated with or endorsed by Poolside.