---

license: apache-2.0

base_model: poolside/Laguna-XS-2.1

base_model_relation: quantized

pipeline_tag: text-generation

library_name: gguf

tags:

- gguf

- moe

- apex

- quantized

- imatrix

- llama.cpp

- laguna

---

Laguna-XS-2.1 — APEX GGUF

APEX (dynamic, role/band-aware bit allocation) and standard GGUF quants of

poolside/Laguna-XS-2.1 — a 40-layer,

256-routed-expert (+ shared expert) MoE with alternating attention/dense blocks, real

BF16 total size 62.35 GiB. This is a genuine instruct/chat model (has a working chat

template with a <think> reasoning channel) — verified with a real chat completion, not

just inferred from metadata.

update:

-v2 files: identical bit allocation to the originals, except the 48 attn_gate coefficients are kept at F32 instead of the tier's attention band type. They parameterise a nonlinearity that multiplies the state path, so quantising them lets error compound along the sequence; F32 costs about 31 MB. Prefer -v2.

Sizes

| tier | precision | file size | bits/weight | PPL (wikitext, 100×512-tok chunks) | Δ vs bf16 |

|---|---|---|---|---:|---:|

| bf16 (reference) | full | 62.35 GiB | 16.01 | 13.9091 | — |

| i-quality | Q6_K/Q4_K/Q3_K by role+band (dynamic-APEX) | 19.18 GiB | 4.59 | 14.0440 | +0.97% |

| APEX-dynamic (16GB-card) | Q3_K/IQ3_XXS/IQ2_XXS by role+band | 11.25 GiB | ~2.87 | 15.7621 | +13.3% |

| IQ3_XXS (stock, imatrix-guided) | uniform | 12.06 GiB | ~3.06 | 15.5641 | +11.9% |

On the 16GB-card size point specifically: dynamic-APEX lands smaller than a stock

IQ3_XXS at the same rough budget (11.25 vs 12.06 GiB) but scores very slightly worse on

raw PPL (15.76 vs 15.56, +1.3% relative to each other) — a real, honestly-reported

result, not a clean win for either approach at this exact size/architecture point. The

value of the dynamic-APEX tier here is landing precisely on a chosen VRAM budget via

role/depth-aware allocation, not a free quality win over a well-tuned stock quant.

i-quality is the tier to reach for if you're not VRAM-constrained to 16GB — it

recovers to within 1% of full bf16 quality at under a third of the size.

Agentic benchmark (OpenCode 2, 42 tasks)

Two runs per tier with the -v2 files. The harness is quantkit bench/ with OpenCode 2's real agentic loop,

graded by a fresh pytest. It ran on a DGX Spark (GB10) with stock llama.cpp 6d0549831, temp 1.0,

top-k 20, top-p 0.95, ctx 262144 and an output cap of 32768.

A full score is the entry bar. Between tiers that clear it, the discriminating number is output tokens on

the tasks every run passed.

| tier | run | fully passed | looping runs | output tokens on the 41 tasks all four runs passed |

|---|---|---|---|---:|

| i-quality-v2 | 1 | 42/42 | 1 | 169.8k |

| i-quality-v2 | 2 | 42/42 | 2 | 265.8k |

| APEX-dynamic-v2 | 1 | 41/42 | 2 | 204.5k |

| APEX-dynamic-v2 | 2 | 41/42 | 4 | 276.4k |

• i-quality-v2 is the pick for agentic coding. It passed every task in both runs.
• APEX-dynamic-v2 failed the same task in both runs. On btree_insert_delete it looped until the

one-hour limit, at about 100k tokens each time. i-quality-v2 solved that task on 5.7k and 38k tokens.

• Token totals on shared tasks favour i-quality-v2 by about 9% on average. That is inside the run-to-run

spread at two runs, because one frontier task alone swings between 28k and 82k tokens.

• APEX-dynamic-v2 is still a capable 12 GB coder if that is the memory you have. Expect more loops, and

set a wall-clock or token limit on long tasks.

Structural allocation (dynamic-APEX)

Both APEX tiers use dynamic_apex.py's structural mode: routed experts are bucketed into

edge / near / middle depth-bands (edge = first/last layers, generally most load-bearing)

and allocated decreasing precision toward the middle:

• i-quality (19.18 GiB, 4.59 bpw): routed edge → Q6_K, near → Q4_K, middle → Q3_K.
• APEX-dynamic (11.25 GiB, ~2.87 bpw): routed edge → Q3_K, near → IQ3_XXS, middle → IQ2_XXS.

Attention, shared-expert, and router tensors stay at higher fixed precision (Q6_K–Q8_0)

in both tiers.

Calibration

Single imatrix (laguna-xs.imatrix) generated with llama-imatrix over a diverse,

code-weighted local corpus, used identically for every quant in this repo — only the

bit-allocation differs between tiers, isolating that as the sole variable in the size/PPL

comparison above.

Files

• laguna-xs.imatrix — the imatrix used for every quant below.
• Laguna-XS-2.1-APEX-i-quality.gguf (19.18 GiB) — recommended default; best

quality-per-byte of the tiers here.

• Laguna-XS-2.1-APEX-dynamic.gguf (11.25 GiB) — role-aware allocation targeting a

16GB-card budget.

• Laguna-XS-2.1-IQ3_XXS.gguf (12.06 GiB) — stock imatrix-guided baseline, included for

the size/PPL comparison above (not the recommended pick — i-quality or APEX-dynamic

both do better per byte).

Attribution

• Base model: Poolside — poolside/Laguna-XS-2.1.
• APEX recipe & toolkit: LocalAI — localai-org/apex-quant.
• Quantization engine: llama.cpp (ggml-org).

Unofficial community quantization; not affiliated with or endorsed by Poolside.