---

base_model: poolside/Laguna-S-2.1

library_name: gguf

license: openmdw-1.1

pipeline_tag: text-generation

tags:

• gguf
• quantized
• llama.cpp
• laguna-s-2.1
• iq4_xs
• q4_k_m
• code
• moe

---

<p align="center">

<img alt="poolside-banner" src="https://poolside.ai/assets/laguna/laguna-s-2-1-banner.svg" width="800px">

</p>

Laguna S 2.1 - GGUF Quants (IQ4_XS & Q4_K_M)

This repository contains GGUF quantizations for poolside/Laguna-S-2.1, including both IQ4_XS and Q4_K_M variants.

• Original Model: poolside/Laguna-S-2.1
• Quantization Formats: GGUF (IQ4_XS, Q4_K_M)
• Model Architecture: 118B MoE (~8B activated parameters per token)

---

Quantization Details

| File Name | Quant Method | Description |

|---|---|---|

| laguna-s-2.1-IQ4_XS.gguf | IQ4_XS | 4-bit importance matrix quantization (extra small). Highly optimized for low memory usage with minimal quality loss. |

| laguna-s-2.1-Q4_K_M.gguf | Q4_K_M | Standard 4-bit K-quantization (medium). Balanced performance, speed, and accuracy. |

---

Usage Guide

1. Running with llama.cpp

Use poolside's llama.cpp fork on the laguna branch for native support:

git clone --branch laguna [https://github.com/poolsideai/llama.cpp](https://github.com/poolsideai/llama.cpp)
cd llama.cpp && cmake -B build && cmake --build build -j

# Download your chosen model file from this repository
# Option A: IQ4_XS
huggingface-cli download Abiray/Laguna-S-2.1-IQ4_XS-Q4_K_M-GGUF laguna-s-2.1-IQ4_XS.gguf --local-dir .

# Option B: Q4_K_M
huggingface-cli download Abiray/Laguna-S-2.1-IQ4_XS-Q4_K_M-GGUF laguna-s-2.1-Q4_K_M.gguf --local-dir .

# Serve with llama-server
./build/bin/llama-server -m Laguna-S-2.1-IQ4_XS.gguf --jinja --port 8000