---

base_model: poolside/Laguna-S-2.1

library_name: gguf

license: openmdw-1.1

pipeline_tag: text-generation

tags:

• gguf
• quantized
• llama.cpp
• laguna-s-2.1
• code
• moe

---

<p align="center">

<img alt="poolside-banner" src="https://poolside.ai/assets/laguna/laguna-s-2-1-banner.svg" width="800px">

</p>

Laguna S 2.1 - Q4_K_M GGUF

This repository contains the GGUF quantization (Q4_K_M) for poolside/Laguna-S-2.1.

• Original Model: poolside/Laguna-S-2.1
• Quantization Format: GGUF (Q4_K_M - 4-bit medium quantization, balanced quality and memory footprint)
• Model Architecture: 118B MoE (~8B activated parameters per token)

---

Quantization Details

| File Name | Quant Method | Description |

|---|---|---|

| laguna-s-2.1-Q4_K_M.gguf | Q4_K_M | 4-bit quantization with medium weight layout. Recommended balance of speed, performance, and VRAM usage. |

---

Usage Guide

1. Running with llama.cpp

Use poolside's llama.cpp fork on the laguna branch for full support:

git clone --branch laguna [https://github.com/poolsideai/llama.cpp](https://github.com/poolsideai/llama.cpp)
cd llama.cpp && cmake -B build && cmake --build build -j

# Download the model file from this repository
huggingface-cli download Abiray/Laguna-S-2.1-Q4_K_M-GGUF laguna-s-2.1-Q4_K_M.gguf --local-dir .

# Serve with llama-server
./build/bin/llama-server -m laguna-s-2.1-Q4_K_M.gguf --jinja --port 8000