---

license: openmdw-1.1

base_model: poolside/Laguna-S-2.1-NVFP4

base_model_relation: quantized

library_name: llama.cpp

pipeline_tag: text-generation

language:

- en

tags:

- gguf

- laguna

- laguna-s-2.1

- nvfp4

- blackwell

- rtx-5090

- llama.cpp

- moe

- mixture-of-experts

- tool-use

- long-context

---

Laguna S 2.1 NVFP4 GGUF — 256K-capable on one RTX 5090

!Laguna S 2.1 NVFP4 on RTX 5090

117.6B total parameters, about 8.5B active per token, and up to a 262,144-token context on a 32 GB RTX 5090 with 64 GB system RAM.

This is Poolside's native NVFP4 Laguna S 2.1 checkpoint packaged as one

67.03 GiB GGUF for llama.cpp. Automatic tensor-aware CPU/GPU placement

keeps the dense execution path and selected tensors on the GPU while

offloading expert weights to RAM. It does not use DFlash or speculative

decoding.

The included runtime correction connects Laguna's native NVFP4 scale

tensors to every relevant graph path. The result is a practical long-context

deployment for Blackwell rather than a smaller, separately requantized

checkpoint.

What has been validated

| Configured context | Validation | Uncached prefill | Generation | Peak VRAM | Host RAM available | Status |

|---:|---|---:|---:|---:|---:|---|

| 65,536 | 65,269 actual tokens; exact needle retrieval | 439.82 tok/s | 25.69 tok/s at full depth; 27.81 tok/s short-depth | 31,092 MiB | about 9.9 GiB | Fast lane |

| 131,072 | 130,800 actual tokens; exact needle retrieval | 417.16 tok/s | 22.51 tok/s at full depth; 23.34 tok/s short-depth | 31,056 MiB | 8.2 GiB occupied | Recommended |

| 262,144 | Exact 262,144-token slot allocation and coherent generation | — | 18.68 tok/s short-depth | 31,016 MiB | 5.2 GiB idle | Maximum-context lane |

The 64K and 128K rows are fully occupied, uncached semantic tests—not empty

KV-cache allocations. The 256K profile admits the complete 262,144-token

slot and generates coherently; a completely occupied 256K semantic pass has

not yet been claimed here.

Read the speed numbers correctly

1,201.93 tok/s is the fastest measured prompt-processing/prefill

result, obtained on the full 65,269-token prompt with the separately tuned

batch 1024 / ubatch 1024 64K speed profile. It is not output-token speed.

Measured output generation is 18.68–27.81 tok/s, depending on configured

context and actual occupied depth. Real Hermes tool-agent generation measured

27.63 tok/s.

The larger ubatch is a 64K prefill optimization, not the defining feature

of this release. The balanced batch 512 / ubatch 256 profile was used for

the 64K → 128K → 256K context ladder and is the recommended starting point

for long-context deployment.

Quality results

| Evaluation | Published result | Sampling |

|---|---:|---|

| HermesAgent-20, corrected verifier semantics | 90/100 | temperature 0, thinking off |

| Tool-Eval, corrected score | 86/100 | temperature 1, thinking off |

| BigCodeBench Hard/Instruct | 28.38% | temperature 1, top_p=1, top_k=20, min_p=0, seed 42, thinking off |

The corrected Hermes score includes the advertised ripgrep dependency,

adequate turn headroom, and hardened parsing for valid tool calls. The

Tool-Eval and BigCodeBench runs use the recommended temperature-1 sampler.

No legacy raw harness score is reported.

Model file

| File | Size | SHA-256 |

|---|---:|---|

| Laguna-S-2.1-NVFP4.gguf | 71,977,030,080 bytes / 67.03 GiB | 5cf866a0b1531c62a6754e811b64b8bd867b9f5cd5d6a69b2cde04077d807e87 |

The routed expert banks account for about 59.485 GiB across 47 sparse

layers. The remaining approximately 7.48 GiB contains attention, router,

dense-leading, shared-expert, output, and other tensors—mostly BF16 where

the source checkpoint excludes them from NVFP4. This is why the complete

native checkpoint is about 67 GiB.

Memory placement

--fit on adjusts the tensor split against actual free VRAM. As the context

grows, llama.cpp reserves more GPU space for KV cache and moves more model

buffers into system RAM.

| Context | Model buffers on GPU | Host-backed model buffers | Q8 global KV + SWA KV | Measured total GPU used |

|---:|---:|---:|---:|---:|

| 64K | about 27.4 GiB | about 41.2 GiB | 1.632 GiB + 57 MiB | 31,092 MiB |

| 128K | about 25.8 GiB | about 42.8 GiB | 3.264 GiB + 57 MiB | 31,056 MiB |

| 256K | about 22.5 GiB | about 46.1 GiB | 6.528 GiB + 57 MiB | 31,016 MiB |

The buffer figures are loader-accounting approximations rather than

additive partitions of the GGUF file. CUDA workspaces and loader bookkeeping

are additional. The whole-device VRAM readings include a roughly 770 MiB

desktop baseline.

Hardware target

The validated target is:

• NVIDIA RTX 5090 / Blackwell GPU with 32 GB VRAM
• 64 GB system RAM
• current NVIDIA driver
• CUDA 12.8 or newer; CUDA 13 recommended
• Linux

The build and commands are not specific to NixOS or to the test machine's

CPU. They contain no architecture-specific CPU compiler flags or topology

masks. The example thread counts are portable starting values; tune them

for your CPU and memory subsystem. Faster host memory materially helps the

RAM-resident expert path.

Native NVFP4 acceleration is Blackwell-specific. Do not assume the same

speed or kernel behavior on pre-Blackwell CUDA, Vulkan, Metal, or CPU-only

backends.

Runtime: small fork required

At the pinned upstream build, Laguna did not pass all native NVFP4 scale

tensors into its attention gate, routed experts, shared experts, dense FFN,

and output projection. This release therefore requires a focused source

correction.

The ready-to-build runtime is:

• ciru-ai/ciruinference
• tested tag: laguna-nvfp4-b10106

It is a pinned llama.cpp source tree with the Laguna scale-wiring correction

committed normally. This repository also includes

patches/0001-laguna-wire-nvfp4-scale-tensors.patch and

scripts/build-linux-cuda.sh as an auditable apply-on-top path. Once

equivalent wiring lands upstream, the fork will no longer be necessary.

Build on mainstream Linux

Ubuntu / Debian

sudo apt-get update
sudo apt-get install -y build-essential cmake ninja-build git curl libcurl4-openssl-dev

Fedora

sudo dnf install -y gcc-c++ cmake ninja-build git curl libcurl-devel

Install the NVIDIA driver and CUDA toolkit from NVIDIA's repository, then:

nvidia-smi
nvcc --version

git clone https://github.com/ciru-ai/ciruinference.git
cd ciruinference
git checkout laguna-nvfp4-b10106

cmake -S . -B build-cuda \
  -DGGML_CUDA=ON \
  -DLLAMA_CURL=ON \
  -DCMAKE_BUILD_TYPE=Release
cmake --build build-cuda --config Release -j "$(nproc)" \
  --target llama-server llama-cli

The build auto-detects the local CUDA target. For a Blackwell cross-build,

add -DCMAKE_CUDA_ARCHITECTURES=120 to the CMake configuration.

Download

Install the Hugging Face CLI, authenticate if required, and download the

single GGUF:

python3 -m pip install -U "huggingface_hub[cli]"

hf download jcbtc/Laguna-S-2.1-NVFP4-GGUF \
  Laguna-S-2.1-NVFP4.gguf \
  --local-dir .

Recommended 128K deployment

This is the best validated balance of real occupied context and generation

speed:

./build-cuda/bin/llama-server \
  --model ./Laguna-S-2.1-NVFP4.gguf \
  --ctx-size 131072 \
  --parallel 1 \
  --batch-size 512 \
  --ubatch-size 256 \
  --threads 8 \
  --threads-batch 16 \
  --flash-attn on \
  --cache-type-k q8_0 \
  --cache-type-v q8_0 \
  --fit on \
  --fit-target 1024 \
  --fit-ctx 131072 \
  --load-mode none \
  --jinja \
  --spec-type none \
  --no-context-shift \
  --cache-ram 0 \
  --no-cache-idle-slots \
  --metrics \
  --slots \
  --no-webui \
  --host 0.0.0.0 \
  --port 8080 \
  --alias laguna-s21-nvfp4-128k

The equivalent portable launcher is included as scripts/serve-128k.sh.

Maximum 256K deployment

The same deployment scales to the model's full 262,144-token context:

./build-cuda/bin/llama-server \
  --model ./Laguna-S-2.1-NVFP4.gguf \
  --ctx-size 262144 \
  --parallel 1 \
  --batch-size 512 \
  --ubatch-size 256 \
  --threads 8 \
  --threads-batch 16 \
  --flash-attn on \
  --cache-type-k q8_0 \
  --cache-type-v q8_0 \
  --fit on \
  --fit-target 1024 \
  --fit-ctx 262144 \
  --load-mode none \
  --jinja \
  --spec-type none \
  --no-context-shift \
  --cache-ram 0 \
  --no-cache-idle-slots \
  --metrics \
  --slots \
  --no-webui \
  --host 0.0.0.0 \
  --port 8080 \
  --alias laguna-s21-nvfp4-256k

Close other GPU-heavy applications before launching. On the validated

machine this profile used 31,016 MiB total VRAM and left 5.2 GiB of system

RAM available at idle.

The equivalent portable launcher is included as scripts/serve-256k.sh.

The OpenAI-compatible endpoint is:

http://127.0.0.1:8080/v1

Example request using the recommended tool-evaluation sampler:

curl http://127.0.0.1:8080/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "laguna-s21-nvfp4-128k",
    "messages": [{"role": "user", "content": "Write a fast parallel file indexer in Rust."}],
    "temperature": 1,
    "top_p": 1,
    "max_tokens": 1024,
    "chat_template_kwargs": {"enable_thinking": false}
  }'

Thinking is supported by the model and was explicitly disabled for the

published agent, tool, and code evaluations.

Optional 64K prefill-speed profile

For workloads dominated by large prompt ingestion rather than maximum

context, change the 128K command to:

--ctx-size 65536
--fit-ctx 65536
--batch-size 1024
--ubatch-size 1024
--alias laguna-s21-nvfp4-64k

That exact profile measured 1,201.93 prefill tok/s on 65,269 uncached prompt

tokens and 23.75 decode tok/s immediately afterward. It is an optional

throughput lane, not the default identity of the model.

Reproducibility

• llama.cpp build 10106
• upstream base commit 1425386fd996511e1f3295e7366c38289a92a271
• included Laguna NVFP4 scale-wiring correction
• RTX 5090, NVIDIA driver 610.43.02
• Q8 K/V cache
• ordinary Flash Attention on
• one sequence
• context shift off
• DFlash off
• speculative decoding off

Credits

• Base and native NVFP4 checkpoint: poolside/Laguna-S-2.1-NVFP4
• Model architecture and training: Poolside
• Runtime: ggml-org/llama.cpp
• Validated patched runtime: ciru-ai/ciruinference
• GGUF packaging, scale-wiring validation, long-context profiling, and evaluations: jcbtc

Laguna S 2.1 is released under OpenMDW-1.1. Review the upstream model card

and license before deployment.