---

license: mit

base_model: upstage/Solar-Open2-250B

tags:

• gguf
• llama.cpp
• solar-open2
• moe
• quantized

---

Solar-Open2-250B GGUF

First GGUF quantizations of Upstage Solar-Open2-250B.

> ## Requires a llama.cpp build with solar_open2 support

>

> Stock llama.cpp does not yet support this architecture. Build from the fork

> that adds it:

>

> Fork: https://github.com/vcruz305/llama.cpp (branch solar-open2-support)

>

> ```bash

> git clone -b solar-open2-support https://github.com/vcruz305/llama.cpp

> cd llama.cpp

> cmake -B build -DGGML_CUDA=ON && cmake --build build -j

> ```

Architecture

Solar-Open2 is a 48-layer hybrid: 36 Kimi Delta Attention (KDA) linear-attn

layers + 12 gated-GQA (NoPE) layers, with a 320-expert DeepSeek-V3-style MoE

(shared expert + sigmoid router + e_score correction bias). ~250B total,

~15B active per token.

Validation

• Logit parity: 8/8 greedy top-1 match vs Upstage transformers reference.
• Coherence: fluent generation on the full 250B model.
• Perplexity (Q2_K, wikitext-2, c=512, 80 chunks): PPL = 5.9322 +/- 0.10022.

Running on a single DGX Spark (GB10, 128GB unified)

Use --no-mmap for the large quants - default mmap double-allocates host+device

and can exceed unified memory:

llama-cli -m Solar-Open2-250B-Q2_K.gguf --no-mmap -ngl 999 \
  -p "The capital of France is"

Larger quants (Q4_K_M and up) exceed a single 128GB box; run them across two

GB10s with llama.cpp RPC (--rpc host:port --no-mmap -ts 0.5,0.5) or on a

larger-memory machine.

Quants

| Rung | Notes |

|------|-------|

| Q8_0 | near-lossless reference |

| Q4_K_M | recommended quality (multi-box / big-RAM) |

| Q2_K | fits a single GB10; PPL 5.93 |

| IQ4_XS / IQ3_XXS / IQ2_M / IQ1_M | imatrix low-bit rungs |

(Rungs upload as they finish quantizing.)

Credit

Base model (c) Upstage. GGUF arch support built on ggml-org/llama.cpp (MIT);

KDA adapted from kimi-linear.cpp, MoE from deepseek-v3, gating from

qwen3next.cpp.