---

license: other

license_name: upstage-solar-license

license_link: LICENSE

base_model: upstage/Solar-Open2-250B

pipeline_tag: text-generation

tags:

- gguf

- llama.cpp

- solar

- solar-open2

- moe

- quantized

---

Solar-Open2-250B-GGUF

> Built with Solar

Unofficial community GGUF quantizations of

upstage/Solar-Open2-250B.

Not affiliated with, sponsored by, or endorsed by Upstage. This is an

independent derivative work.

Available quantizations

• Q4_K_M (~141 GB) — Solar-Open2-250B-Q4_K_M--of-.gguf. The 128-wide

KDA gate projections (ssm_f_b / ssm_g_b) fall back q4_K→q5_0, as is

normal for Kimi-family linear-attention models.

• Q8_0 (~250 GB) — Solar-Open2-250B-Q8_0--of-.gguf. Near-lossless; no

fallbacks (Q8_0's 32-wide blocks quantize the 128-wide gates cleanly).

Both are split into <50 GB shards; point llama.cpp at the first shard of a

set and it loads the rest automatically.

Modification notice: these files were modified from the original —

converted to GGUF and quantized from the original bf16 weights.

⚠️ Requires a patched llama.cpp

Solar-Open2 is a hybrid MoE (KDA linear attention + gated GQA + DeepSeek-V3-style

MoE) and its architecture is not in upstream llama.cpp. To run these GGUFs you

must build llama.cpp from the fork that adds the solar-open2 architecture:

> https://github.com/llamaraspberryrabbit/llama.cpp/tree/add-solar-open2

⚠️ This fork is AI-created and will not be upstreamed

That fork — the HF→GGUF converter, the C++ inference graph, the reasoning-parser

patch, and these quantizations — was created by AI (Claude Opus 4.8). It has

not been numerically validated against Upstage's reference implementation, and

no pull request will be opened against upstream llama.cpp. Treat it as an

experimental, community-grade artifact, and review the code yourself before

relying on it — you are responsible for anything you run.

It has been verified to load and generate coherent, factually correct output

on a CUDA build, and reasoning is parsed into reasoning_content — but see the

caveat above.

Usage

# reasoning model — a repeat penalty is recommended
llama-cli -m Solar-Open2-250B-Q4_K_M-00001-of-000NN.gguf \
    -ngl 99 -c 8192 --jinja --reasoning-format deepseek \
    --repeat-penalty 1.2 -cnv

llama.cpp loads the shard set automatically when you point it at the first shard.

License

Built with Solar. Licensed under the Upstage Solar License (see the

LICENSE file in this repo, included per Section 4(e)). The original

Solar-Open2 model is © Upstage Co., Ltd. Per the license, this derivative's name

begins with "Solar" and displays "Built with Solar." Use of the "Solar" name does

not imply any endorsement by or official relationship with Upstage.