README.md

---
license: other
license_name: upstage-solar-license
license_link: LICENSE
base_model: upstage/Solar-Open2-250B
pipeline_tag: text-generation
tags:
  - gguf
  - llama.cpp
  - solar
  - solar-open2
  - moe
  - quantized
---

# Solar-Open2-250B-GGUF

> **Built with Solar**

Unofficial community **GGUF** quantizations of
[upstage/Solar-Open2-250B](https://huggingface.co/upstage/Solar-Open2-250B).
**Not affiliated with, sponsored by, or endorsed by Upstage.** This is an
independent derivative work.

## Available quantizations

- **`Q4_K_M`** (~141 GB) — `Solar-Open2-250B-Q4_K_M-*-of-*.gguf`. The 128-wide
  KDA gate projections (`ssm_f_b` / `ssm_g_b`) fall back `q4_K`→`q5_0`, as is
  normal for Kimi-family linear-attention models.
- **`Q8_0`** (~250 GB) — `Solar-Open2-250B-Q8_0-*-of-*.gguf`. Near-lossless; no
  fallbacks (Q8_0's 32-wide blocks quantize the 128-wide gates cleanly).

Both are split into <50 GB shards; point llama.cpp at the **first** shard of a
set and it loads the rest automatically.

**Modification notice:** these files were **modified** from the original —
converted to GGUF and quantized from the original bf16 weights.

## ⚠️ Requires a patched llama.cpp

Solar-Open2 is a hybrid MoE (KDA linear attention + gated GQA + DeepSeek-V3-style
MoE) and its architecture is **not in upstream llama.cpp**. To run these GGUFs you
must build llama.cpp from the fork that adds the `solar-open2` architecture:

> **https://github.com/llamaraspberryrabbit/llama.cpp/tree/add-solar-open2**

### ⚠️ This fork is AI-created and will not be upstreamed

That fork — the HF→GGUF converter, the C++ inference graph, the reasoning-parser
patch, and these quantizations — was **created by AI** (Claude Opus 4.8). It has
**not** been numerically validated against Upstage's reference implementation, and
**no pull request will be opened** against upstream llama.cpp. Treat it as an
experimental, community-grade artifact, and **review the code yourself** before
relying on it — you are responsible for anything you run.

It has been verified to **load and generate coherent, factually correct output**
on a CUDA build, and reasoning is parsed into `reasoning_content` — but see the
caveat above.

## Usage

```bash
# reasoning model — a repeat penalty is recommended
llama-cli -m Solar-Open2-250B-Q4_K_M-00001-of-000NN.gguf \
    -ngl 99 -c 8192 --jinja --reasoning-format deepseek \
    --repeat-penalty 1.2 -cnv
```

llama.cpp loads the shard set automatically when you point it at the first shard.

## License

**Built with Solar.** Licensed under the **Upstage Solar License** (see the
[`LICENSE`](LICENSE) file in this repo, included per Section 4(e)). The original
Solar-Open2 model is © Upstage Co., Ltd. Per the license, this derivative's name
begins with "Solar" and displays "Built with Solar." Use of the "Solar" name does
not imply any endorsement by or official relationship with Upstage.