README.md

---
language:
- en
- ko
license: other
license_name: solar-apache-2.0
tags:
- upstage
- solar
- moe
- llm
- pruning
- reap
base_model:
- upstage/Solar-Open-100B
---

<p align="center">
  <img src="./Solar-Open-69B-REAP.png" alt="Solar Open 69B REAP" width="100%">
</p>

> [!NOTE]
> Due to the specific pruning focus, this current version is **not optimized for math**.



# Solar Open to 69B REAP

This repository contains a pruned version of Upstage's flagship Solar-Open-100B. Using **REAP (Router Expert Activation Pruning)**, the model has been compressed from its original size to a highly efficient **69B parameter** model.

## Model Highlights

* **Pruning Method:** REAP (Router Expert Activation Pruning) via a modified [Cerebras Research](https://github.com/CerebrasResearch/reap) implementation.
* **Optimization:** Pruned on **4x NVIDIA A100 SXM** using ~100 samples from the `nickrosh/Evol-Instruct-Code-80k-v1` dataset.
* **Chat Template Fix:** Updated on **Jan 7th, 2026 (~19:00)** to resolve infinite reasoning loops and "non-stop yapping" issues.

## Recommended Quants

For GGUF usage, we recommend these versions from the **mradermacher** team:

- [Solar-Open-69B-REAP-i1-GGUF (imatrix)](https://huggingface.co/mradermacher/Solar-Open-69B-REAP-i1-GGUF)
- [Solar-Open-69B-REAP-GGUF (Standard)](https://huggingface.co/mradermacher/Solar-Open-69B-REAP-GGUF)

---

## Technical Details & Acknowledgments

The goal of this REAP was to reduce the overhead of the massive 102B MoE architecture while maintaining elite performance in instruction following and coding.

### Chat Template Fix
The custom chat template implemented on **Jan 7th** "dumbs down" the reasoning length. This ensures the model stays on task and provides concise responses rather than getting lost in long-winded internal monologues.

**Special Thanks:** Shoutout to **[Barney Greenway](https://huggingface.co/McG-221)** for identifying the nonstop reasoning behavior and providing the feedback needed to fix the template.

### Future Roadmap
Any future REAP uploads to this profile will include specialized experts for:
* Advanced Mathematics
* Function-calling
* SWE-environment (Software Engineering)


---

## Usage with llama-cpp-python

To ensure the **Jan 7th template fix** is applied (overriding the older metadata inside the GGUF), we recommend downloading the template directly from this repository's `tokenizer_config.json` using the code below:

```python
from llama_cpp import Llama
from transformers import AutoTokenizer

# 1. Pull the latest fixed template from this HF Repo
repo_id = "Akicou/Solar-Open-69B-REAP" 
tokenizer = AutoTokenizer.from_pretrained(repo_id)
fixed_template = tokenizer.chat_template

# 2. Initialize the Llama model
llm = Llama(
    model_path="./Solar-Open-69B-REAP.Q4_K_M.gguf",
    n_ctx=4096,
    n_gpu_layers=-1 # Use -1 to offload all layers to GPU
)

# 3. Create completion with the injected template
messages = [
    {"role": "system", "content": "You are a concise and helpful assistant."},
    {"role": "user", "content": "Explain the benefits of MoE pruning."}
]

response = llm.create_chat_completion(
    messages=messages,
    chat_template=fixed_template, # Force-uses the latest Jan 7th fix
    temp=0.7
)

print(response["choices"][0]["message"]["content"])

```

## License

The model weights are licensed under the **Solar-Apache License 2.0**.