| --- |
| language: |
| - en |
| - ko |
| tags: |
| - speculative-decoding |
| - solar |
| license: other |
| license_name: solar-open2 |
| license_link: LICENSE |
| --- |
| |
| # Solar-Open2 250B + Speculative Draft (combined, 500K draft) |
|
|
| This repository bundles **Upstage Solar Open2 250B-A15B** (original weights at the |
| repository root, unchanged — see `LICENSE` and the tech report) together with our |
| **block-draft speculative-decoding draft model** in `draft/`. |
|
|
| - **Root**: Solar Open2 250B-A15B — 48 layers (12 NoPE GQA + 36 linear-attention), |
| 320-expert MoE top-8, vocab 196,608. Original Upstage distribution files. |
| - **`draft/`**: 5-layer draft (core 0.90B, total 2.51B with frozen target |
| embedding/LM head), conditioned on five taps of the target's hidden states |
| (layers 3/15/27/39/47). Proposes up to γ=7 tokens per step; the target |
| verifies the block in one forward. |
|
|
| ## Draft results (500K-scale, 2 epochs) |
|
|
| | Metric | Value | |
| |---|---| |
| | val accept_len (γ=7) | 3.431 | |
| | Serving accept_len γ=3 / γ=7 | 2.75 / 3.54 | |
| | gsm8k (5-shot greedy, spec on) | 0.930–0.940 (lossless band) | |
|
|
| Training: 500K-scale multi-domain collection (415K-prompt manifest), |
| **~1.2B tokens** through the target for hidden-state distillation. |
| 2 epochs, chunked KL, 2048 stratified anchors per batch. |
|
|
| ## Usage |
|
|
| Serve the root model as usual for non-speculative inference. For speculative |
| decoding, point your serving stack's block-draft speculative option at the |
| `draft/` folder (block size 4–7 recommended, draft unquantized). Requires a |
| serving build with block-draft speculative decoding support for this draft |
| format. |
|
|
| ## License |
|
|
| Root model weights follow the Upstage Solar Open2 license (`LICENSE`). The |
| draft contains the target's frozen embedding/LM-head weights and follows the |
| same terms. |
|
|
