---
license: apache-2.0
library_name: transformers
---

# solar-open2-tiny-dummy

**Random-weight** tiny dummy of the [SolarOpen2](https://huggingface.co/upstage/Solar-Open2-250B) architecture,
used for `transformers` CI integration tests. **Not a trained model — outputs are meaningless.**

Structure mirrors the public model at small scale: hybrid attention with the default `gqa_interval=3`-derived
pattern (`[full_attention, linear_attention x3] x3`, 12 layers), NoPE full attention with output sigmoid gate,
factored Kimi-Delta-Attention projections (head_dim 128), and a 16-expert top-4 MoE with one shared expert.
Tokenizer files are copied from `upstage/Solar-Open2-250B`. Weights are randomly initialized with a fixed seed
(`torch.manual_seed(42)`) in bfloat16 (617M parameters).