| --- |
| title: Opus Research |
| --- |
| |
| # Opus Research |
|
|
| Independent research on small language models — pretraining from scratch, |
| post-training, and the practical engineering around getting models to run on |
| constrained hardware. |
|
|
| We publish weights, datasets, and methodology, including the parts that did not |
| work. |
|
|
| ## Models |
|
|
| **[opus-1.5](https://huggingface.co/opus-research/opus-1.5)** — 0.88B parameter |
| language model pretrained from scratch on 2× RTX 4090. Trained in 42 hours; |
| energy use documented in the model card. |
|
|
| **[opus-2.0](https://huggingface.co/opus-research/opus-2.0)** — successor model, |
| targeting a larger parameter count. **Training is currently paused: we do not |
| have the compute budget to continue.** The architecture and data pipeline are |
| ready; the run is blocked on GPU access rather than on research. |
|
|
| **[gemma-2-2b-thinking](https://huggingface.co/opus-research/gemma-2-2b-thinking)** |
| — experiment in adding explicit chain-of-thought to a base model that lacks it. |
| LoRA on Gemma 2 2B, trained on our own reasoning dataset. Training loss 3.56 → |
| 0.69 over 566 steps. Exploratory; not benchmarked. |
|
|
| **[bernard-gpt-oss-20b-lora](https://huggingface.co/opus-research/bernard-gpt-oss-20b-lora)** |
| — style-transfer LoRA for gpt-oss-20b, a 21B mixture-of-experts model. Notable |
| mainly for the training notes: a four-run ablation showing that the checkpoint |
| with the *worst* validation loss was the correct one to ship, plus a set of |
| gpt-oss-specific engineering gotchas we could not find documented elsewhere. |
|
|
| ## Datasets |
|
|
| **[opus-thinking-10k](https://huggingface.co/datasets/opus-research/opus-thinking-10k)** |
| — ~9,000 chain-of-thought examples, used to train `gemma-2-2b-thinking`. |
|
|
| **[opus-thinking](https://huggingface.co/datasets/opus-research/opus-thinking)** |
| — earlier iteration of the same dataset. |
|
|
| ## What we work on |
|
|
| - Pretraining small models end-to-end on consumer hardware |
| - Post-training: LoRA, style transfer, capability injection |
| - Dataset construction and filtering methodology |
| - Quantization and CPU inference for models that would otherwise need a |
| datacenter |
|
|
| Our constraint is compute, not ideas. Everything above was produced on consumer |
| GPUs, and the ceiling that imposes is the main thing limiting what we publish |
| next. |
|
|
| ## Contact |
|
|
| Open to collaboration, compute partnerships, and grant programs. Reach us |
| through the organization page. |
|
|
