|
|
--- |
|
|
license: apache-2.0 |
|
|
language: |
|
|
- en |
|
|
tags: |
|
|
- Research |
|
|
--- |
|
|
|
|
|
# Model Card for OLMo-2-1B-Exp |
|
|
|
|
|
This model is a research variant of [OLMo-2-0425-1B](https://huggingface.co/allenai/OLMo-2-0425-1B). |
|
|
|
|
|
It was pretrained from scratch on 210B tokens with additional experimental [modifications to the training data](https://huggingface.co/datasets/sbordt/OLMo-2-1B-Exp-Dataset). |
|
|
|
|
|
The baseline model, trained on the same data without any experiments, is [here](https://huggingface.co/sbordt/OLMo-2-1B-Decayed-Early). |
|
|
|
|
|
The model is described in the paper "Train Once, Answer All: Many Pretraining Experiments for the Cost of One". |
|
|
|
|
|
## Usage |
|
|
|
|
|
```python |
|
|
from transformers import AutoModelForCausalLM, AutoTokenizer |
|
|
olmo = AutoModelForCausalLM.from_pretrained("sbordt/OLMo-2-1B-Exp") |
|
|
tokenizer = AutoTokenizer.from_pretrained("sbordt/OLMo-2-1B-Exp") |
|
|
``` |
|
|
|
|
|
### Citation Information |
|
|
|
|
|
``` |
|
|
@article{bordt2025trainonce, |
|
|
title = {Train Once, Answer All: Many Pretraining Experiments for the Cost of One}, |
|
|
author = {Bordt, Sebastian and Pawelczyk, Martin}, |
|
|
journal = {arXiv preprint arXiv:2509.23383}, |
|
|
year = {2025}, |
|
|
} |
|
|
``` |
