OLMo-150M and OLMo-1B Pretrained Models
Collection
Pretrained models from scratch used in "Echo Chamber: RL Post-training Amplifies Behaviors Learned in Pretraining". • 12 items • Updated • 4
OLMo-150M-4xtg
150M OLMo model pretrained on 4 passes of the TinyGSM dataset.
Reading Model Names
Model names contain the contents of the pretraining dataset, delimited by underscores.
- as: Algebraic Stack
- fm3: FineMath3+
- tg: TinyGSM
- omi1: OpenMathInstruct1
- omi2: OpenMathInstruct2
If there is a {n}x in front of the dataset abbreviation, that means it was repeated n times during pretraining. For instance, 2xtg refers to two passes over the TinyGSM dataset.
- Downloads last month
- 7