---
{}
---
# Model Details
- **Architecture**: Basic/default GPT-2, decoder only
- **Num params**: ~810M
- **Num tokens seen**: ~2 B
- **Dataset**: PubMed Abstracts subset of The Pile