based
Collection
These language model checkpoints are trained at the 360M and 1.3Bn parameter scales for up to 50Bn tokens on the Pile corpus, for research purposes. • 14 items • Updated • 9
Configuration Parsing Warning:In config.json: "architectures" must be an array
Based model but uses layernorm instead of QK.sum(-1) for the normalization, for better hardware efficiency.
- Downloads last month
- 1
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support