metadata
title: README
emoji: 💤
colorFrom: indigo
colorTo: blue
sdk: static
pinned: true
We build stuff and share it with the community.
That's pretty much it. And yes, "stuff" was the right word to use.
Why "FromZero"?
FromZero is inspired by Re:Zero - Starting Life in Another World. Just like the main character, all of our models begin from zero.
Thankfully, they don't spend their time dying repeatedly or having mental breakdowns.
I mean, training is already painful enough.
Our goal
Well, there isn't actually a definitive goal we're chasing. I mean, yeah, we want to be the greatest. But so does everyone else.
We're just here to build weird models, share them with the community, and grow side-by-side with everyone.
Current Models
- Syn-2.6M: A 2.6M-parameter Tiny Language Model (TLM) trained only on synthetic data (tiny-textbooks, tiny-stories-v2, longpage, and finephrase).
- Er-Medium: A 12.4M-parameter model trained on 34B tokens. It achieves state-of-the-art performance on many benchmarks.
Future Releases
- Orez - Tiny (1.3M), Small (5M), Medium (12.4M), Large (30M): A family of Small Language Models ranging from 1.2M to 30M parameters. Each of them will be massively overtrained on a minium of 1000+ tokens per parameter.
Discord
Join our Discord community: FromZero.
Website
Test all of our models free of charge on our website: AskJASP