| --- |
| title: README |
| emoji: 💤 |
| colorFrom: indigo |
| colorTo: blue |
| sdk: static |
| pinned: true |
| --- |
| |
| We build **stuff** and **share it** with the community. |
|
|
| That's pretty much it. And yes, "stuff" was the right word to use. |
|
|
| ### Why "**FromZero**"? |
|
|
| **FromZero** is inspired by **Re:Zero - Starting Life in Another World**. Just like the main character, all of our models begin from zero. |
|
|
| Thankfully, they don't spend their time dying repeatedly or having mental breakdowns. |
|
|
| I mean, training is already painful enough. |
|
|
| ### Our goal |
|
|
| Well, there isn't actually a definitive goal we're chasing. I mean, yeah, we want to be the **greatest**. But so does everyone else. |
|
|
| We're just here to build ***weird*** models, share them with the community, and grow side-by-side with everyone. |
|
|
| ### Current Models |
|
|
| - **Syn-2.6M**: A 2.6M-parameter Tiny Language Model (TLM) trained only on synthetic data (tiny-textbooks, tiny-stories-v2, longpage, and finephrase). |
| - **Er-Medium**: A 12.4M-parameter model trained on 34B tokens. It achieves state-of-the-art performance on many benchmarks. |
| |
| ### Future Releases |
|
|
| - **Orez** - Tiny (1.3M), Small (5M), Medium (12.4M), Large (30M): A family of Small Language Models ranging from 1.2M to 30M parameters. Each of them will be massively overtrained on a minium of 1000+ tokens per parameter. |
|
|
| ### Discord |
|
|
| Join our Discord community: [FromZero](https://discord.gg/2W6kampan). |
|
|
| ### Website |
|
|
| Test all of our models free of charge on our website: **[AskJASP](https://askjasp.com)** |
