---
title: README
emoji: 💤
colorFrom: indigo
colorTo: blue
sdk: static
pinned: true
---

We build **stuff** and **share it** with the community.

That's pretty much it. And yes, "stuff" was the right word to use.

### Why "**FromZero**"?

**FromZero** is inspired by **Re:Zero - Starting Life in Another World**. Just like the main character, all of our models begin from zero.

Thankfully, they don't spend their time dying repeatedly or having mental breakdowns.

I mean, training is already painful enough. 

### Our goal

Well, there isn't actually a definitive goal we're chasing. I mean, yeah, we want to be the **greatest**. But so does everyone else.

We're just here to build ***weird*** models, share them with the community, and grow side-by-side with everyone.

### Current Models

- **Syn-2.6M**: A 2.6M-parameter Tiny Language Model (TLM) trained only on synthetic data (tiny-textbooks, tiny-stories-v2, longpage, and finephrase).
- **Er-Medium**: A 12.4M-parameter model trained on 34B tokens. It achieves state-of-the-art performance on many benchmarks.
  
### Future Releases

- **Orez** - Tiny (1.3M), Small (5M), Medium (12.4M), Large (30M): A family of Small Language Models ranging from 1.2M to 30M parameters. Each of them will be massively overtrained on a minium of 1000+ tokens per parameter.

### Discord

Join our Discord community: [FromZero](https://discord.gg/2W6kampan).

### Website

Test all of our models free of charge on our website: **[AskJASP](https://askjasp.com)**