Post
107
# 🔥 Nova-1 Beta: Test Our New LLMs!
**Smilyai Labs** is building **Nova-1** — open-source LLMs with novel architectures. Join our beta program!
## 🎯 Available Now:
**Nova-1-Standard (1.2B)** — Phase 2 of pretraining in progress
- PPL 13.5 (beats GPT-2 Large!)
- 48K tok/s on consumer GPUs
- Great for code, reasoning, edge deployment
**Nova-1-Large (3.5B)** — Training live RIGHT NOW
- Current: 30.9 PPL, improving fast, loss at 3.5 right now
- Will finish with ~1.7B tokens today
- Better reasoning & longer context
**Nova-1-XL (10B MoE)** — Coming soon (We dont know yet! haha)
- Final Specs not decided yet
## What Makes Nova Special?
✨ **Mixture of Depths (MoD)** — Routes tokens dynamically, 30% faster
✨ **Grouped Query Attention** — Efficient like LLaMA 2/3
✨ **Phased Training** — Fresh 1B tokens each phase (no overfitting!)
✨ **RoPE** — Context extendable to 8K+
## 🤝 Join Beta Testing:
👉 **[Smilyai-labs-beta-testers](
Smilyai-labs-beta-testers
Get early access, shape the roadmap, and help build transparent open-source AI!
**Smilyai Labs** is building **Nova-1** — open-source LLMs with novel architectures. Join our beta program!
## 🎯 Available Now:
**Nova-1-Standard (1.2B)** — Phase 2 of pretraining in progress
- PPL 13.5 (beats GPT-2 Large!)
- 48K tok/s on consumer GPUs
- Great for code, reasoning, edge deployment
**Nova-1-Large (3.5B)** — Training live RIGHT NOW
- Current: 30.9 PPL, improving fast, loss at 3.5 right now
- Will finish with ~1.7B tokens today
- Better reasoning & longer context
**Nova-1-XL (10B MoE)** — Coming soon (We dont know yet! haha)
- Final Specs not decided yet
## What Makes Nova Special?
✨ **Mixture of Depths (MoD)** — Routes tokens dynamically, 30% faster
✨ **Grouped Query Attention** — Efficient like LLaMA 2/3
✨ **Phased Training** — Fresh 1B tokens each phase (no overfitting!)
✨ **RoPE** — Context extendable to 8K+
## 🤝 Join Beta Testing:
👉 **[Smilyai-labs-beta-testers](
Get early access, shape the roadmap, and help build transparent open-source AI!