Raising Bars, Not Parameters: LilMoo Compact Language Model for Hindi
Paper
• 2603.03508 • Published
• 3
A 0.6-billion-parameter Hindi language models trained entirely from scratch.
Note 🧱 Base model pretrained only with Hindi text.
Note 🧱 Base model pretrained with a Hindi+ English mixture.
Note 📚 Pretraining dataset.
Note 📚 Annotations to train classifiers/filters (Educational).
Note 📚 Annotations to train classifiers/filters (Toxicity).
Note 🎯 Quality Filter (Educational)
Note 🎯 Quality Filter (Toxicity)
Note 📚 Data used to train the LilMoo tokenizer.