We're excited to release BananaMind 2 Mini the first model in our BananaMind 2 series!
BananaMind 2 Mini features a custom digit-aware BPE tokenizer that keeps every digit isolated, fixing the core arithmetic weakness of our previous models. It's trained on 30B tokens from FineWeb-Edu, DCLM, Cosmopedia-v2 and FineMath-4+, and already outperforms Pythia-31M despite having fewer parameters. Check it out at BananaMind/BananaMind-2-Mini