OpenBanana/BananaAI-Developer-1

🍌 BananaAI Developer 1

A 1.3B parameter coding model trained completely from scratch. ?

• No pretrained weights - initialized from random
• No fine-tuning - trained from scratch on raw data
• No APIs - pure open-source training
• Full transparency - all code, data, and weights public

Model Details

• Parameters: 1.3B or less (scaling to 3B)
• Architecture: GPT-NeoX style decoder
• Context Length: 4096 tokens
• Training Data: 30B+ tokens
  • The Stack (code)
  • OpenWebMath
  • Wikipedia
  • Reasoning chains

Training Progress

Status: Preparing for training
Start Date: TBD (awaiting compute allocation)
Training Logs: Coming soon

(I really hope this gets approved and to build my own AI...)