EMO: Pretraining mixture of experts for emergent modularity
• 37
AI & ML interests
Building breatkthrough AI to solve the world's biggest problems.
Recent Activity
View all activity
Papers
EMO: Pretraining Mixture of Experts for Emergent Modularity
MolmoAct2: Action Reasoning Models for Real-world Deployment
Articles
allenai/DataDecide-falcon-and-cc-qc-tulu-10p-300M
0.4B • Updated • 8
allenai/DataDecide-falcon-and-cc-300M
0.4B • Updated • 14
allenai/DataDecide-falcon-and-cc-qc-20p-300M
0.4B • Updated • 11
allenai/DataDecide-falcon-and-cc-qc-orig-10p-530M
0.6B • Updated • 7
allenai/DataDecide-dclm-baseline-qc-7p-fw2-300M
0.4B • Updated • 11
allenai/DataDecide-dolma1_7-no-flan-300M
0.4B • Updated • 65
allenai/DataDecide-dolma1_7-no-math-code-300M
0.4B • Updated • 62
allenai/DataDecide-dclm-baseline-50p-dolma1.7-50p-300M
0.4B • Updated • 9
allenai/DataDecide-falcon-and-cc-qc-orig-10p-300M
0.4B • Updated • 11
allenai/DataDecide-dolma1_6plus-530M
0.6B • Updated • 12
allenai/DataDecide-dclm-baseline-25p-dolma1.7-75p-530M
0.6B • Updated • 8
allenai/DataDecide-falcon-530M
0.6B • Updated • 10
allenai/DataDecide-dolma1_7-no-code-530M
0.6B • Updated • 10
allenai/DataDecide-dclm-baseline-qc-fw-3p-530M
0.6B • Updated • 8
allenai/DataDecide-dclm-baseline-qc-7p-fw2-530M
0.6B • Updated • 12
allenai/DataDecide-dolma1_7-no-math-code-530M
0.6B • Updated • 10
allenai/DataDecide-dclm-baseline-530M
0.6B • Updated • 9
allenai/DataDecide-c4-530M
0.6B • Updated • 35
allenai/DataDecide-dolma1_7-530M
0.6B • Updated • 53
allenai/DataDecide-dclm-baseline-75p-dolma1.7-25p-530M
0.6B • Updated • 10
allenai/DataDecide-dclm-baseline-50p-dolma1.7-50p-530M
0.6B • Updated • 9
allenai/DataDecide-dolma1_7-no-flan-530M
0.6B • Updated • 10
allenai/DataDecide-falcon-and-cc-qc-20p-530M
0.6B • Updated • 9
allenai/DataDecide-falcon-and-cc-530M
0.6B • Updated • 9
allenai/DataDecide-falcon-and-cc-qc-tulu-10p-530M
0.6B • Updated • 14
allenai/DataDecide-falcon-and-cc-qc-10p-530M
0.6B • Updated • 9
allenai/DataDecide-dclm-baseline-qc-7p-fw3-530M
0.6B • Updated • 14
allenai/DataDecide-dolma1_7-no-reddit-530M
0.6B • Updated • 7
allenai/DataDecide-dclm-baseline-qc-20p-530M
0.6B • Updated • 10
allenai/DataDecide-dclm-baseline-qc-10p-530M
0.6B • Updated • 11