Submission: Museko-125M (123M, from scratch)
#57
by TobiasLogic - opened
Hi, I would like to submit Museko-125M for the leaderboard.
Model: https://huggingface.co/TobiasLogic/Museko-125M
Params: about 123M (sub-150M)
Weights: open, Apache 2.0
Architecture: Llama style decoder, trained from scratch on FineWeb-Edu with a small synthetic arithmetic mix.
Results (0-shot, lm-evaluation-harness plus the ArithMark 2.0 script):
| Benchmark | Museko-125M | Previous best in class |
|---|---|---|
| HellaSwag | 33.2 | 30.6 |
| ARC-Easy | 54.0 | 48.6 |
| ARC-Challenge | 25.7 | 25.4 |
| PIQA | 63.2 | 64.5 |
| ArithMark 2.0 | 74.3 | 66.7 |
By the leaderboard averaging (ARC-Easy and ARC-Challenge combined) this comes to about 52.6, ahead of the previous top model at 49.71. It leads on four of the five benchmarks.
Happy to have it independently verified. Thanks for running this leaderboard.
Sry for the delay but merged!
Datdanboi25 changed discussion status to closed