PolyPythias
- Preview • Updated • 48
EleutherAI/pile-preshuffled-seeds
Updated • 204 • 1Note Training data information for each seed.
-
EleutherAI/pythia-14m-deduped
Text Generation • 39.2M • Updated • 50.6k • 28 -
EleutherAI/pythia-14m-seed1
Text Generation • Updated • 149 -
EleutherAI/pythia-14m-seed2
Text Generation • Updated • 213 -
EleutherAI/pythia-14m-seed3
Text Generation • Updated • 70 -
EleutherAI/pythia-14m-seed4
Text Generation • Updated • 68 -
EleutherAI/pythia-14m-seed5
Text Generation • Updated • 121 -
EleutherAI/pythia-14m-seed6
Text Generation • Updated • 115 -
EleutherAI/pythia-14m-seed7
Text Generation • Updated • 118 -
EleutherAI/pythia-14m-seed8
Text Generation • Updated • 110 -
EleutherAI/pythia-14m-seed9
Text Generation • Updated • 91 -
EleutherAI/pythia-31m-deduped
Text Generation • 55.7M • Updated • 3.02k • 5 -
EleutherAI/pythia-31m-seed1
Text Generation • Updated • 236 -
EleutherAI/pythia-31m-seed2
Text Generation • Updated • 231 -
EleutherAI/pythia-31m-seed3
Text Generation • Updated • 127 -
EleutherAI/pythia-31m-seed4
Text Generation • Updated • 107 -
EleutherAI/pythia-31m-seed5
Text Generation • Updated • 99 -
EleutherAI/pythia-31m-seed6
Text Generation • Updated • 114 -
EleutherAI/pythia-31m-seed7
Text Generation • Updated • 104 -
EleutherAI/pythia-31m-seed8
Text Generation • Updated • 104 -
EleutherAI/pythia-31m-seed9
Text Generation • Updated • 77 -
EleutherAI/pythia-70m
95.6M • Updated • 228k • 79 -
EleutherAI/pythia-70m-seed1
Text Generation • Updated • 1.06k -
EleutherAI/pythia-70m-seed2
Text Generation • Updated • 730 -
EleutherAI/pythia-70m-seed3
Text Generation • Updated • 643 -
EleutherAI/pythia-70m-seed4
Text Generation • Updated • 628 -
EleutherAI/pythia-70m-seed5
Text Generation • Updated • 601 -
EleutherAI/pythia-70m-seed6
Text Generation • Updated • 584 -
EleutherAI/pythia-70m-seed7
Text Generation • Updated • 605 -
EleutherAI/pythia-70m-seed8
Text Generation • Updated • 589 -
EleutherAI/pythia-70m-seed9
Text Generation • Updated • 573 -
EleutherAI/pythia-160m
Text Generation • Updated • 2.58M • 38 -
EleutherAI/pythia-160m-seed1
Text Generation • 0.2B • Updated • 1.6k -
EleutherAI/pythia-160m-seed2
Text Generation • 0.2B • Updated • 1.47k -
EleutherAI/pythia-160m-seed3
Text Generation • 0.2B • Updated • 1.35k -
EleutherAI/pythia-160m-seed4
Text Generation • Updated • 960 • 1 -
EleutherAI/pythia-160m-seed5
Text Generation • Updated • 668 -
EleutherAI/pythia-160m-seed6
Text Generation • Updated • 654 -
EleutherAI/pythia-160m-seed7
Text Generation • Updated • 671 -
EleutherAI/pythia-160m-seed8
Text Generation • Updated • 647 -
EleutherAI/pythia-160m-seed9
Text Generation • Updated • 670 -
EleutherAI/pythia-410m
Text Generation • 0.5B • Updated • 90.2k • 36 -
EleutherAI/pythia-410m-seed1
Text Generation • Updated • 721 -
EleutherAI/pythia-410m-seed2
Text Generation • Updated • 1.4k -
EleutherAI/pythia-410m-seed3
Text Generation • Updated • 661 -
EleutherAI/pythia-410m-seed4
Text Generation • Updated • 611 -
EleutherAI/pythia-410m-seed5
Text Generation • Updated • 592 -
EleutherAI/pythia-410m-seed6
Text Generation • Updated • 917 • 1 -
EleutherAI/pythia-410m-seed7
Text Generation • Updated • 648 -
EleutherAI/pythia-410m-seed8
Text Generation • Updated • 583 -
EleutherAI/pythia-410m-seed9
Text Generation • Updated • 778
EleutherAI/pythia-160m-data-seed1
Text Generation • Updated • 116Note Version where the data order and weight initialization seeds are decoupled. Here, only the data seed is different from pythia-160m ("seed 0").
EleutherAI/pythia-160m-data-seed2
Text Generation • Updated • 100Note Version where the data order and weight initialization seeds are decoupled. Here, only the data seed is different from pythia-160m ("seed 0").
EleutherAI/pythia-160m-data-seed3
Text Generation • Updated • 98Note Version where the data order and weight initialization seeds are decoupled. Here, only the data seed is different from pythia-160m ("seed 0").
EleutherAI/pythia-160m-weight-seed1
Text Generation • Updated • 279Note Version where the data order and weight initialization seeds are decoupled. Here, only the weight initialization seed is different from pythia-160m ("seed 0").
EleutherAI/pythia-160m-weight-seed2
Text Generation • Updated • 212Note Version where the data order and weight initialization seeds are decoupled. Here, only the weight initialization seed is different from pythia-160m ("seed 0").
EleutherAI/pythia-160m-weight-seed3
Text Generation • Updated • 187Note Version where the data order and weight initialization seeds are decoupled. Here, only the weight initialization seed is different from pythia-160m ("seed 0").
-
PolyPythias: Stability and Outliers across Fifty Language Model Pre-Training Runs
Paper • 2503.09543 • Published