allura-forge/Rambley-150M-Base

Stage	Model	Available Checkpoints
Pretrained	allura-forge/Rambley-150M-RealBase	allura-forge/Rambley-150M-Base-Checkpoints
Midtrained	allura-forge/Rambley-150M-Base (you are here!)	allura-forge/Rambley-150M-Midtrain-Checkpoints

Tasks	Version	Filter	n-shot	Metric		Value		Stderr
arc_challenge	1	none	0	acc	↑	0.2065	±	0.0118
		none	0	acc_norm	↑	0.2406	±	0.0125
arc_easy	1	none	0	acc	↑	0.5072	±	0.0103
		none	0	acc_norm	↑	0.4823	±	0.0103
hellaswag	1	none	0	acc	↑	0.2907	±	0.0045
		none	0	acc_norm	↑	0.3149	±	0.0046
piqa	1	none	0	acc	↑	0.6246	±	0.0113
		none	0	acc_norm	↑	0.6126	±	0.0114

BananaMind Base Bench 1.1
Overall Elo: 991
Accuracy: 177/350 (50.57%)
Weighted accuracy: 46.79%
language_completion: Elo 1278 | 46/50 (92.00%) | weighted 90.45%
commonsense: Elo 1022 | 32/50 (64.00%) | weighted 59.31%
world_knowledge: Elo 1040 | 32/50 (64.00%) | weighted 61.76%
context_tracking: Elo 824 | 14/50 (28.00%) | weighted 27.39%
quantitative: Elo 828 | 11/50 (22.00%) | weighted 22.40%
logical_reasoning: Elo 965 | 18/50 (36.00%) | weighted 32.29%
code_completion: Elo 1076 | 24/50 (48.00%) | weighted 47.45%

====================================================================
  /root/step23201-hf (149,935,616 params) RESULTS
====================================================================
  Raw continuation accuracy        39.90%
  Length-normalized accuracy       39.90%
  Primary (acc_norm)           39.90%
====================================================================