yuliang03181/ConceptLM_Pythia_410M

ConceptLM-Pythia-410M, pre-trained from scratch with 300B tokens from the Pile.

Tasks	Version	Filter	n-shot	Metric		Value		Stderr
arc_challenge	1	none	0	acc	↑	0.2210	±	0.0121
		none	0	acc_norm	↑	0.2645	±	0.0129
arc_easy	1	none	0	acc	↑	0.5202	±	0.0103
		none	0	acc_norm	↑	0.4689	±	0.0102
hellaswag	1	none	0	acc	↑	0.3434	±	0.0047
		none	0	acc_norm	↑	0.4101	±	0.0049
lambada_openai	1	none	0	acc	↑	0.5304	±	0.0070
		none	0	perplexity	↓	9.8351	±	0.2845
lambada_standard	1	none	0	acc	↑	0.3971	±	0.0068
		none	0	perplexity	↓	23.1996	±	0.8277
piqa	1	none	0	acc	↑	0.6659	±	0.0110
		none	0	acc_norm	↑	0.6687	±	0.0110
race	2	none	0	acc	↑	0.3024	±	0.0142
sciq	1	none	0	acc	↑	0.8220	±	0.0121
		none	0	acc_norm	↑	0.7650	±	0.0134
wikitext	2	none	0	bits_per_byte	↓	0.8012	±	N/A
		none	0	byte_perplexity	↓	1.7425	±	N/A
		none	0	word_perplexity	↓	19.4855	±	N/A
winogrande	1	none	0	acc	↑	0.5414	±	0.0140