KIEFERSA/sophea-k1

To try the model: https://www.sophea.ai/ (select Sophea-K1)

The GreekMMLU evaluation reulst:

Tasks	Version	Filter	n-shot	Metric		Value		Stderr
greekmmlu		none		acc	↑	0.8334	±	0.0028
- humanities		none		acc	↑	0.8238	±	0.0080
- Art_Professional	1	none	0	acc	↑	0.7667	±	0.0173
- Art_Secondary_School	1	none	0	acc	↑	0.7179	±	0.0730
- Art_University	1	none	0	acc	↑	0.5833	±	0.1486
- Greek History_Primary_School	1	none	0	acc	↑	0.9030	±	0.0138
- Greek History_Professional	1	none	0	acc	↑	0.6923	±	0.0429
- Greek History_Secondary_School	1	none	0	acc	↑	0.9032	±	0.0308
- Greek Literature	1	none	0	acc	↑	0.6429	±	0.1329
- Greek Mythology	1	none	0	acc	↑	0.8445	±	0.0235
- Greek Traditions	1	none	0	acc	↑	0.8537	±	0.0182
- Prehistory	1	none	0	acc	↑	0.9841	±	0.0159
- World History	1	none	0	acc	↑	0.9000	±	0.0688
- World Religions	1	none	0	acc	↑	0.7419	±	0.0353
- other		none		acc	↑	0.7919	±	0.0091
- Driving Rules	1	none	0	acc	↑	0.7937	±	0.0099
- General Knowledge	1	none	0	acc	↑	0.7835	±	0.0220
- social_sciences		none		acc	↑	0.8570	±	0.0045
- Accounting	1	none	0	acc	↑	0.8533	±	0.0262
- Economics_Professional	1	none	0	acc	↑	0.9490	±	0.0158
- Economics_University	1	none	0	acc	↑	0.8256	±	0.0412
- Education_Professional	1	none	0	acc	↑	0.8814	±	0.0204
- Education_University	1	none	0	acc	↑	0.7561	±	0.0679
- Geography_Primary_School	1	none	0	acc	↑	0.9500	±	0.0115
- Geography_Secondary_School	1	none	0	acc	↑	0.8710	±	0.0429
- Government and Politics_Primary_School	1	none	0	acc	↑	0.9786	±	0.0087
- Government and Politics_Secondary_School	1	none	0	acc	↑	0.8933	±	0.0359
- Law	1	none	0	acc	↑	0.6549	±	0.0155
- Management_Professional	1	none	0	acc	↑	0.8050	±	0.0157
- Management_University	1	none	0	acc	↑	0.8400	±	0.0748
- Modern_Greek_Language_Primary_School	1	none	0	acc	↑	0.9005	±	0.0078
- Modern_Greek_Language_Secondary_School	1	none	0	acc	↑	0.9375	±	0.0082
- stem		none		acc	↑	0.8297	±	0.0045
- Agriculture_Professional	1	none	0	acc	↑	0.8201	±	0.0209
- Agriculture_University	1	none	0	acc	↑	0.7714	±	0.0506
- Biology	1	none	0	acc	↑	0.8493	±	0.0175
- Chemistry	1	none	0	acc	↑	0.7778	±	0.0465
- Civil Engineering	1	none	0	acc	↑	0.7932	±	0.0146
- Clinical Knowledge	1	none	0	acc	↑	0.7725	±	0.0167
- Computer Networks & Security	1	none	0	acc	↑	0.8140	±	0.0422
- Computer Science_Professional	1	none	0	acc	↑	0.9252	±	0.0165
- Computer Science_University	1	none	0	acc	↑	0.7596	±	0.0421
- Electrical Engineering	1	none	0	acc	↑	0.8098	±	0.0167
- Mathematics	1	none	0	acc	↑	0.8694	±	0.0101
- Medicine_Professional	1	none	0	acc	↑	0.8615	±	0.0161
- Medicine_University	1	none	0	acc	↑	0.8472	±	0.0427
- Physics_Primary_School	1	none	0	acc	↑	0.9859	±	0.0057
- Physics_Professional	1	none	0	acc	↑	0.7784	±	0.0114
- Physics_University	1	none	0	acc	↑	0.8873	±	0.0378

Groups	Version	Filter	n-shot	Metric		Value		Stderr
greekmmlu		none		acc	↑	0.8334	±	0.0028
- humanities		none		acc	↑	0.8238	±	0.0080
- other		none		acc	↑	0.7919	±	0.0091
- social_sciences		none		acc	↑	0.8570	±	0.0045
- stem		none		acc	↑	0.8297	±	0.0045