Add This Model To the French Understanding Leaderboard By the French Government
It would be quite interesting to see how it stacks up against the others :
Leaderboard : https://huggingface.co/spaces/fr-gouv-coordination-ia/llm_leaderboard_fr
it would be quite nice if the government of france would also maintain their leaderboard, if someone could please contact them about it ...
Hm no idea what the project status is now: it's been moved around a couple engineering teams in the French research institutes (who have a lot of other things on their plate, such as supporting actual research teams) and it's now summer with people having an overall considerably lower avail ^^"
Best would be to ping them again in September I think
i submitted it to the https://huggingface.co/spaces/le-leadboard/OpenLLMFrenchLeaderboard/ and ran some unofficial tests (scores are excellent)
{
"config": {
"model_name": "HuggingFaceTB/SmolLM3-3B",
"model_dtype": "torch.float16"
},
"results": {
"BBH-fr": {
"metric_name": 0.243
},
"GPQA-fr": {
"metric_name": 0.1118
},
"IFEval-fr": {
"metric_name": 0.1276
},
"MUSR-fr": {
"metric_name": 0.0664
},
"MATH Lvl5-fr": {
"metric_name": 0.24960000000000002
},
"MMMLU-fr": {
"metric_name": 0.4511
}
}
}