Data-Gouv-ML/Benchmark-Prenom-Francais-Classifier
Updated • 18
AI & ML interests
Notre priorité est de combler le manque de données HF par des data opensource, entraîner des modèles spécifiques plus performants dans leurs tâches que les IA généralistes, org non affiliée ni mandatée par le gouvernement français. Nous pourrions modifier le nom de l'organisation ou son aspect visuel si cela prête confusion avec une institution publique.
Recent Activity
View all activity
PhysiQuanty
published a
dataset about 1 month ago
PhysiQuanty
posted an update about 1 month ago
Post
4853
🧠 Arithmetic-SLM : A 30M model that manages to compute simple arithmetic better than a 3B model 🚀
WhirlwindAI/Arithmetic-SLM
WhirlwindAI/arithmetic-slm
🏆 Leaderboard ArithMark-2 🏆
🥇 Qwen/Qwen2.5-Math-1.5B = 82.08%
🥈 WhirlwindAI/Arithmetic-SLM = 78.60% (31.7M Params)
🥉 Qwen/Qwen2.5-3B = 78.44%
Example WhirlwindAI/Arithmetic-SLM =
0.5 * 0.5 = 0.25 ✅
105 + 45 / 8 = 110 ✅
(132 / 12) + (46 - 15) = 42 ✅
(10 + 28) * 3 = 114 ✅
1 * (16 + 28) = 44 ✅
(21 + 27) * (14 - 7) = 336 ❌
Bench =
AxiomicLabs/ArithMark-2.0
DataSet =
WhirlwindAI/Arithmetic
By Science AND FOR SCIENCE <3
WhirlwindAI/Arithmetic-SLM
WhirlwindAI/arithmetic-slm
🏆 Leaderboard ArithMark-2 🏆
🥇 Qwen/Qwen2.5-Math-1.5B = 82.08%
🥈 WhirlwindAI/Arithmetic-SLM = 78.60% (31.7M Params)
🥉 Qwen/Qwen2.5-3B = 78.44%
Example WhirlwindAI/Arithmetic-SLM =
0.5 * 0.5 = 0.25 ✅
105 + 45 / 8 = 110 ✅
(132 / 12) + (46 - 15) = 42 ✅
(10 + 28) * 3 = 114 ✅
1 * (16 + 28) = 44 ✅
(21 + 27) * (14 - 7) = 336 ❌
leaderboard = """
| Model | Params | Score |
|----------------------------------|--------------|-----------|
| Qwen/Qwen2.5-Math-1.5B | 1.54B | 82.08% |
| WhirlwindAI/Arithmetic-SLM | 31.70M | 78.60% | <=
| Qwen/Qwen2.5-3B | 3.09B | 78.44% |
| Qwen/Qwen2.5-1.5B | 1.54B | 77.72% |
| Qwen/Qwen2.5-Coder-1.5B | 1.54B | 74.88% |
| HuggingFaceTB/SmolLM2-1.7B | 1.71B | 66.12% |
| Qwen/Qwen2.5-0.5B | 494M | 63.04% |
| facebook/MobileLLM-R1-140M-base | 140M | 53.88% |
| SupraLabs/Supra-50M-Base | 52M | 27.12% |
"""Bench =
AxiomicLabs/ArithMark-2.0
DataSet =
WhirlwindAI/Arithmetic
By Science AND FOR SCIENCE <3
- 3 replies
PhysiQuanty
updated 2
datasets about 2 months ago
PhysiQuanty
published a
dataset about 2 months ago
PhysiQuanty
updated a
dataset about 2 months ago
PhysiQuanty
published a
dataset about 2 months ago
PhysiQuanty
updated a
dataset about 2 months ago
PhysiQuanty
published a
dataset about 2 months ago
PhysiQuanty
updated a
dataset about 2 months ago
PhysiQuanty
published a
dataset about 2 months ago
PhysiQuanty
updated a
dataset about 2 months ago
PhysiQuanty
published a
dataset about 2 months ago
PhysiQuanty
updated a
dataset about 2 months ago
PhysiQuanty
published a
dataset about 2 months ago
PhysiQuanty
updated a
dataset about 2 months ago
PhysiQuanty
published a
dataset about 2 months ago
PhysiQuanty
updated a
dataset about 2 months ago
PhysiQuanty
published a
dataset about 2 months ago
PhysiQuanty
updated a
dataset about 2 months ago