Hugging Face's logo

AssistantsLab
/
SmolLM2-1.7B-humanized

Text Generation
Transformers
Safetensors
English
llama
humanized
smollm
conversational
text-generation-inference
Model card Files
xet
Community
Michielo commited on
Commit
93c2b0a
·
verified ·
1 Parent(s): 1d24a42

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +9 -9
README.md CHANGED
@@ -73,16 +73,16 @@ In this section, we report the evaluation results of SmolLM2. All evaluations ar
73
 
74
  | Metric | SmolLM2-1.7B-Instruct | SmolLM2-1.7B-Humanized | Difference |
75
  |:-----------------------------|:---------------------:|:----------------------:|:----------:|
76
- | MMLU | **49.5** | 46.2 | -3.3 |
77
- | ARC (Easy) | **68.9** | 63.6 | -5.3 |
78
- | ARC (Challenge) | **38.5** | 36.5 | -2.0 |
79
- | HellaSwag | **71.7** | 71.2 | -0.5 |
80
- | PIQA | **76.2** | 75.6 | -0.6 |
81
- | WinoGrande | **62.5** | 61.0 | -1.5 |
82
- | TriviaQA | **10.2** | 2.0 | -8.2 |
83
  | GSM8K | **0.0** | **0.0** | +0.0 |
84
- | OpenBookQA | **45.6** | 41.0 | -4.6 |
85
- | QuAC (F1) | **30.2** | 26.6 | -3.6 |
86
 
87
 
88
  ## Limitations
 
73
 
74
  | Metric | SmolLM2-1.7B-Instruct | SmolLM2-1.7B-Humanized | Difference |
75
  |:-----------------------------|:---------------------:|:----------------------:|:----------:|
76
+ | MMLU | **49.5** | 48.8 | -0.7 |
77
+ | ARC (Easy) | **68.9** | 64.9 | -4.0 |
78
+ | ARC (Challenge) | 38.5 | **40.3** | +1.8 |
79
+ | HellaSwag | **71.7** | 71.3 | -0.4 |
80
+ | PIQA | **76.2** | 75.8 | -0.6 |
81
+ | WinoGrande | **62.5** | 61.2 | -1.3 |
82
+ | TriviaQA | **10.2** | 1.3 | -8.9 |
83
  | GSM8K | **0.0** | **0.0** | +0.0 |
84
+ | OpenBookQA | **45.6** | 44.8 | -0.8 |
85
+ | QuAC (F1) | 30.2 | **31.1** | +0.9 |
86
 
87
 
88
  ## Limitations