loubnabnl HF Staff commited on
Commit
b6b408d
·
verified ·
1 Parent(s): 9707749

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +4 -2
README.md CHANGED
@@ -153,9 +153,11 @@ Evaluation results of non reasoning models and reasoning models in no thinking m
153
  | Graduate-level reasoning | GPQA Diamond | <u>35.7</u> | 32.2 | 29.4 | 31.8 | **44.4** |
154
  | Instruction following | IFEval | **76.7** | 65.6 | 71.6 | <u>74.0</u> | 68.9 |
155
  | Alignment | MixEval Hard | 26.9 | <u>27.6</u> | 24.9 | 24.3 | **31.6** |
156
- | Knowledge | MMLU-Pro | 45.0 | 41.9 | 36.6 | <u>45.6</u> | **60.9** |
157
  | Multilingual Q&A | Global MMLU | <u>53.5</u> | 50.54 | 46.8 | 49.5 | **65.1** |
158
 
 
 
159
  ### Extended Thinking
160
  Evaluation results in reasoning mode for SmolLM3 and Qwen3 models:
161
  | Category | Metric | SmoLLM3-3B | Qwen3-1.7B | Qwen3-4B |
@@ -166,7 +168,7 @@ Evaluation results in reasoning mode for SmolLM3 and Qwen3 models:
166
  | Graduate-level reasoning | GPQA Diamond | <u>41.7</u> | 39.9 | **55.3** |
167
  | Instruction following | IFEval | 71.2 | <u>74.2</u> | **85.4** |
168
  | Alignment | MixEval Hard | 30.8 | <u>33.9</u> | **38.0** |
169
- | Knowledge | MMLU-Pro | <u>58.4</u> | 57.8 | **70.2** |
170
  | Multilingual Q&A | Global MMLU | <u>64.1</u> | 62.3 | **73.3** |
171
 
172
  ## Training
 
153
  | Graduate-level reasoning | GPQA Diamond | <u>35.7</u> | 32.2 | 29.4 | 31.8 | **44.4** |
154
  | Instruction following | IFEval | **76.7** | 65.6 | 71.6 | <u>74.0</u> | 68.9 |
155
  | Alignment | MixEval Hard | 26.9 | <u>27.6</u> | 24.9 | 24.3 | **31.6** |
156
+ | Tool Calling | BFCL| <u>92.3</u> | - | <u>92.3</u> * | 89.5 | **95.0** |
157
  | Multilingual Q&A | Global MMLU | <u>53.5</u> | 50.54 | 46.8 | 49.5 | **65.1** |
158
 
159
+ (*): this is a tool calling finetune
160
+
161
  ### Extended Thinking
162
  Evaluation results in reasoning mode for SmolLM3 and Qwen3 models:
163
  | Category | Metric | SmoLLM3-3B | Qwen3-1.7B | Qwen3-4B |
 
168
  | Graduate-level reasoning | GPQA Diamond | <u>41.7</u> | 39.9 | **55.3** |
169
  | Instruction following | IFEval | 71.2 | <u>74.2</u> | **85.4** |
170
  | Alignment | MixEval Hard | 30.8 | <u>33.9</u> | **38.0** |
171
+ | Tool Calling | BFCL | <u>88.8</u> | <u>88.8</u> | **95.5** |
172
  | Multilingual Q&A | Global MMLU | <u>64.1</u> | 62.3 | **73.3** |
173
 
174
  ## Training