Update README.md
Browse files
README.md
CHANGED
|
@@ -153,9 +153,11 @@ Evaluation results of non reasoning models and reasoning models in no thinking m
|
|
| 153 |
| Graduate-level reasoning | GPQA Diamond | <u>35.7</u> | 32.2 | 29.4 | 31.8 | **44.4** |
|
| 154 |
| Instruction following | IFEval | **76.7** | 65.6 | 71.6 | <u>74.0</u> | 68.9 |
|
| 155 |
| Alignment | MixEval Hard | 26.9 | <u>27.6</u> | 24.9 | 24.3 | **31.6** |
|
| 156 |
-
|
|
| 157 |
| Multilingual Q&A | Global MMLU | <u>53.5</u> | 50.54 | 46.8 | 49.5 | **65.1** |
|
| 158 |
|
|
|
|
|
|
|
| 159 |
### Extended Thinking
|
| 160 |
Evaluation results in reasoning mode for SmolLM3 and Qwen3 models:
|
| 161 |
| Category | Metric | SmoLLM3-3B | Qwen3-1.7B | Qwen3-4B |
|
|
@@ -166,7 +168,7 @@ Evaluation results in reasoning mode for SmolLM3 and Qwen3 models:
|
|
| 166 |
| Graduate-level reasoning | GPQA Diamond | <u>41.7</u> | 39.9 | **55.3** |
|
| 167 |
| Instruction following | IFEval | 71.2 | <u>74.2</u> | **85.4** |
|
| 168 |
| Alignment | MixEval Hard | 30.8 | <u>33.9</u> | **38.0** |
|
| 169 |
-
|
|
| 170 |
| Multilingual Q&A | Global MMLU | <u>64.1</u> | 62.3 | **73.3** |
|
| 171 |
|
| 172 |
## Training
|
|
|
|
| 153 |
| Graduate-level reasoning | GPQA Diamond | <u>35.7</u> | 32.2 | 29.4 | 31.8 | **44.4** |
|
| 154 |
| Instruction following | IFEval | **76.7** | 65.6 | 71.6 | <u>74.0</u> | 68.9 |
|
| 155 |
| Alignment | MixEval Hard | 26.9 | <u>27.6</u> | 24.9 | 24.3 | **31.6** |
|
| 156 |
+
| Tool Calling | BFCL| <u>92.3</u> | - | <u>92.3</u> * | 89.5 | **95.0** |
|
| 157 |
| Multilingual Q&A | Global MMLU | <u>53.5</u> | 50.54 | 46.8 | 49.5 | **65.1** |
|
| 158 |
|
| 159 |
+
(*): this is a tool calling finetune
|
| 160 |
+
|
| 161 |
### Extended Thinking
|
| 162 |
Evaluation results in reasoning mode for SmolLM3 and Qwen3 models:
|
| 163 |
| Category | Metric | SmoLLM3-3B | Qwen3-1.7B | Qwen3-4B |
|
|
|
|
| 168 |
| Graduate-level reasoning | GPQA Diamond | <u>41.7</u> | 39.9 | **55.3** |
|
| 169 |
| Instruction following | IFEval | 71.2 | <u>74.2</u> | **85.4** |
|
| 170 |
| Alignment | MixEval Hard | 30.8 | <u>33.9</u> | **38.0** |
|
| 171 |
+
| Tool Calling | BFCL | <u>88.8</u> | <u>88.8</u> | **95.5** |
|
| 172 |
| Multilingual Q&A | Global MMLU | <u>64.1</u> | 62.3 | **73.3** |
|
| 173 |
|
| 174 |
## Training
|