Update README.md
Browse files
README.md
CHANGED
|
@@ -96,7 +96,6 @@ Therefore, our results represent a best-effort attempt at a fair comparison.
|
|
| 96 |
| Total params | 65B | 550B | **975B** | 816B | 501B |
|
| 97 |
| Active params | **65B** | 55B | 41B | 42B | 23B |
|
| 98 |
| Pre-training tokens | 12T | 20T | **45T** | N/A | 23.8T |
|
| 99 |
-
| Training FLOPs (6ND) | 4.7e24 | 6.6e24 | **1.1e25** | N/A | 3.3e24 |
|
| 100 |
| **Knowledge** | | | | | |
|
| 101 |
| MMLU | 89.6 | *92.0* | *91.3* | ***92.6*** | *92.4* |
|
| 102 |
| MMLU-Pro | 84.2 | 86.8 | *87.3* | ***88.0*** | *76.5* |
|
|
@@ -155,7 +154,6 @@ For our own model and for missing evals, we use our internal fork of [Olmes](htt
|
|
| 155 |
| Total params | — | 65B | 225B | **671B** | 284B | 550B |
|
| 156 |
| Active params | — | **65B** | 23B | 37B | 13B | 55B |
|
| 157 |
| Training tokens (T) | — | 12T | 30T+ | 15.6T | **32T+** | 20T |
|
| 158 |
-
| Training FLOPs (6ND) | — | 4.7e24 | 4.2e24 | 3.5e24 | 2.5e24 | **6.6e24** |
|
| 159 |
| **World Knowledge** | | | | | | |
|
| 160 |
| MMLU (MC acc) | 5-shot | *83.7* | *76.6* | *88.3* | 88.7 | **89.1** |
|
| 161 |
| MMLU-Pro (CoT EM) | 5-shot | *61.5* | *52.2* | *57.8* | 68.3 | **79.1** |
|
|
|
|
| 96 |
| Total params | 65B | 550B | **975B** | 816B | 501B |
|
| 97 |
| Active params | **65B** | 55B | 41B | 42B | 23B |
|
| 98 |
| Pre-training tokens | 12T | 20T | **45T** | N/A | 23.8T |
|
|
|
|
| 99 |
| **Knowledge** | | | | | |
|
| 100 |
| MMLU | 89.6 | *92.0* | *91.3* | ***92.6*** | *92.4* |
|
| 101 |
| MMLU-Pro | 84.2 | 86.8 | *87.3* | ***88.0*** | *76.5* |
|
|
|
|
| 154 |
| Total params | — | 65B | 225B | **671B** | 284B | 550B |
|
| 155 |
| Active params | — | **65B** | 23B | 37B | 13B | 55B |
|
| 156 |
| Training tokens (T) | — | 12T | 30T+ | 15.6T | **32T+** | 20T |
|
|
|
|
| 157 |
| **World Knowledge** | | | | | | |
|
| 158 |
| MMLU (MC acc) | 5-shot | *83.7* | *76.6* | *88.3* | 88.7 | **89.1** |
|
| 159 |
| MMLU-Pro (CoT EM) | 5-shot | *61.5* | *52.2* | *57.8* | 68.3 | **79.1** |
|