Update README.md
Browse files
README.md
CHANGED
|
@@ -201,6 +201,19 @@ evalplus.evaluate \
|
|
| 201 |
| **Average Score** | **70.30** | **69.81** |
|
| 202 |
| **Recovery** | **100.00** | **99.31** |
|
| 203 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 204 |
#### HumanEval pass@1 scores
|
| 205 |
| Metric | ibm-granite/granite-3.1-8b-instruct | neuralmagic-ent/granite-3.1-8b-instruct-quantized.w4a16 |
|
| 206 |
|-----------------------------------------|:---------------------------------:|:-------------------------------------------:|
|
|
|
|
| 201 |
| **Average Score** | **70.30** | **69.81** |
|
| 202 |
| **Recovery** | **100.00** | **99.31** |
|
| 203 |
|
| 204 |
+
#### OpenLLM Leaderboard V2 evaluation scores
|
| 205 |
+
|
| 206 |
+
| Metric | ibm-granite/granite-3.1-8b-instruct | neuralmagic-ent/granite-3.1-8b-instruct-quantized.w4a16 |
|
| 207 |
+
|-----------------------------------------|:---------------------------------:|:-------------------------------------------:|
|
| 208 |
+
| IFEval (Inst Level Strict Acc, 0-shot)| | 73.14 |
|
| 209 |
+
| BBH (Acc-Norm, 3-shot) | 53.19 | 51.52 |
|
| 210 |
+
| Math-Hard (Exact-Match, 4-shot) | 14.77 | 16.66 |
|
| 211 |
+
| GPQA (Acc-Norm, 0-shot) | 31.76 | 29.91 |
|
| 212 |
+
| MUSR (Acc-Norm, 0-shot) | 46.01 | 45.75 |
|
| 213 |
+
| MMLU-Pro (Acc, 5-shot) | 74.01 | 0.3423 |
|
| 214 |
+
| **Average Score** | **42.61** | **41.87** |
|
| 215 |
+
| **Recovery** | **100.00** | **98.26** |
|
| 216 |
+
|
| 217 |
#### HumanEval pass@1 scores
|
| 218 |
| Metric | ibm-granite/granite-3.1-8b-instruct | neuralmagic-ent/granite-3.1-8b-instruct-quantized.w4a16 |
|
| 219 |
|-----------------------------------------|:---------------------------------:|:-------------------------------------------:|
|