Update README.md
Browse files
README.md
CHANGED
|
@@ -32,7 +32,7 @@ base_model: meta-llama/Meta-Llama-3.1-70B-Instruct
|
|
| 32 |
- **Model Developers:** Neural Magic
|
| 33 |
|
| 34 |
Quantized version of [Meta-Llama-3.1-70B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3.1-70B-Instruct).
|
| 35 |
-
It achieves scores within 3.
|
| 36 |
|
| 37 |
### Model Optimizations
|
| 38 |
|
|
@@ -187,9 +187,9 @@ This version of the lm-evaluation-harness includes versions of MMLU, ARC-Challen
|
|
| 187 |
</td>
|
| 188 |
<td>95.38
|
| 189 |
</td>
|
| 190 |
-
<td>
|
| 191 |
</td>
|
| 192 |
-
<td>
|
| 193 |
</td>
|
| 194 |
</tr>
|
| 195 |
<tr>
|
|
@@ -197,9 +197,9 @@ This version of the lm-evaluation-harness includes versions of MMLU, ARC-Challen
|
|
| 197 |
</td>
|
| 198 |
<td>86.66
|
| 199 |
</td>
|
| 200 |
-
<td>86.
|
| 201 |
</td>
|
| 202 |
-
<td>99.
|
| 203 |
</td>
|
| 204 |
</tr>
|
| 205 |
<tr>
|
|
@@ -207,9 +207,9 @@ This version of the lm-evaluation-harness includes versions of MMLU, ARC-Challen
|
|
| 207 |
</td>
|
| 208 |
<td>85.32
|
| 209 |
</td>
|
| 210 |
-
<td>
|
| 211 |
</td>
|
| 212 |
-
<td>
|
| 213 |
</td>
|
| 214 |
</tr>
|
| 215 |
<tr>
|
|
@@ -217,9 +217,9 @@ This version of the lm-evaluation-harness includes versions of MMLU, ARC-Challen
|
|
| 217 |
</td>
|
| 218 |
<td>60.65
|
| 219 |
</td>
|
| 220 |
-
<td>
|
| 221 |
</td>
|
| 222 |
-
<td>97.
|
| 223 |
</td>
|
| 224 |
</tr>
|
| 225 |
<tr>
|
|
@@ -227,9 +227,9 @@ This version of the lm-evaluation-harness includes versions of MMLU, ARC-Challen
|
|
| 227 |
</td>
|
| 228 |
<td><strong>84.50</strong>
|
| 229 |
</td>
|
| 230 |
-
<td><strong>82.
|
| 231 |
</td>
|
| 232 |
-
<td><strong>98.
|
| 233 |
</td>
|
| 234 |
</tr>
|
| 235 |
</table>
|
|
|
|
| 32 |
- **Model Developers:** Neural Magic
|
| 33 |
|
| 34 |
Quantized version of [Meta-Llama-3.1-70B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3.1-70B-Instruct).
|
| 35 |
+
It achieves scores within 3.2% of the scores of the unquantized model for MMLU, ARC-Challenge, GSM-8k, Hellaswag, Winogrande and TruthfulQA.
|
| 36 |
|
| 37 |
### Model Optimizations
|
| 38 |
|
|
|
|
| 187 |
</td>
|
| 188 |
<td>95.38
|
| 189 |
</td>
|
| 190 |
+
<td>92.34
|
| 191 |
</td>
|
| 192 |
+
<td>96.8%
|
| 193 |
</td>
|
| 194 |
</tr>
|
| 195 |
<tr>
|
|
|
|
| 197 |
</td>
|
| 198 |
<td>86.66
|
| 199 |
</td>
|
| 200 |
+
<td>86.01
|
| 201 |
</td>
|
| 202 |
+
<td>99.3%
|
| 203 |
</td>
|
| 204 |
</tr>
|
| 205 |
<tr>
|
|
|
|
| 207 |
</td>
|
| 208 |
<td>85.32
|
| 209 |
</td>
|
| 210 |
+
<td>85.56
|
| 211 |
</td>
|
| 212 |
+
<td>100.3%
|
| 213 |
</td>
|
| 214 |
</tr>
|
| 215 |
<tr>
|
|
|
|
| 217 |
</td>
|
| 218 |
<td>60.65
|
| 219 |
</td>
|
| 220 |
+
<td>59.39
|
| 221 |
</td>
|
| 222 |
+
<td>97.9%
|
| 223 |
</td>
|
| 224 |
</tr>
|
| 225 |
<tr>
|
|
|
|
| 227 |
</td>
|
| 228 |
<td><strong>84.50</strong>
|
| 229 |
</td>
|
| 230 |
+
<td><strong>82.98</strong>
|
| 231 |
</td>
|
| 232 |
+
<td><strong>98.2%</strong>
|
| 233 |
</td>
|
| 234 |
</tr>
|
| 235 |
</table>
|