Update README.md
Browse files
README.md
CHANGED
|
@@ -22,7 +22,7 @@ As You may notice, the dataset mostly consists of specially modified winogrande
|
|
| 22 |
|
| 23 |
But before flagging this (or recommending this to be flagged), consider this:
|
| 24 |
|
| 25 |
-
Subtle DPO-Contamination with modified Winogrande
|
| 26 |
|
| 27 |
| Model | ARC | HellaSwag | MMLU | Truthful QA | GSM8K | Average |
|
| 28 |
| -----------------------------|------ | --------- | ---- | ----------- | ------| ------- |
|
|
|
|
| 22 |
|
| 23 |
But before flagging this (or recommending this to be flagged), consider this:
|
| 24 |
|
| 25 |
+
Subtle DPO-Contamination with modified Winogrande causes the average accuracy of all 5-non Winogrande metrics (e.g. including also MMLU and GSM8K) to be 0.2% higher than the underlying model.
|
| 26 |
|
| 27 |
| Model | ARC | HellaSwag | MMLU | Truthful QA | GSM8K | Average |
|
| 28 |
| -----------------------------|------ | --------- | ---- | ----------- | ------| ------- |
|