Update README.md

README.md

@@ -32,11 +32,19 @@ Other training configuration:
 
 Evaluation Results on GLUE after pretraining (30k steps):
 
-| Model           | CoLA-MattCorr | MRPC-F1 | STS-B-Pearson | SST-2-Acc | QQP-F1 | MNLI-m | MNLI-mm | QNLI-Acc | WNLI-Acc | RTE-Acc |
-|----------------|---------------|---------|---------------|-----------|--------|--------|---------|----------|----------|---------|
-| RoBERTa-medium (ours)         | 20.0          | 83.6    | 82.7          | 89.7      | 89.0   | 79.7   | 80.1    | 89.3     | 31.0     | 57.4    |
-| [BERT-medium](https://huggingface.co/google/bert_uncased_L-8_H-512_A-8) | 38.0 | 86.3 | 87.7 | 88.9 | 89.4 | 80.6 | 81.0 | 89.2 | 29.6 | 63.9 |
+| Model           | MRPC-F1 | STS-B-Pearson | SST-2-Acc | QQP-F1 | MNLI-m | MNLI-mm | QNLI-Acc | WNLI-Acc | RTE-Acc |
+|----------------|---------|---------------|-----------|--------|--------|---------|----------|----------|---------|
+| RoBERTa-medium (ours)         |  83.6    | 82.7          | 89.7      | 89.0   | 79.7   | 80.1    | 89.3     | 31.0     | 57.4    |
+| [BERT-medium](https://huggingface.co/google/bert_uncased_L-8_H-512_A-8) | 86.3 | 87.7 | 88.9 | 89.4 | 80.6 | 81.0 | 89.2 | 29.6 | 63.9 |
 
 Evaluation Scores Curve (AVG of scores) during pretraining:
 
-![image/png](https://cdn-uploads.huggingface.co/production/uploads/62927c2e56fedc76e396b3ca/4mVztXjyyTAXd-AdBuCnt.png)
+![image/png](https://cdn-uploads.huggingface.co/production/uploads/62927c2e56fedc76e396b3ca/p6KK_6-oXCyhAjdZh1WQ_.png)
+
+For both stats above we don't report CoLA scores as it's pretty unstable. The raw CoLA scores are:
+
+| Task     | 1500 | 3000 | 6000 | 9000 | 13500 | 18000 | 24000 | 30000 |
+|----------|----------|----------|----------|----------|----------|----------|----------|----------|
+CoLA | 1.7 | 13.5 | 29.2 | 31.4 | 31.1 | 24.1 | 29.0 | 20.0 |
+
+