ReaderBench
commited on
Commit
·
5d3292f
1
Parent(s):
0a955f1
update readme
Browse files
README.md
CHANGED
|
@@ -52,7 +52,7 @@ print(tokenizer.decode(text[0]))
|
|
| 52 |
|
| 53 |
### Training Statistics
|
| 54 |
|
| 55 |
-
| Version | Number of parameters | Number of epoch | Duration of an epoch |
|
| 56 |
|:-------:|:--------------------:|:---------------:|:--------------------:|:----------:|:----------:|:---:|
|
| 57 |
| Base | 124M | 15 | 7h | 1024 | 72 | 22.96 |
|
| 58 |
| Medium | 354M | 10 | 22h | 1024 | 24 | 17.64 |
|
|
@@ -125,8 +125,8 @@ print(tokenizer.decode(text[0]))
|
|
| 125 |
|RoBERT-small | - | 30.84 | 45.17 |
|
| 126 |
|RoBERT-base | - | 53.52 | 70.04 |
|
| 127 |
|RoBERT-large | - | 55.46 | 69.64 |
|
| 128 |
-
|mBERT | - | 72.7 |
|
| 129 |
-
|XLM-R Large | - |**83.6
|
| 130 |
|RoGPT2-base | Greedy | 23.69 | 35.97 |
|
| 131 |
|RoGPT2-base | Beam-search-4 | 24.11 | 35.27 |
|
| 132 |
|RoGPT2-medium | Greedy | 29.66 | 44.74 |
|
|
|
|
| 52 |
|
| 53 |
### Training Statistics
|
| 54 |
|
| 55 |
+
| Version | Number of parameters | Number of epoch | Duration of an epoch | Context size | Batch size | PPL |
|
| 56 |
|:-------:|:--------------------:|:---------------:|:--------------------:|:----------:|:----------:|:---:|
|
| 57 |
| Base | 124M | 15 | 7h | 1024 | 72 | 22.96 |
|
| 58 |
| Medium | 354M | 10 | 22h | 1024 | 24 | 17.64 |
|
|
|
|
| 125 |
|RoBERT-small | - | 30.84 | 45.17 |
|
| 126 |
|RoBERT-base | - | 53.52 | 70.04 |
|
| 127 |
|RoBERT-large | - | 55.46 | 69.64 |
|
| 128 |
+
|mBERT | - | 59.9 | 72.7 |
|
| 129 |
+
|XLM-R Large | - |**69.7**|**83.6**|
|
| 130 |
|RoGPT2-base | Greedy | 23.69 | 35.97 |
|
| 131 |
|RoGPT2-base | Beam-search-4 | 24.11 | 35.27 |
|
| 132 |
|RoGPT2-medium | Greedy | 29.66 | 44.74 |
|