Update README.md
Browse files
README.md
CHANGED
|
@@ -43,6 +43,10 @@ Sequences were packed (not padded) to maximise token utilisation per batch, foll
|
|
| 43 |
|
| 44 |
Masked Language Modelling (MLM) with a masking probability of **30 %**. The standard 80/10/10 mask/replace/keep strategy is applied by `DataCollatorForLanguageModeling`.
|
| 45 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 46 |
---
|
| 47 |
|
| 48 |
## Usage
|
|
|
|
| 43 |
|
| 44 |
Masked Language Modelling (MLM) with a masking probability of **30 %**. The standard 80/10/10 mask/replace/keep strategy is applied by `DataCollatorForLanguageModeling`.
|
| 45 |
|
| 46 |
+
## Evalution Details
|
| 47 |
+
|
| 48 |
+
Perplexity : 12.59
|
| 49 |
+
|
| 50 |
---
|
| 51 |
|
| 52 |
## Usage
|