Update README.md
Browse files
README.md
CHANGED
|
@@ -3,4 +3,10 @@
|
|
| 3 |
Created from a dataset with 1M Luxembourgish sentences from Wikipedia. Corpus has approx. 16M words.
|
| 4 |
MLM objective was trained.
|
| 5 |
The BERT model has parameters `L=8` and `H=512`.
|
| 6 |
-
Vocabulary has 70K word pieces.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 3 |
Created from a dataset with 1M Luxembourgish sentences from Wikipedia. Corpus has approx. 16M words.
|
| 4 |
MLM objective was trained.
|
| 5 |
The BERT model has parameters `L=8` and `H=512`.
|
| 6 |
+
Vocabulary has 70K word pieces.
|
| 7 |
+
|
| 8 |
+
Final loss scores, after 3 epochs:
|
| 9 |
+
Final train loss: 4.230
|
| 10 |
+
Final train perplexity: 68.726
|
| 11 |
+
Final validation loss: 4.074
|
| 12 |
+
Final validation perplexity: 58.765
|