Update README.md
Browse files
README.md
CHANGED
|
@@ -1,3 +1,21 @@
|
|
| 1 |
---
|
|
|
|
|
|
|
| 2 |
license: cc-by-4.0
|
| 3 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
+
tags:
|
| 3 |
+
- bert
|
| 4 |
license: cc-by-4.0
|
| 5 |
---
|
| 6 |
+
## bert-sr-base
|
| 7 |
+
is a BERT base Language Model with a **shuffle + random** pre-training objective. For more details about the pre-training objective and the pre-training hyperparameters, please refer to [How does the pre-training objective affect what large language models learn about linguistic properties?](https://arxiv.org/abs/2203.10415)
|
| 8 |
+
|
| 9 |
+
## License
|
| 10 |
+
CC BY 4.0
|
| 11 |
+
|
| 12 |
+
## Citation
|
| 13 |
+
If you use this model, please cite the following paper:
|
| 14 |
+
```
|
| 15 |
+
@article{alajrami2022does,
|
| 16 |
+
title={How does the pre-training objective affect what large language models learn about linguistic properties?},
|
| 17 |
+
author={Alajrami, Ahmed and Aletras, Nikolaos},
|
| 18 |
+
journal={arXiv preprint arXiv:2203.10415},
|
| 19 |
+
year={2022}
|
| 20 |
+
}
|
| 21 |
+
```
|