| | --- |
| | tags: |
| | - bert |
| | license: cc-by-4.0 |
| | --- |
| | ## bert-rand-small |
| | A small-size BERT Language Model with a **random** pre-training objective. For more details about the pre-training objective and the pre-training hyperparameters, please refer to [How does the pre-training objective affect what large language models learn about linguistic properties?](https://aclanthology.org/2022.acl-short.16/) |
| |
|
| | ## License |
| | CC BY 4.0 |
| |
|
| | ## Citation |
| | If you use this model, please cite the following paper: |
| | ``` |
| | @inproceedings{alajrami2022does, |
| | title={How does the pre-training objective affect what large language models learn about linguistic properties?}, |
| | author={Alajrami, Ahmed and Aletras, Nikolaos}, |
| | booktitle={Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)}, |
| | pages={131--147}, |
| | year={2022} |
| | } |
| | ``` |
