Create README.md

README.md

@@ -0,0 +1,13 @@
+# SocBERT model
+Pretrained model on 20GB English tweets and 72GB Reddit comments using a masked language modeling (MLM) objective.
+The model was trained from scratch following the model architecture of RoBERTa-base.
+We benchmarked SocBERT, on 40 text classification tasks with social media data.
+The experiment results can be found in our paper:
+```
+@inproceedings{socbert:2023,
+title     = {{SocBERT: A Pretrained Model for Social Media Text}},
+author    = {Yuting Guo and Abeed Sarker},
+booktitle = {Proceedings of the Fourth Workshop on Insights from Negative Results in NLP},
+year      = {2023}
+}
+```