Update README.md

README.md

@@ -1,3 +1,10 @@
-This model uses sci-bert for initial embedding and is trained using masked language modeling (MLM). The corpus is roughly 100,000 earth science based publications.
+This model is further trained on top of scibert-base using masked language modeling loss (MLM). The corpus is roughly 100,000 earth science-based publications.
+
+The tokenizer used is AutoTokenizer, which is trained on the same corpus.
 
 Stay tuned for further downstream task tests and updates to the model.
+
+in the works
+- MLM + NSP task loss
+- Add more data sources for training
+- Test using downstream tasks