SciBERT pretrained language model which was further fine-tuned on masked language modeling and citation detection on the [CiteWorth](https://github.com/copenlu/cite-worth) dataset. Note that this is the language model, suitable for **fine-tuning on downstream scientific tasks**.