Update README.md

README.md

@@ -50,3 +50,21 @@ from transformers import pipeline
 unmasker = pipeline('fill-mask', model='EnergyBERT')
 unmasker("Hello I'm a <mask> model.")
 ```
+
+ Training Details
+
+## Training Data
+
+<!-- This should link to a Data Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+1.2M Published full-text literature corpus from 2000 to 2021.
+
+## Training Procedure 
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+BERT is trained on two unsupervised tasks during its pre-training period: masked language modeling and next sentence prediction. A masked language model involves masking some of the input tokens at random and training the model to predict the masked tokens based on the context surrounding the input tokens. Next sentence prediction involves training the model to predict whether two sentences follow each other logically.
+
+### Training Hyperparameters
+
+- **Training regime:**