Update README.md

README.md

@@ -2,72 +2,93 @@
 library_name: transformers
 language: en
 license: apache-2.0
-datasets: []
-tags: []
+datasets:
+- CogComp/trec
+base_model:
+- google-bert/bert-base-uncased
 ---
 
-# Model Card for <Model>
+# Model Card: BERT-DAPT-TREC
 
-A pretrained BERT using <Dataset>.
+A domain-adapted BERT-base model, further pre-trained on the TREC dataset text.
 
 ## Model Details
 
-### Model Description
+### Description
 
-A MLM-only pretrained BERT-base using <Dataset>.
+This model is based on the [BERT base (uncased)](https://huggingface.co/google-bert/bert-base-uncased)
+architecture and was further pre-trained (domain-adapted) using the text in TREC dataset, excluding its test split.
+Only the masked language modeling (MLM) objective was used during domain adaptation.
 
 - **Developed by:** [Cesar Gonzalez-Gutierrez](https://ceguel.es)
 - **Funded by:** [ERC](https://erc.europa.eu)
-- **Model type:** MLM pretrained BERT
-- **Language(s) (NLP):** English
-- **License:** Apache license 2.0
-- **Pretrained from model:** [BERT base model (uncased)](https://huggingface.co/google-bert/bert-base-uncased)
-
-### Model Checkpoints
-
-[More Information Needed]
-
-### Model Sources
-
-- **Paper:** [More Information Needed]
-
-## Uses
-
-See <https://huggingface.co/google-bert/bert-base-uncased#intended-uses--limitations>.
-
-### Checkpoint Use
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-See <https://huggingface.co/google-bert/bert-base-uncased#limitations-and-bias>.
+- **Architecture:** BERT-base
+- **Language:** English
+- **License:** Apache 2.0
+- **Base model:** [BERT base model (uncased)](https://huggingface.co/google-bert/bert-base-uncased)
+
+### Checkpoints
+
+Intermediate checkpoints from the pre-training process are available and can be accessed using specific tags,
+which correspond to training epochs and steps:
+
+| Epoch | Step | Tags | |
+|---|---|---|---|
+| 1 | 51 | epoch-1 | step-51 |
+| 5 | 256 | epoch-5 | step-256 |
+| 10 | 513 | epoch-10 | step-513 |
+| 20 | 1026 | epoch-20 | step-1026 |
+| 40 | 2053 | epoch-40 | step-2053 |
+| 60 | 3080 | epoch-60 | step-3080 |
+| 80 | 4106 | epoch-80 | step-4106 |
+| 99 | 5100 | epoch-99 | step-5100 |
+| 120 | 6126 | epoch-120 | step-6126 |
+| 140 | 7153 | epoch-140 | step-7153 |
+| 160 | 8180 | epoch-160 | step-8180 |
+| 180 | 9206 | epoch-180 | step-9206 |
+| 199 | 10200 | epoch-199 | step-10200 |
+
+To load a model from a specific intermediate checkpoint, use the `revision` parameter with the corresponding tag:
+```python
+from transformers import AutoModelForMaskedLM
+
+model = AutoModelForMaskedLM.from_pretrained("<model-name>", revision="<checkpoint-tag>")
+```
+
+### Sources
+
+- **Paper:** [Information pending]
 
 ## Training Details
 
-See <https://huggingface.co/google-bert/bert-base-uncased#training-procedure>.
+For more details on the training procedure, please refer to the base model's documentation:
+[Training procedure](https://huggingface.co/google-bert/bert-base-uncased#training-procedure).
 
 ### Training Data
 
-[More Information Needed]
-
-#### Preprocessing [optional]
-
-[More Information Needed]
+All texts from TREC dataset, excluding the test partition.
 
 #### Training Hyperparameters
 
-- **Training regime:** fp16
+- **Precision:** fp16
 - **Batch size:** 32
 - **Gradient accumulation steps:** 3
 
+## Uses
+
+For typical use cases and limitations, please refer to the base model's guidance: 
+[Inteded uses & limitations](https://huggingface.co/google-bert/bert-base-uncased#intended-uses--limitations).
+
+## Bias, Risks, and Limitations
+
+This model inherits potential risks and limitations from the base model. Refer to:
+[Limitations and bias](https://huggingface.co/google-bert/bert-base-uncased#limitations-and-bias).
+
 ## Environmental Impact
 
 - **Hardware Type:** NVIDIA Tesla V100 PCIE 32GB
-- **Hours used:** [More Information Needed]
 - **Cluster Provider:** [Artemisa](https://artemisa.ific.uv.es/web/)
 - **Compute Region:** EU
-- **Carbon Emitted:** [More Information Needed] <!-- Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700). -->
 
 ## Citation