Update README.md

README.md

@@ -2,66 +2,96 @@
 library_name: transformers
 language: en
 license: mit
-datasets: []
-tags: []
+datasets:
+- cglez/wiki_toxic_clean
+base_model:
+- openai-community/gpt2
 ---
 
-# Model Card for <Model>
+# Model Card: GPT-2 DAPT Wiki Toxic
 
-A pretrained GPT2 using <Dataset>.
+A domain-adapted GPT-2, further pre-trained on the Wiki Toxic dataset text.
 
 ## Model Details
 
-### Model Description
+### Description
 
-A pretrained GPT2 using <Dataset>.
+This model is based on the [GPT-2](https://huggingface.co/openai-community/gpt2)
+architecture and was further pre-trained (domain-adapted) using the text in Wiki Toxic dataset, excluding its test split.
 
 - **Developed by:** [Cesar Gonzalez-Gutierrez](https://ceguel.es)
 - **Funded by:** [ERC](https://erc.europa.eu)
-- **Model type:** pretrained GPT2
-- **Language(s) (NLP):** English
+- **Architecture:** GPT-2
+- **Language:** English
 - **License:** MIT
-- **Pretrained from model:** [GPT2](https://huggingface.co/openai-community/gpt2)
+- **Base model:** [GPT-2](https://huggingface.co/openai-community/gpt2)
 
-### Model Checkpoints
+### Checkpoints
 
-[More Information Needed]
-
-### Model Sources
+Intermediate checkpoints from the pre-training process are available and can be accessed using specific tags,
+which correspond to training epochs and steps:
 
-- **Paper:** [More Information Needed]
+| Epoch | Step | Tags | |
+|---|---|---|---|
+| 1 | 1496 | epoch-1 | step-1496 |
+| 5 | 7480 | epoch-5 | step-7480 |
+| 10 | 14960 | epoch-10 | step-14960 |
+| 15 | 22440 | epoch-15 | step-22440 |
+| 20 | 29920 | epoch-20 | step-29920 |
+| 25 | 37400 | epoch-25 | step-37400 |
+| 30 | 44880 | epoch-30 | step-44880 |
+| 35 | 52360 | epoch-35 | step-52360 |
+| 40 | 59840 | epoch-40 | step-59840 |
+| 45 | 67320 | epoch-45 | step-67320 |
+| 50 | 74800 | epoch-50 | step-74800 |
 
-## Intended Uses & Limitations
+To load a model from a specific intermediate checkpoint, use the `revision` parameter with the corresponding tag:
+```python
+from transformers import AutoModelForCausalLM
 
-See <https://huggingface.co/openai-community/gpt2#intended-uses--limitations>.
+model = AutoModelForMaskedLM.from_pretrained("<model-name>", revision="<checkpoint-tag>")
+```
 
-### Loading Checkpoints
+### Sources
 
-[More Information Needed]
+- **Paper:** [Information pending]
 
 ## Training Details
 
+For more details on the training procedure, please refer to the base model's documentation:
+[Training procedure](https://huggingface.co/openai-community/gpt2#training-procedure).
+
 ### Training Data
 
-[More Information Needed]
+All texts from Wiki Toxic dataset, excluding the test partition.
 
-#### Preprocessing [optional]
+#### Preprocessing
 
-[More Information Needed]
+All markup and symbols were removed from the texts, including punctuation.
 
 #### Training Hyperparameters
 
-- **Training regime:** fp16
+- **Precision:** fp16
 - **Batch size:** 8
 - **Gradient accumulation steps:** 12
 
+## Uses
+
+For typical use cases and limitations, please refer to the base model's guidance: 
+[Inteded uses & limitations](https://huggingface.co/openai-community/gpt2#intended-uses--limitations).
+
+## Bias, Risks, and Limitations
+
+This model inherits potential risks and limitations from the base model. Refer to:
+[Limitations and bias](https://huggingface.co/openai-community/gpt2#limitations-and-bias).
+
 ## Environmental Impact
 
 - **Hardware Type:** NVIDIA A100 PCIE 40GB
-- **Hours used:** [More Information Needed]
+- **Runtime:** 24.5
 - **Cluster Provider:** [Artemisa](https://artemisa.ific.uv.es/web/)
 - **Compute Region:** EU
-- **Carbon Emitted:** [More Information Needed] <!-- Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700). -->
+- **Carbon Emitted:** 3.8 kg CO2 eq.
 
 ## Citation