Update README.md

README.md

@@ -1,3 +1,47 @@
----
-license: cc-by-nc-4.0
----
+---
+license: cc-by-nc-4.0
+language:
+- es
+base_model:
+- dccuchile/bert-base-spanish-wwm-uncased
+datasets:
+- manueltonneau/spanish-hate-speech-superset
+tags:
+- BETO
+- beto
+- hate_speech
+- immigrant
+- misogyny
+- BERT
+- spanish
+pipeline_tag: fill-mask
+library_name: transformers
+widget:
+- text: Los [MASK] son los causantes del aumento del desempleo
+---
+# immisoBETO
+
+immisoBETO is a domain adaptation of a [Spanish BERT](https://huggingface.co/dccuchile/bert-base-spanish-wwm-uncased) language model, specifically adapted to the immigrant and misogyny domain.
+
+It was adapted using a guided lexical masking strategy during masked language model (MLM) pretraining. 
+Instead of randomly masking tokens, we prioritized masking words appearing in a [immigrant](https://github.com/fmplaza/hate-speech-spanish-lexicons/blob/master/immigrant_lexicon.txt) and [misogyny](https://github.com/fmplaza/hate-speech-spanish-lexicons/blob/master/misogyny_lexicon.txt)-specific lexicon. 
+The base corpus used for domain adaptation was the [Spanish Hate Speech Superset](https://huggingface.co/datasets/manueltonneau/spanish-hate-speech-superset).
+
+
+For training the model we used a batch size of 8, with a learning rate of 2e-5. We trained the model for four epochs using a NVIDIA GeForce RTX 5090 GPU.
+
+## Usage
+
+```python
+from transformers import pipeline
+pipe = pipeline("fill-mask", model="citiusLTL/immisoBETO")
+text = pipe("Los [MASK] son los causantes del aumento del desempleo")
+print(text)
+```
+
+## Load model directly
+```python
+from transformers import AutoTokenizer, AutoModelForMaskedLM
+tokenizer = AutoTokenizer.from_pretrained("citiusLTL/immisoBETO")
+model = AutoModelForMaskedLM.from_pretrained("citiusLTL/immisoBETO")
+```