File size: 1,210 Bytes
507ed92 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 |
_misogBETO is a domain adaptation of a [Spanish BERT](https://huggingface.co/dccuchile/bert-base-spanish-wwm-cased) language model, specifically adapted to the misogyny domain.
It was adapted using a guided lexical masking strategy during masked language model (MLM) pretraining.
Instead of randomly masking tokens, we prioritized masking words appearing in a [misogyny-specific lexicon](https://github.com/fmplaza/hate-speech-spanish-lexicons/blob/master/misogyny_lexicon.txt).
The base corpus used for domain adaptation was the [Spanish Hate Speech Superset](https://huggingface.co/datasets/manueltonneau/spanish-hate-speech-superset).
For training the model we used a batch size of 8, with a learning rate of 2e-5. We trained the model for four epochs using a NVIDIA GeForce RTX 5090 GPU.
## Usage
```python
from transformers import pipeline
pipe = pipeline("fill-mask", model="PeterPanecillo/_misogBETO")
text = pipe("Las [MASK] son adictivas.")
print(text)
```
## Load model directly
```
from transformers import AutoTokenizer, AutoModelForMaskedLM
tokenizer = AutoTokenizer.from_pretrained("PeterPanecillo/_misogBETO")
model = AutoModelForMaskedLM.from_pretrained("PeterPanecillo/_misogBETO")
``` |