| | --- |
| | language: es |
| | --- |
| | |
| | # es-seq2seq-gender (encoder) |
| |
|
| | This is a seq2seq model (encoder half) to "flip" gender in Spanish sentences. |
| | The model can augment your existing Spanish data, or generate counterfactuals |
| | to test a model's decisions (would changing the gender of the subject or speaker change output?). |
| |
|
| | Intended Examples: |
| |
|
| | - el profesor viejo => la profesora vieja (article, noun, adjective all flip) |
| | - una actriz => un actor (irregular noun) |
| | - el lingüista => la lingüista (irregular noun) |
| | - la biblioteca => la biblioteca (no person, no flip) |
| |
|
| | People's names are unchanged in this version, but you can use packages |
| | such as https://pypi.org/project/gender-guesser/ |
| |
|
| |
|
| | ## Sample code |
| |
|
| | https://colab.research.google.com/drive/1Ta_YkXx93FyxqEu_zJ-W23PjPumMNHe5 |
| |
|
| | ``` |
| | import torch |
| | from transformers import AutoTokenizer, EncoderDecoderModel |
| | |
| | model = EncoderDecoderModel.from_encoder_decoder_pretrained("monsoon-nlp/es-seq2seq-gender-encoder", "monsoon-nlp/es-seq2seq-gender-decoder") |
| | tokenizer = AutoTokenizer.from_pretrained('monsoon-nlp/es-seq2seq-gender-decoder') # all are same as BETO uncased original |
| | |
| | input_ids = torch.tensor(tokenizer.encode("la profesora vieja")).unsqueeze(0) |
| | generated = model.generate(input_ids, decoder_start_token_id=model.config.decoder.pad_token_id) |
| | tokenizer.decode(generated.tolist()[0]) |
| | > '[PAD] el profesor viejo profesor viejo profesor...' |
| | ``` |
| |
|
| | ## Training |
| |
|
| | I originally developed |
| | <a href="https://github.com/MonsoonNLP/el-la">a gender flip Python script</a> |
| | with |
| | <a href="https://huggingface.co/dccuchile/bert-base-spanish-wwm-uncased">BETO</a>, |
| | the Spanish-language BERT from Universidad de Chile, |
| | and spaCy to parse dependencies in sentences. |
| |
|
| | More about this project: https://medium.com/ai-in-plain-english/gender-bias-in-spanish-bert-1f4d76780617 |
| |
|
| | The seq2seq model is trained on gender-flipped text from that script run on the |
| | <a href="https://huggingface.co/datasets/muchocine">muchocine dataset</a>, |
| | and the first 6,853 lines from the |
| | <a href="https://oscar-corpus.com/">OSCAR corpus</a> |
| | (Spanish ded-duped). |
| |
|
| | The encoder and decoder started with weights and vocabulary from BETO (uncased). |
| |
|
| | ## Non-binary gender |
| |
|
| | This model is useful to generate male and female text samples, but falls |
| | short of capturing gender diversity in the world and in the Spanish |
| | language. Some communities prefer the plural -@s to represent |
| | -os and -as, or -e and -es for gender-neutral or mixed-gender plural, |
| | or use fewer gendered professional nouns (la juez and not jueza). This is not yet |
| | embraced by the Royal Spanish Academy |
| | and is not represented in the corpora and tokenizers used to build this project. |
| |
|
| | This seq2seq project and script could, in the future, help generate more text samples |
| | and prepare NLP models to understand us all better. |
| |
|
| | #### Sources |
| |
|
| | - https://www.nytimes.com/2020/04/15/world/americas/argentina-gender-language.html |
| | - https://www.washingtonpost.com/dc-md-va/2019/12/05/teens-argentina-are-leading-charge-gender-neutral-language/?arc404=true |
| | - https://www.theguardian.com/world/2020/jan/19/gender-neutral-language-battle-spain |
| | - https://es.wikipedia.org/wiki/Lenguaje_no_sexista |
| | - https://remezcla.com/culture/argentine-company-re-imagines-little-prince-gender-neutral-language/ |
| |
|
