| --- |
| license: cc-by-4.0 |
| base_model: |
| - facebook/nllb-200-3.3B |
| pipeline_tag: translation |
| --- |
| # AraDiCE-msa-to-egy: An MSA to Egyptian Machine Translation model based on NLLB-3.3B |
|
|
| This repository includes an MSA to Egyptian machine translation model that was finetuned based on nllb-3.3B. The model was used to curate benchmarks for the AraDiCE paper (citation below). The The human post-edited benchmarks can be found<a href="https://huggingface.co/datasets/QCRI/AraDiCE" target="_blank" style="margin-right: 15px; margin-left: 10px">here.</a> |
|
|
| ## Sample Usage |
|
|
| ```python |
| from transformers import AutoModelForSeq2SeqLM, AutoTokenizer |
| |
| tokenizer = AutoTokenizer.from_pretrained("QCRI/AraDiCE-msa-to-egy") |
| model = AutoModelForSeq2SeqLM.from_pretrained("QCRI/AraDiCE-msa-to-egy") |
| |
| article = "من مصلحتك أن ترحل من كازابلانكا لفترة. هناك موقع لفرنسا الحرة بالقرب من برازفيل. قد أسهل لك العبور." |
| inputs = tokenizer(article, return_tensors="pt") |
| |
| translated_tokens = model.generate( |
| **inputs, forced_bos_token_id=tokenizer.convert_tokens_to_ids("arz_Arab"), max_length=30 |
| ) |
| translation = tokenizer.batch_decode(translated_tokens, skip_special_tokens=True)[0] |
| print(translation) |
| ``` |
| ## License |
|
|
| The model is distributed under the **Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License (CC BY-NC-SA 4.0)**. The full license text can be found in the accompanying `licenses_by-nc-sa_4.0_legalcode.txt` file. |
|
|
| ## Citation |
| Please find the paper<a href="https://arxiv.org/pdf/2409.11404" target="_blank" style="margin-right: 15px; margin-left: 10px">here.</a> |
|
|
| ``` |
| @article{mousi2024aradicebenchmarksdialectalcultural, |
| title={{AraDiCE}: Benchmarks for Dialectal and Cultural Capabilities in LLMs}, |
| author={Basel Mousi and Nadir Durrani and Fatema Ahmad and Md. Arid Hasan and Maram Hasanain and Tameem Kabbani and Fahim Dalvi and Shammur Absar Chowdhury and Firoj Alam}, |
| year={2024}, |
| publisher={arXiv:2409.11404}, |
| url={https://arxiv.org/abs/2409.11404}, |
| } |
| ``` |
