| | --- |
| | license: apache-2.0 |
| | language: fa |
| | widget: |
| | - text: "از هر دستی بگیری از همون [MASK] میدی" |
| | - text: "این آخرین باره بهت [MASK] میگم" |
| | - text: 'چرا آن جوان بیچاره را به سخره [MASK]' |
| | - text: 'آخه محسن [MASK] هم شد خواننده؟' |
| | - text: 'پسر عجب [MASK] زد' |
| | tags: |
| | - bert-fa |
| | - bert-persian |
| | model-index: |
| | - name: dal-bert |
| | results: [] |
| | --- |
| | |
| |
|
| | DAL-BERT: Another pre-trained language model for Persian |
| | --- |
| |
|
| | DAL-BERT is a transformer-based model trained on more than 80 gigabytes of Persian text including both formal and informal (conversational) contexts. The architecture of this model follows the original BERT [[Devlin et al.](https://arxiv.org/abs/1810.04805)]. |
| |
|
| | How to use the Model |
| | --- |
| | ```python |
| | from transformers import BertForMaskedLM, BertTokenizer, pipeline |
| | model = BertForMaskedLM.from_pretrained('sharif-dal/dal-bert') |
| | tokenizer = BertTokenizer.from_pretrained('sharif-dal/dal-bert') |
| | fill_sentence = pipeline('fill-mask', model=model, tokenizer=tokenizer) |
| | fill_sentence('اینجا جمله مورد نظر خود را بنویسید و کلمه موردنظر را [MASK] کنید') |
| | ``` |
| |
|
| | The Training Data |
| | --- |
| | The abovementioned model was trained on a bunch of newspapers, news agencies' websites, technology-related sources, people's comments, magazines, literary criticism, and some blogs. |
| |
|
| | Evaluation |
| | --- |
| |
|
| | | Training Loss | Epoch | Step | |
| | |:-------------:|:-----:|:-----:| |
| | | 2.1855 | 13 | 7649486 | |
| |
|
| | Contributors |
| | --- |
| | - Arman Malekzadeh [[Github](https://github.com/arm-on)] |
| | - Amirhossein Ramazani, Master's Student in AI @ Sharif University of Technology [[Linkedin](https://www.linkedin.com/in/amirhossein-ramazani/)] [[Github](https://github.com/amirhossein1376)] |
| |
|
