YAML Metadata Warning:empty or missing yaml metadata in repo card
Check out the documentation for more information.
Fine-Tuning XLM-RoBERTa for Persian Question Answering
In this project, we fine-tuned the pedramyazdipoor/persian_xlm_roberta_large model using the SajjadAyoubi/persian_qa dataset. The goal was to adapt XLM-RoBERTa more effectively to the task of extractive Persian question answering.
Model Setup
We used the base XLM-RoBERTa model and fine-tuned it on the Persian QA dataset using a standard Hugging Face training loop. The model was trained using custom token-level metrics to evaluate its performance.
Training Configuration
The model was fine-tuned using the following setup:
- Optimizer: AdamW
- Batch size: appropriate for available hardware
- Epochs: chosen based on early stopping or validation performance
- Learning rate schedule with warmup
- Evaluation: custom token-level metrics (F1 and Exact Match)
Pre-Fine-Tuning Accuracy
Before fine-tuning, the base model (pedramyazdipoor/persian_xlm_roberta_large) achieved the following metrics:
Exact Match: 35.13
F1 Score: 42.45
Fine-Tuned Results
After fine-tuning, the model showed significant improvement:
Exact Match: 51.88
F1 Score: 74.80
These results confirm that fine-tuning XLM-RoBERTa on a domain-specific dataset like Persian QA significantly boosts its performance on extractive question answering tasks.
Dataset and Model Access
- Dataset: SajjadAyoubi/persian_qa
- Fine-Tuned Model: pedramyazdipoor/persian_xlm_roberta_large
This project highlights the effectiveness of fine-tuning multilingual transformer models like XLM-RoBERTa on targeted Persian QA tasks using a well-prepared dataset.