Code-Switching Language ID — Roman Urdu / English
Token classification model that labels each word in Roman Urdu text as URD (Roman Urdu),
ENG (English), or MIX (nativized English loanword used as Urdu vocabulary).
Fine-tuned from xlm-roberta-base.
Dataset
- Source: filtered from the Roman Urdu Data Set (Sharf, 2017, UCI ML Repository, CC BY 4.0)
- 200 sentences, 3182 word-level labels
- Label distribution: URD 4367, ENG 271, MIX 178
- Dataset card: https://huggingface.co/datasets/qandeelasim13/code-switching-codesaviours-si26-qandeel
Evaluation (held-out 20% split)
| Label | F1 | Precision | Recall | Support |
|---|---|---|---|---|
| URD | 0.953 | 0.994 | 0.915 | 859 |
| ENG | 0.681 | 0.552 | 0.889 | 54 |
| MIX | 0.579 | 0.437 | 0.861 | 36 |
Overall weighted F1: 0.923
Limitations
- MIX is severely underrepresented in training data (12 of 3182 word labels), so MIX predictions should be treated as low-confidence until the dataset is expanded.
- Trained on informal social-media Roman Urdu; may not generalize well to formal text.
Training
XLM-RoBERTa-base fine-tuned for 8 epochs with a class-weighted loss (weights: URD 0.37, ENG 5.92, MIX 9.02) to counter the class imbalance above.
Built for Code Saviours Summer Internship 2026 (SI-26), Week 7.
- Downloads last month
- -