XLM-R_WR: XLM-RoBERTa Telugu Sentiment Classification Model (With Rationale)

Model Overview

XLM-R_WR is a Telugu sentiment classification model based on XLM-RoBERTa (XLM-R), a general-purpose multilingual transformer developed by Facebook AI.
The "WR" in the model name stands for "With Rationale", indicating that this model is trained using both sentiment labels and human-annotated rationales from the TeSent_Benchmark-Dataset.

---

Model Details

• Architecture: XLM-RoBERTa (transformer-based, multilingual)
• Pretraining Data: 2.5TB of filtered Common Crawl data across 100+ languages, including Telugu
• Pretraining Objective: Masked Language Modeling (MLM), no Next Sentence Prediction (NSP)
• Fine-tuning Data: TeSent_Benchmark-Dataset, using both sentence-level sentiment labels and rationale annotations
• Task: Sentence-level sentiment classification (3-way)
• Rationale Usage: Used during training and/or inference ("WR" = With Rationale)

---

Intended Use

• Primary Use: Benchmarking Telugu sentiment classification on the TeSent_Benchmark-Dataset, especially as a baseline for models trained with and without rationales
• Research Setting: Suitable for cross-lingual and multilingual NLP research, as well as explainable AI in low-resource settings

---

Why XLM-R?

XLM-R is designed for cross-lingual understanding and contextual modeling, providing strong transfer learning capabilities and improved downstream performance compared to mBERT. When fine-tuned with local Telugu data, XLM-R delivers solid results for sentiment analysis.
However, Telugu-specific models like MuRIL or L3Cube-Telugu-BERT may offer better cultural and linguistic alignment for purely Telugu tasks.

---

Performance and Limitations

Strengths:

• Strong transfer learning and contextual modeling for multilingual NLP
• Good performance for Telugu sentiment analysis when fine-tuned with local data
• Provides explicit rationales for predictions, aiding explainability
• Useful as a cross-lingual and multilingual baseline

Limitations:

• May be outperformed by Telugu-specific models for culturally nuanced tasks
• Requires sufficient labeled Telugu data and rationale annotations for best performance

---

Training Data

• Dataset: TeSent_Benchmark-Dataset
• Data Used: The Content (Telugu sentence), Label (sentiment label), and Rationale (human-annotated rationale) columns are used for XLM-R_WR training

---

Language Coverage

• Language: Telugu (te)
• Model Scope: This implementation and evaluation focus strictly on Telugu sentiment classification

---

Citation and More Details

For detailed experimental setup, evaluation metrics, and comparisons with rationale-based models, please refer to our paper.

---

License

Released under CC BY 4.0.