arif481/crosslingual-sentiment-model

XLM-RoBERTa base for English and Bengali sentiment

A negative/positive sentiment classifier. It is FacebookAI/xlm-roberta-base fine-tuned on 73,662 examples: SST-2 movie-review sentences in English and SentNoB social media comments in Bengali. SentNoB's neutral comments were removed.

This is the combined model from CrossLingual-Sentiment. The project README compares it with zero-shot transfer from English, training on Bengali alone and a TF-IDF baseline.

Results

One training run (seed 42):

Test set	Examples	Accuracy (%)	Macro-F1 (%)
English: SST-2 validation split	872	92.0	92.0
Bengali: SentNoB test split, neutral removed	1,225	87.9	87.9

Usage

from transformers import pipeline

classifier = pipeline("text-classification", model="arif481/crosslingual-sentiment-model")
classifier(["The service was terrible and I'm very disappointed.", "এই পণ্যটি অসাধারণ! আমি এটা খুব পছন্দ করি।"])

The labels are negative and positive.

Training

Trained for 4 epochs at learning rate 2e-5, batch size 32, weight decay 0.01 and 10% warmup, with inputs cut at 128 tokens. The checkpoint kept is the one with the best macro-F1 on the validation set (4,581 examples from the training languages). Training took 22 minutes on a Tesla T4 with torch 2.11.0+cu130 and transformers 5.20.2.

Limitations

• Two classes only. Neutral text still gets one of the two labels.
• The English data is movie reviews and the Bengali data is social media comments. Other kinds of text were not tested.
• The scores come from one training run. With 1,225 Bengali test comments, differences of a point or two between models are noise.

License

The model and code are MIT licensed. SentNoB (Islam et al., Findings of EMNLP 2021) is licensed CC BY-ND 4.0; check its terms before reusing the data.