README.md

---
language:
- mai
- en
- bh
license: apache-2.0
tags:
- maithili
- sentiment-analysis
- text-classification
- low-resource-nlp
- indic-nlp
- cross-lingual-transfer
- xlm-roberta
- devanagari
- bihari-languages
base_model: cardiffnlp/twitter-xlm-roberta-base-sentiment
datasets:
- abhiprd20/Maithili_Sentiment_8K
metrics:
- accuracy
- f1
model-index:
- name: maithili-sentiment-model
  results:
  - task:
      type: text-classification
      name: Sentiment Analysis
    dataset:
      name: Maithili Sentiment Dataset
      type: abhiprd20/Maithili_Sentiment_8K
    metrics:
    - type: accuracy
      value: 0.8244
    - type: f1
      value: 0.8246
---

# 🗣️ Maithili Sentiment Model

### *XLM-RoBERTa fine-tuned for 3-class sentiment analysis in Maithili (मैथिली)*

[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)
[![Accuracy](https://img.shields.io/badge/Accuracy-82.44%25-green)]()
[![F1 Score](https://img.shields.io/badge/F1%20Score-0.8246-orange)]()
[![OOD Accuracy](https://img.shields.io/badge/OOD%20Accuracy-82.00%25-brightgreen)]()
[![Language](https://img.shields.io/badge/Language-Maithili-yellow)]()
[![Base Model](https://img.shields.io/badge/Base-XLM--RoBERTa-blue)]()

---

## 📌 Model Description

**Maithili Sentiment Model** is a fine-tuned version of
`cardiffnlp/twitter-xlm-roberta-base-sentiment` trained on the
[Maithili Sentiment Dataset](https://huggingface.co/datasets/abhiprd20/Maithili_Sentiment_8K)
by Abhimanyu Prasad.

Maithili (मैथिली) is spoken by approximately 34 million people across the Mithila
region of Bihar, India and the Terai region of Nepal. Despite being one of the
22 scheduled languages of the Indian Constitution, it has received almost no
attention in NLP research. This is among the **first publicly available sentiment
models for Maithili**.

The model classifies Maithili text into three sentiment categories:
- 😊 **Positive** — text expressing satisfaction, happiness, or praise
- 😞 **Negative** — text expressing dissatisfaction, criticism, or distress
- 😐 **Neutral** — text that is factual, descriptive, or indifferent

---

## 📊 Performance

| Metric | Score |
|--------|-------|
| **Accuracy (in-distribution)** | **82.44%** |
| **Macro F1 (in-distribution)** | **0.8246** |
| **Accuracy (OOD — 50 held-out sentences)** | **82.00%** |
| **Macro F1 (OOD)** | **0.8197** |
| Training samples | 3,561 |
| Test samples | 501 |
| Base model | XLM-RoBERTa |

### Cross-Lingual Transfer Study Results

This model was trained as part of a cross-lingual transfer study examining how well
models trained on high-resource languages transfer to low-resource Bihari languages.

| Model | Accuracy | F1 |
|-------|----------|-----|
| English BERT on English (baseline) | 84.58% | 0.7928 |
| English BERT → Maithili (zero-shot) | 33.33% | 0.1667 |
| XLM-RoBERTa → Maithili (zero-shot) | 68.66% | 0.6787 |
| mBERT fine-tuned on Maithili | 70.86% | 0.7100 |
| **XLM-RoBERTa fine-tuned on Maithili (this model)** | **82.44%** | **0.8246** |

**Key finding:** English BERT collapses from 84.58% on English to 33.33% on Maithili
— a 51.25% drop — confirming the severity of the resource gap. Fine-tuning
XLM-RoBERTa recovers performance to 82.44%, with OOD accuracy nearly identical
at 82.00%, indicating genuine generalisation rather than memorisation.

---

## ⚡ Quick Start

```python
from transformers import pipeline

classifier = pipeline(
    "text-classification",
    model="abhiprd20/maithili-sentiment-model"
)

result = classifier("ई पोथी बहुत नीक ऐछ।")
print(result)
# → [{'label': 'positive', 'score': 0.94}]
```

---

## 🔍 More Examples

```python
from transformers import pipeline

classifier = pipeline(
    "text-classification",
    model="abhiprd20/maithili-sentiment-model"
)

texts = [
    "अहाँक काज बहुत सुन्दर ऐछ।",       # Your work is very beautiful.
    "हमरा ई खाना नीक नहि लागल।",        # I did not like this food.
    "हम काल्हि पटना जाइब।",              # I will go to Patna tomorrow.
    "परीक्षा मे हमरा नीक अंक भेटल।",    # I got good marks in the exam.
    "दोकानदार हमरा ठगि लेलक।",           # The shopkeeper cheated me.
]

for text in texts:
    result = classifier(text)[0]
    print(f"Text  : {text}")
    print(f"Label : {result['label']} ({round(result['score']*100, 1)}% confident)\n")
```

---

## 🧪 Use With AutoTokenizer and AutoModel

```python
from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

tokenizer = AutoTokenizer.from_pretrained("abhiprd20/maithili-sentiment-model")
model     = AutoModelForSequenceClassification.from_pretrained("abhiprd20/maithili-sentiment-model")

text   = "गीत सुनि कऽ मोन खुस भऽ गेल।"  # Listening to the song made my heart happy.
inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=128)

with torch.no_grad():
    outputs = model(**inputs)

probs    = torch.softmax(outputs.logits, dim=1)
label_id = torch.argmax(probs).item()
id2label = {0: "negative", 1: "neutral", 2: "positive"}

print(f"Label      : {id2label[label_id]}")
print(f"Confidence : {probs[0][label_id].item():.4f}")
```

---

## 🗂️ Training Details

| Parameter | Value |
|-----------|-------|
| Base model | `cardiffnlp/twitter-xlm-roberta-base-sentiment` |
| Task | Sequence Classification |
| Number of labels | 3 (negative, neutral, positive) |
| Epochs | 3 |
| Batch size | 16 |
| Max sequence length | 128 |
| Training samples | 3,561 (1,187 per class) |
| Test samples | 501 (167 per class) |
| Optimizer | AdamW (default) |
| Hardware | NVIDIA T4 GPU (Google Colab) |
| Framework | Hugging Face Transformers |

---

## 📦 Training Dataset

This model was trained on the
[Maithili Sentiment Dataset](https://huggingface.co/datasets/abhiprd20/Maithili_Sentiment_8K),
a 55,000-sentence corpus of Maithili text in Devanagari script with 3-class sentiment labels.

Training used **3,561 verified rows** (1,187 per class), balanced across all three
sentiment classes. The full dataset produces artificially high in-distribution accuracy
due to translation-pipeline homogeneity — a limitation documented in the paper.
The 3,561-row configuration produces honest, generalisable results as confirmed by
the near-identical OOD accuracy (82.00%).

---

## 🏷️ Label Mapping

| Label ID | Label | Meaning |
|----------|-------|---------|
| 0 | negative | Dissatisfaction, criticism, distress, anger |
| 1 | neutral | Factual, descriptive, balanced, indifferent |
| 2 | positive | Satisfaction, happiness, praise, appreciation |


---

## 🔗 Related Resources

| Resource | Link |
|----------|------|
| Training dataset | [abhiprd20/Maithili_Sentiment_8K](https://huggingface.co/datasets/abhiprd20/Maithili_Sentiment_8K) |
| Bhojpuri sentiment model | [abhiprd20/bhojpuri-sentiment-model](https://huggingface.co/abhiprd20/bhojpuri-sentiment-model) |
| Hindi sentiment model | [abhiprd20/hindi-sentiment-model](https://huggingface.co/abhiprd20/hindi-sentiment-model) |
| English baseline model | [abhiprd20/nlp-sentiment-model](https://huggingface.co/abhiprd20/nlp-sentiment-model) |

---

## ⚖️ License

This model is released under the **Apache License 2.0** — free for both
research and commercial use.

Copyright 2026 Abhimanyu Prasad

---

## 📎 Citation

If you use this model in your research or project, please cite:

```bibtex
@misc{prasad2025maithilisentiment,
  title        = {Maithili Sentiment Model: XLM-RoBERTa Fine-tuned for Low-Resource Sentiment Analysis},
  author       = {Prasad, Abhimanyu},
  year         = {2026},
  publisher    = {Hugging Face},
  howpublished = {\url{https://huggingface.co/abhiprd20/maithili-sentiment-model}},
  note         = {Accuracy: 82.44\%, F1: 0.8246, OOD Accuracy: 82.00\%.
                  Part of cross-lingual transfer study on low-resource Bihari languages.}
}
```

---

## 👤 Author

**Abhimanyu Prasad**
🤗 Hugging Face: [abhiprd20](https://huggingface.co/abhiprd20)
📦 Dataset: [abhiprd20/Maithili_Sentiment_8K](https://huggingface.co/datasets/abhiprd20/Maithili_Sentiment_8K)

---

*If this model helped your research, consider giving it a ⭐ — it helps others working on low-resource Indic NLP find it too!*

---

## 📊 Cross-Language Evaluation

Each model was evaluated on all 4 languages (300 sentences per language, 100 per class).
This shows how well models trained on one language transfer to others.

### Accuracy Matrix

| Model | English | Hindi | Maithili | Bhojpuri |
|---|---|---|---|---|
| **English model** | **79.5%** ✓ | 34.0% | 33.3% | 33.0% |
| **Hindi model** | 60.0% | **68.0%** ✓ | 63.3% | 61.7% |
| ⭐ **Maithili model** _(this model)_ | 63.0% | 59.0% | **90.3%** ✓ | 75.0% |
| **Bhojpuri model** | 59.0% | 47.3% | 47.3% | **98.0%** ✓ |

### F1 Matrix (macro)

| Model | English | Hindi | Maithili | Bhojpuri |
|---|---|---|---|---|
| **English model** | **0.5424** ✓ | 0.1912 | 0.1667 | 0.1654 |
| **Hindi model** | 0.4362 | **0.6778** ✓ | 0.6319 | 0.6042 |
| ⭐ **Maithili model** _(this model)_ | 0.4443 | 0.5757 | **0.9035** ✓ | 0.7458 |
| **Bhojpuri model** | 0.4250 | 0.4166 | 0.4114 | **0.9801** ✓ |

### Key Findings

- Strong transfer to Bhojpuri (**75%**), a related Bihari language, significantly outperforming both English (33%) and Hindi (61.7%) on Bhojpuri.
- Suggests **Bihari language family** similarity enables strong cross-lingual transfer.
- Maithili → Bhojpuri transfer (75%) is asymmetric: Bhojpuri → Maithili is only 47.3%.

> **Full paper:** This cross-evaluation is part of a research study on cross-lingual transfer for low-resource Bihari languages. See the companion datasets and models: [Maithili](https://huggingface.co/abhiprd20/maithili-sentiment-model) | [Bhojpuri](https://huggingface.co/abhiprd20/bhojpuri-sentiment-model) | [Hindi](https://huggingface.co/abhiprd20/hindi-sentiment-model) | [English](https://huggingface.co/abhiprd20/nlp-sentiment-model)