ABHIMANYU PRASAD commited on
Add cross-language evaluation matrix (4-model × 4-language study)
Browse files
README.md
CHANGED
|
@@ -173,4 +173,37 @@ If you use this model, please cite:
|
|
| 173 |
publisher = {HuggingFace},
|
| 174 |
url = {https://huggingface.co/abhiprd20/hindi-sentiment-model}
|
| 175 |
}
|
| 176 |
-
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 173 |
publisher = {HuggingFace},
|
| 174 |
url = {https://huggingface.co/abhiprd20/hindi-sentiment-model}
|
| 175 |
}
|
| 176 |
+
```
|
| 177 |
+
|
| 178 |
+
---
|
| 179 |
+
|
| 180 |
+
## 📊 Cross-Language Evaluation
|
| 181 |
+
|
| 182 |
+
Each model was evaluated on all 4 languages (300 sentences per language, 100 per class).
|
| 183 |
+
This shows how well models trained on one language transfer to others.
|
| 184 |
+
|
| 185 |
+
### Accuracy Matrix
|
| 186 |
+
|
| 187 |
+
| Model | English | Hindi | Maithili | Bhojpuri |
|
| 188 |
+
|---|---|---|---|---|
|
| 189 |
+
| **English model** | **79.5%** ✓ | 34.0% | 33.3% | 33.0% |
|
| 190 |
+
| ⭐ **Hindi model** _(this model)_ | 60.0% | **68.0%** ✓ | 63.3% | 61.7% |
|
| 191 |
+
| **Maithili model** | 63.0% | 59.0% | **90.3%** ✓ | 75.0% |
|
| 192 |
+
| **Bhojpuri model** | 59.0% | 47.3% | 47.3% | **98.0%** ✓ |
|
| 193 |
+
|
| 194 |
+
### F1 Matrix (macro)
|
| 195 |
+
|
| 196 |
+
| Model | English | Hindi | Maithili | Bhojpuri |
|
| 197 |
+
|---|---|---|---|---|
|
| 198 |
+
| **English model** | **0.5424** ✓ | 0.1912 | 0.1667 | 0.1654 |
|
| 199 |
+
| ⭐ **Hindi model** _(this model)_ | 0.4362 | **0.6778** ✓ | 0.6319 | 0.6042 |
|
| 200 |
+
| **Maithili model** | 0.4443 | 0.5757 | **0.9035** ✓ | 0.7458 |
|
| 201 |
+
| **Bhojpuri model** | 0.4250 | 0.4166 | 0.4114 | **0.9801** ✓ |
|
| 202 |
+
|
| 203 |
+
### Key Findings
|
| 204 |
+
|
| 205 |
+
- Hindi transfers significantly better than English to both Maithili (**63.3%**) and Bhojpuri (**61.7%**), nearly doubling English performance.
|
| 206 |
+
- Supports the hypothesis that **linguistic proximity** (Hindi → Bihari languages) aids cross-lingual transfer.
|
| 207 |
+
- Hindi model performs reasonably on English (60%), suggesting partial bidirectional transfer.
|
| 208 |
+
|
| 209 |
+
> **Full paper:** This cross-evaluation is part of a research study on cross-lingual transfer for low-resource Bihari languages. See the companion datasets and models: [Maithili](https://huggingface.co/abhiprd20/maithili-sentiment-model) | [Bhojpuri](https://huggingface.co/abhiprd20/bhojpuri-sentiment-model) | [Hindi](https://huggingface.co/abhiprd20/hindi-sentiment-model) | [English](https://huggingface.co/abhiprd20/nlp-sentiment-model)
|