Add cross-language evaluation matrix (4-model × 4-language study)
Browse files
README.md
CHANGED
|
@@ -238,4 +238,37 @@ If you use this model in your research or project, please cite:
|
|
| 238 |
|
| 239 |
---
|
| 240 |
|
| 241 |
-
*If this model helped your project, consider giving it a ⭐ — it helps others find it too!*
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 238 |
|
| 239 |
---
|
| 240 |
|
| 241 |
+
*If this model helped your project, consider giving it a ⭐ — it helps others find it too!*
|
| 242 |
+
|
| 243 |
+
---
|
| 244 |
+
|
| 245 |
+
## 📊 Cross-Language Evaluation
|
| 246 |
+
|
| 247 |
+
Each model was evaluated on all 4 languages (300 sentences per language, 100 per class).
|
| 248 |
+
This shows how well models trained on one language transfer to others.
|
| 249 |
+
|
| 250 |
+
### Accuracy Matrix
|
| 251 |
+
|
| 252 |
+
| Model | English | Hindi | Maithili | Bhojpuri |
|
| 253 |
+
|---|---|---|---|---|
|
| 254 |
+
| ⭐ **English model** _(this model)_ | **79.5%** ✓ | 34.0% | 33.3% | 33.0% |
|
| 255 |
+
| **Hindi model** | 60.0% | **68.0%** ✓ | 63.3% | 61.7% |
|
| 256 |
+
| **Maithili model** | 63.0% | 59.0% | **90.3%** ✓ | 75.0% |
|
| 257 |
+
| **Bhojpuri model** | 59.0% | 47.3% | 47.3% | **98.0%** ✓ |
|
| 258 |
+
|
| 259 |
+
### F1 Matrix (macro)
|
| 260 |
+
|
| 261 |
+
| Model | English | Hindi | Maithili | Bhojpuri |
|
| 262 |
+
|---|---|---|---|---|
|
| 263 |
+
| ⭐ **English model** _(this model)_ | **0.5424** ✓ | 0.1912 | 0.1667 | 0.1654 |
|
| 264 |
+
| **Hindi model** | 0.4362 | **0.6778** ✓ | 0.6319 | 0.6042 |
|
| 265 |
+
| **Maithili model** | 0.4443 | 0.5757 | **0.9035** ✓ | 0.7458 |
|
| 266 |
+
| **Bhojpuri model** | 0.4250 | 0.4166 | 0.4114 | **0.9801** ✓ |
|
| 267 |
+
|
| 268 |
+
### Key Findings
|
| 269 |
+
|
| 270 |
+
- This model achieves **79.5%** on English sentiment but drops to **~33%** on Maithili and Bhojpuri — equivalent to random chance on a 3-class task.
|
| 271 |
+
- Demonstrates that monolingual English training does **not** transfer to low-resource Bihari languages.
|
| 272 |
+
- English also transfers poorly to Hindi (34%), confirming the language barrier extends beyond Bihari languages.
|
| 273 |
+
|
| 274 |
+
> **Full paper:** This cross-evaluation is part of a research study on cross-lingual transfer for low-resource Bihari languages. See the companion datasets and models: [Maithili](https://huggingface.co/abhiprd20/maithili-sentiment-model) | [Bhojpuri](https://huggingface.co/abhiprd20/bhojpuri-sentiment-model) | [Hindi](https://huggingface.co/abhiprd20/hindi-sentiment-model) | [English](https://huggingface.co/abhiprd20/nlp-sentiment-model)
|