abhiprd20 commited on
Commit
ea5c0ab
·
verified ·
1 Parent(s): 75ee46c

Add cross-language evaluation matrix (4-model × 4-language study)

Browse files
Files changed (1) hide show
  1. README.md +34 -1
README.md CHANGED
@@ -238,4 +238,37 @@ If you use this model in your research or project, please cite:
238
 
239
  ---
240
 
241
- *If this model helped your project, consider giving it a ⭐ — it helps others find it too!*
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
238
 
239
  ---
240
 
241
+ *If this model helped your project, consider giving it a ⭐ — it helps others find it too!*
242
+
243
+ ---
244
+
245
+ ## 📊 Cross-Language Evaluation
246
+
247
+ Each model was evaluated on all 4 languages (300 sentences per language, 100 per class).
248
+ This shows how well models trained on one language transfer to others.
249
+
250
+ ### Accuracy Matrix
251
+
252
+ | Model | English | Hindi | Maithili | Bhojpuri |
253
+ |---|---|---|---|---|
254
+ | ⭐ **English model** _(this model)_ | **79.5%** ✓ | 34.0% | 33.3% | 33.0% |
255
+ | **Hindi model** | 60.0% | **68.0%** ✓ | 63.3% | 61.7% |
256
+ | **Maithili model** | 63.0% | 59.0% | **90.3%** ✓ | 75.0% |
257
+ | **Bhojpuri model** | 59.0% | 47.3% | 47.3% | **98.0%** ✓ |
258
+
259
+ ### F1 Matrix (macro)
260
+
261
+ | Model | English | Hindi | Maithili | Bhojpuri |
262
+ |---|---|---|---|---|
263
+ | ⭐ **English model** _(this model)_ | **0.5424** ✓ | 0.1912 | 0.1667 | 0.1654 |
264
+ | **Hindi model** | 0.4362 | **0.6778** ✓ | 0.6319 | 0.6042 |
265
+ | **Maithili model** | 0.4443 | 0.5757 | **0.9035** ✓ | 0.7458 |
266
+ | **Bhojpuri model** | 0.4250 | 0.4166 | 0.4114 | **0.9801** ✓ |
267
+
268
+ ### Key Findings
269
+
270
+ - This model achieves **79.5%** on English sentiment but drops to **~33%** on Maithili and Bhojpuri — equivalent to random chance on a 3-class task.
271
+ - Demonstrates that monolingual English training does **not** transfer to low-resource Bihari languages.
272
+ - English also transfers poorly to Hindi (34%), confirming the language barrier extends beyond Bihari languages.
273
+
274
+ > **Full paper:** This cross-evaluation is part of a research study on cross-lingual transfer for low-resource Bihari languages. See the companion datasets and models: [Maithili](https://huggingface.co/abhiprd20/maithili-sentiment-model) | [Bhojpuri](https://huggingface.co/abhiprd20/bhojpuri-sentiment-model) | [Hindi](https://huggingface.co/abhiprd20/hindi-sentiment-model) | [English](https://huggingface.co/abhiprd20/nlp-sentiment-model)