ABHIMANYU PRASAD commited on
Commit
d61b0cd
·
verified ·
1 Parent(s): df35a78

Add cross-language evaluation matrix (4-model × 4-language study)

Browse files
Files changed (1) hide show
  1. README.md +34 -1
README.md CHANGED
@@ -173,4 +173,37 @@ If you use this model, please cite:
173
  publisher = {HuggingFace},
174
  url = {https://huggingface.co/abhiprd20/hindi-sentiment-model}
175
  }
176
- ```
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
173
  publisher = {HuggingFace},
174
  url = {https://huggingface.co/abhiprd20/hindi-sentiment-model}
175
  }
176
+ ```
177
+
178
+ ---
179
+
180
+ ## 📊 Cross-Language Evaluation
181
+
182
+ Each model was evaluated on all 4 languages (300 sentences per language, 100 per class).
183
+ This shows how well models trained on one language transfer to others.
184
+
185
+ ### Accuracy Matrix
186
+
187
+ | Model | English | Hindi | Maithili | Bhojpuri |
188
+ |---|---|---|---|---|
189
+ | **English model** | **79.5%** ✓ | 34.0% | 33.3% | 33.0% |
190
+ | ⭐ **Hindi model** _(this model)_ | 60.0% | **68.0%** ✓ | 63.3% | 61.7% |
191
+ | **Maithili model** | 63.0% | 59.0% | **90.3%** ✓ | 75.0% |
192
+ | **Bhojpuri model** | 59.0% | 47.3% | 47.3% | **98.0%** ✓ |
193
+
194
+ ### F1 Matrix (macro)
195
+
196
+ | Model | English | Hindi | Maithili | Bhojpuri |
197
+ |---|---|---|---|---|
198
+ | **English model** | **0.5424** ✓ | 0.1912 | 0.1667 | 0.1654 |
199
+ | ⭐ **Hindi model** _(this model)_ | 0.4362 | **0.6778** ✓ | 0.6319 | 0.6042 |
200
+ | **Maithili model** | 0.4443 | 0.5757 | **0.9035** ✓ | 0.7458 |
201
+ | **Bhojpuri model** | 0.4250 | 0.4166 | 0.4114 | **0.9801** ✓ |
202
+
203
+ ### Key Findings
204
+
205
+ - Hindi transfers significantly better than English to both Maithili (**63.3%**) and Bhojpuri (**61.7%**), nearly doubling English performance.
206
+ - Supports the hypothesis that **linguistic proximity** (Hindi → Bihari languages) aids cross-lingual transfer.
207
+ - Hindi model performs reasonably on English (60%), suggesting partial bidirectional transfer.
208
+
209
+ > **Full paper:** This cross-evaluation is part of a research study on cross-lingual transfer for low-resource Bihari languages. See the companion datasets and models: [Maithili](https://huggingface.co/abhiprd20/maithili-sentiment-model) | [Bhojpuri](https://huggingface.co/abhiprd20/bhojpuri-sentiment-model) | [Hindi](https://huggingface.co/abhiprd20/hindi-sentiment-model) | [English](https://huggingface.co/abhiprd20/nlp-sentiment-model)