Update README.md
Browse files
README.md
CHANGED
|
@@ -12,24 +12,28 @@ widget:
|
|
| 12 |
- text: "You wont believe what happened to me today :)"
|
| 13 |
- text: "You wont believe what happened to me today :("
|
| 14 |
---
|
| 15 |
-
This is an emotion classification model based on
|
| 16 |
See the paper, [LEIA: Linguistic Embeddings for the Identification of Affect](https://arxiv.org/abs/2304.10973) for further details.
|
| 17 |
|
| 18 |
## Evaluation
|
| 19 |
-
We evaluated LEIA-multilingual on Vent posts with self-annotated emotion labels
|
| 20 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 21 |
|
| 22 |
-
language | Macro-F1
|
| 23 |
-
ar | 44.18[43.07,45.29]
|
| 24 |
-
da |65.44[60.96,69.83]
|
| 25 |
-
de |60.47[57.58,63.38]
|
| 26 |
-
es |61.67[60.79,62.55]
|
| 27 |
-
fi |45.1[40.96,49.14]
|
| 28 |
-
fr |65.78[63.19,68.36]
|
| 29 |
-
it |63.37[59.67,67.1]
|
| 30 |
-
pt |57.27[55.15,59.4]
|
| 31 |
-
tl |58.37[55.51,61.23]
|
| 32 |
-
tr |45.42[41.17,49.79]
|
| 33 |
|
| 34 |
|
| 35 |
## Citation
|
|
|
|
| 12 |
- text: "You wont believe what happened to me today :)"
|
| 13 |
- text: "You wont believe what happened to me today :("
|
| 14 |
---
|
| 15 |
+
This is an emotion classification model based on fine-tuning of a Bernice model (which is a multilingual pre-trained model trained on multilingual Twitter data) on self-labeled emotion dataset (Lykousas et al., 2019) in English that corresponds to Anger, Fear, Sadness, Joy, and Affection.
|
| 16 |
See the paper, [LEIA: Linguistic Embeddings for the Identification of Affect](https://arxiv.org/abs/2304.10973) for further details.
|
| 17 |
|
| 18 |
## Evaluation
|
| 19 |
+
We evaluated LEIA-multilingual on Vent posts with self-annotated emotion labels identified as non-English using an ensemble of language identefication tools.
|
| 20 |
+
The table below shows the macro-F1 scores across emotion categories and languages:
|
| 21 |
+
|
| 22 |
+
|
| 23 |
+
|
| 24 |
+
|Language|Macro-F1|
|
| 25 |
+
|:---:|:---:|
|
| 26 |
+
|ar |44.18[43.07,45.29]|
|
| 27 |
+
|da |65.44[60.96,69.83] |
|
| 28 |
+
|de |60.47[57.58,63.38] |
|
| 29 |
+
|es |61.67[60.79,62.55] |
|
| 30 |
+
|fi |45.1[40.96,49.14] |
|
| 31 |
+
|fr |65.78[63.19,68.36] |
|
| 32 |
+
|it |63.37[59.67,67.1] |
|
| 33 |
+
|pt |57.27[55.15,59.4] |
|
| 34 |
+
|tl |58.37[55.51,61.23] |
|
| 35 |
+
|tr |45.42[41.17,49.79]|
|
| 36 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 37 |
|
| 38 |
|
| 39 |
## Citation
|