license: mit
language:
- multilingual
library_name: transformers
pipeline_tag: text-classification
widget:
- text: You wont believe what happened to me today
- text: You wont believe what happened to me today!
- text: You wont believe what happened to me today...
- text: You wont believe what happened to me today <3
- text: You wont believe what happened to me today :)
- text: You wont believe what happened to me today :(
This is an emotion classification model based on finetuning of a Bernice model (a multilingual pre-trained model trained on multilingual Twitter data) on self-labeled emotion dataset (Lykousas et al., 2019) in English that corresponds to Anger, Fear, Sadness, Joy, and Affection. See the paper, LEIA: Linguistic Embeddings for the Identification of Affect for further details.
Evaluation
We evaluated LEIA-multilingual on Vent posts with self-annotated emotion labels that was identified (using an ensemble of language identefication tools) to be non-English. See the below for the macro-F1 scores across emotion categories and languages:
language | Macro-F1 ar | 44.18[43.07,45.29] da |65.44[60.96,69.83] de |60.47[57.58,63.38] es |61.67[60.79,62.55] fi |45.1[40.96,49.14] fr |65.78[63.19,68.36] it |63.37[59.67,67.1] pt |57.27[55.15,59.4] tl |58.37[55.51,61.23] tr |45.42[41.17,49.79]
Citation
Please cite the following paper if you find the model useful for your work:
@article{aroyehun2023leia,
title={LEIA: Linguistic Embeddings for the Identification of Affect},
author={Aroyehun, Segun Taofeek and Malik, Lukas and Metzler, Hannah and Haimerl, Nikolas and Di Natale, Anna and Garcia, David},
journal={EPJ Data Science},
volume={12},
year={2023},
publisher={Springer}
}