SkyWater21/lvbert-lv-go-emotions

Fine-tuned LVBERT for multi-label emotion classification task.

Model was trained on lv_go_emotions dataset. This dataset is Latvian translation of GoEmotions dataset. Google Translate was used to generate the machine translation.

Labels:

0: admiration
1: amusement
2: anger
3: annoyance
4: approval
5: caring
6: confusion
7: curiosity
8: desire
9: disappointment
10: disapproval
11: disgust
12: embarrassment
13: excitement
14: fear
15: gratitude
16: grief
17: joy
18: love
19: nervousness
20: optimism
21: pride
22: realization
23: relief
24: remorse
25: sadness
26: surprise
27: neutral

Seed used for random number generator is 42:

def set_seed(seed=42):
    random.seed(seed)
    np.random.seed(seed)
    torch.manual_seed(seed)
    if torch.cuda.is_available():
        torch.cuda.manual_seed_all(seed)

Training parameters:

max_length: null
batch_size: 32
shuffle: True
num_workers: 2
pin_memory: False
drop_last: False

optimizer: adam
lr: 0.00001
weight_decay: 0

problem_type: multi_label_classification

num_epochs: 5

Evaluation results on test split of lv_go_emotions

	Precision	Recall	F1-Score	AUC-ROC	Support
admiration	0.64	0.64	0.64	0.92	504
amusement	0.76	0.85	0.80	0.96	264
anger	0.51	0.21	0.29	0.86	198
annoyance	0.49	0.15	0.23	0.78	320
approval	0.35	0.33	0.34	0.80	351
caring	0.43	0.39	0.41	0.89	135
confusion	0.53	0.33	0.41	0.94	153
curiosity	0.49	0.42	0.45	0.94	284
desire	0.63	0.37	0.47	0.92	83
disappointment	0.45	0.11	0.18	0.82	151
disapproval	0.45	0.25	0.32	0.84	267
disgust	0.63	0.29	0.40	0.92	123
embarrassment	0.50	0.14	0.21	0.85	37
excitement	0.55	0.16	0.24	0.89	103
fear	0.65	0.58	0.61	0.95	78
gratitude	0.88	0.91	0.90	0.99	352
grief	0.00	0.00	0.00	0.78	6
joy	0.61	0.39	0.47	0.93	161
love	0.80	0.69	0.74	0.97	238
nervousness	0.00	0.00	0.00	0.95	23
optimism	0.57	0.47	0.52	0.90	186
pride	0.00	0.00	0.00	0.73	16
realization	0.29	0.08	0.13	0.76	145
relief	0.00	0.00	0.00	0.85	11
remorse	0.54	0.68	0.60	0.98	56
sadness	0.60	0.50	0.54	0.93	156
surprise	0.65	0.41	0.50	0.92	141
neutral	0.67	0.50	0.57	0.81	1787
micro avg	0.62	0.46	0.53	0.93	6329
macro avg	0.49	0.35	0.39	0.88	6329
weighted avg	0.60	0.46	0.51	0.87	6329
samples avg	0.52	0.48	0.49	nan	6329