roberta-base-finetuned-emotion

This model is a fine-tuned version of roberta-base on the emotion dataset. It achieves the following results on the evaluation set:

• Loss: 0.1669
• Accuracy: 0.94
• F1: 0.9404

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

The model has trained twice. In the first run hyperparameters was the same except num_epochs was 3. So results below actually shows 8 epoch of fine-tuning in total.

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 64
• eval_batch_size: 64
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 5

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1
0.1508	1.0	250	0.1969	0.934	0.9334
0.1035	2.0	500	0.1660	0.9335	0.9341
0.0926	3.0	750	0.1626	0.935	0.9359
0.0855	4.0	1000	0.1680	0.934	0.9337
0.0682	5.0	1250	0.1669	0.94	0.9404

Framework versions

• Transformers 4.13.0
• Pytorch 1.13.1+cu116
• Datasets 2.8.0
• Tokenizers 0.10.3