bert-base-arabic-finetuned-emotion

This model is a fine-tuned version of asafaya/bert-base-arabic on the emotone_ar dataset. It achieves the following results on the evaluation set:

• Loss: 0.8965
• Accuracy: 0.7416
• F1: 0.7406

Cite this model

-Noaman, H. (2023). Improved Emotion Detection Framework for Arabic Text using Transformer Models.
Advanced Engineering Technology and Application, 12(2), 1-11.

@article{noaman2023improved,
  title={Improved Emotion Detection Framework for Arabic Text using Transformer Models},
  author={Noaman, Hatem},
  journal={Advanced Engineering Technology and Application},
  volume={12},
  number={2},
  pages={1--11},
  year={2023},
  publisher={Fayoum University}
}

Load Pretrained Model

You can use this model by

from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("hatemnoaman/bert-base-arabic-finetuned-emotion")
model = AutoModel.from_pretrained("hatemnoaman/bert-base-arabic-finetuned-emotion")

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 64
• eval_batch_size: 64
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 6

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1
1.3476	1.0	142	0.8911	0.7008	0.6812
0.8204	2.0	284	0.8175	0.7276	0.7212
0.6227	3.0	426	0.8392	0.7376	0.7302
0.4816	4.0	568	0.8531	0.7435	0.7404
0.378	5.0	710	0.8817	0.7396	0.7388
0.3134	6.0	852	0.8965	0.7416	0.7406

Framework versions

• Transformers 4.25.1
• Pytorch 1.13.0+cu116
• Datasets 2.8.0
• Tokenizers 0.13.2