| | --- |
| | library_name: transformers |
| | tags: |
| | - emotion |
| | - text-classification |
| | - pytorch |
| | license: apache-2.0 |
| | datasets: |
| | - dair-ai/emotion |
| | language: |
| | - en |
| | metrics: |
| | - accuracy |
| | - f1 |
| | pipeline_tag: text-classification |
| | --- |
| | |
| | # RoBERTa for Emotion Classification |
| | ## Model Description |
| |
|
| | This model is a fine-tuned version of `RoBERTaForSequenceClassification` trained to classify text into six emotion categories: Sadness, Joy, Love, Anger, Fear, and Surprise. |
| | - [RoBERTa](https://huggingface.co/docs/transformers/v4.41.3/en/model_doc/roberta#transformers.RobertaForSequenceClassification) |
| | - Special thanks to [bhadresh-savani](https://huggingface.co/bhadresh-savani/roberta-base-emotion), whose notebook was the main guide for this work. |
| |
|
| | ## Intended Use |
| |
|
| | The model is intended for classifying emotions in text data. It can be used in applications involving sentiment analysis, chatbots, social media monitoring, diary entries. |
| |
|
| | ### Limitations |
| |
|
| | - The model is trained on a specific emotion dataset and may not generalize well to other datasets or domains. |
| | - It might not perform well on text with mixed or ambiguous emotions. |
| |
|
| | ## How to use the model |
| | ```python |
| | from transformers import pipeline |
| | classifier = pipeline(model="Dimi-G/roberta-base-emotion") |
| | emotions=classifier("i feel very happy and excited since i learned so many things", top_k=None) |
| | print(emotions) |
| | |
| | """ |
| | Output: |
| | [{'label': 'Joy', 'score': 0.9991986155509949}, |
| | {'label': 'Love', 'score': 0.0003064649645239115}, |
| | {'label': 'Sadness', 'score': 0.0001680034474702552}, |
| | {'label': 'Anger', 'score': 0.00012623333896044642}, |
| | {'label': 'Surprise', 'score': 0.00011396403715480119}, |
| | {'label': 'Fear', 'score': 8.671794785186648e-05}] |
| | """ |
| | ``` |
| |
|
| | ## Training Details |
| |
|
| | The model was trained on a randomized subset of the [dar-ai/emotion](https://huggingface.co/datasets/dair-ai/emotion) dataset from the Hugging Face datasets library. Here are the training parameters: |
| |
|
| | - **Batch size**: 64 |
| | - **Number of epochs**: 10 |
| | - **Learning rate**: 5e-5 |
| | - **Warmup steps**: 500 |
| | - **Weight decay**: 0.03 |
| | - **Evaluation strategy**: epoch |
| | - **Save strategy**: epoch |
| | - **Metric for best model**: F1 score |
| |
|
| | ## Evaluation |
| | ```python |
| | {'eval_loss': 0.18195335566997528, |
| | 'eval_accuracy': 0.94, |
| | 'eval_f1': 0.9396676959491667, |
| | 'eval_runtime': 1.1646, |
| | 'eval_samples_per_second': 858.685, |
| | 'eval_steps_per_second': 13.739, |
| | 'epoch': 10.0} |
| | ``` |
| |
|
| | ## Model Resources |
| | Link to the notebook with details on fine-tuning the model and our approach with other models for emotion classification: |
| | - **Repository:** [Beginners Guide to Emotion Classification](https://github.com/Dimi-G/Capstone_Project/blob/main/Beginners_guide_to_emotion_classification.ipynb) |
| |
|
| |
|
| | ## Environmental Impact |
| |
|
| | <!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly --> |
| |
|
| | Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700). |
| |
|
| |
|
| | ## Citation |
| | - [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://huggingface.co/papers/1907.11692) |
| |
|
| |
|
