samarthruckstar/emotion_label_model

Emotion Label Model (RoBERTa fine-tuned on GoEmotions)

This is a fine-tuned roberta-base model for multi-label emotion classification. It was trained on the GoEmotions dataset to detect 28 distinct emotional categories in text.

Model Details

• Model Type: Multi-label Text Classification
• Base Model: roberta-base
• Language: English
• Number of Labels: 28

Emotion Labels

The model can predict the following 28 emotions: admiration, amusement, anger, annoyance, approval, caring, confusion, curiosity, desire, disappointment, disapproval, disgust, embarrassment, excitement, fear, gratitude, grief, joy, love, nervousness, optimism, pride, realization, relief, remorse, sadness, surprise, neutral.

Example Code

from transformers import pipeline

Initialize the pipeline (using top_k=None to get all scores)

emotion_classifier = pipeline("text-classification", model="samarthrawat18/emotion_label_model", top_k=None) text = "I love this movie! It's absolutely amazing and I'm so happy!" results = emotion_classifier(text)

Filter out predictions below a certain threshold (e.g., 0.5)

threshold = 0.5 predicted_emotions = {res['label']: res['score'] for res in results[0] if res['score'] > threshold} print(predicted_emotions)

Expected Output format: {'joy': 0.89, 'admiration': 0.75}

Training Details

Training Dataset The model was trained on a processed version of the GoEmotions dataset, which consists of Reddit comments labeled with emotions plus a neutral category.

Training Configuration The model was fine-tuned using the following hyperparameters:

• Optimizer: AdamW
• Loss Function: BCEWithLogitsLoss (Binary Cross Entropy for multi-label)
• Learning Rate: 2e-05 (or 3e-05)
• Batch Size: 24 (per device)
• Number of Epochs: 2
• Weight Decay: 0.01
• Warmup Steps: 100
• Max Sequence Length: 128
• metrics: - name: Micro F1-Score type: f1-micro value: 58.34 - name: Macro F1-Score type: f1-macro value: 43.57 - name: Precision (Micro) type: precision-micro value: 70.00 - name: Recall (Micro) type: recall-micro value: 50.00 - name: Gratitude F1-Score type: f1-gratitude value: 91.00 - name: Amusement F1-Score type: f1-amusement value: 82.00 - name: Love F1-Score type: f1-love value: 82.00 - name: Admiration F1-Score type: f1-admiration value: 71.00

The model shows a Micro F1-Score of 0.58 and performs exceptionally well on emotions like Gratitude (0.91 F1), Amusement (0.82 F1), and Love (0.82 F1). It struggles with rare classes such as Grief and Pride, where the test set has very few samples.

Evaluation

Evaluation Summary

• Micro F1-Score: 0.5834
• Macro F1-Score: 0.4357
• Micro Precision: 0.70
• Micro Recall: 0.50

Key Findings

• Top Performing Emotions:
  • Gratitude: F1-score 0.91 (Precision: 0.93, Recall: 0.90)
  • Amusement: F1-score 0.82 (Precision: 0.78, Recall: 0.86)
  • Love: F1-score 0.82 (Precision: 0.81, Recall: 0.84)
  • Admiration: F1-score 0.71 (Precision: 0.69, Recall: 0.73)
• Low Performing Emotions:
  • Grief, Nervousness, Pride, Relief: F1-score 0.00 (Likely due to very low support in the test set: 7, 25, 18, and samples respectively).
  • Disappointment: F1-score 0.12 (Low recall of 0.06).
• Neutral Class: F1-score 0.62 (High support: 1,787 samples).

Full Classification Report

            precision    recall  f1-score   support
admiration       0.69      0.73      0.71       504
 amusement       0.78      0.86      0.82       264
     anger       0.69      0.56      0.61       248
 annoyance       0.61      0.25      0.36       360
  approval       0.54      0.30      0.38       351
    caring       0.59      0.39      0.47       135
 confusion       0.53      0.29      0.38       157
 curiosity       0.53      0.42      0.47       284
    desire       0.73      0.34      0.47        88

disappointment 0.53 0.06 0.12 154 disapproval 0.49 0.32 0.39 268 disgust 0.70 0.31 0.43 123 embarrassment 0.60 0.08 0.14 37 excitement 0.68 0.31 0.43 109 fear 0.68 0.64 0.66 80 gratitude 0.93 0.90 0.91 352 grief 0.00 0.00 0.00 7 joy 0.72 0.57 0.64 164 love 0.81 0.84 0.82 243 nervousness 0.00 0.00 0.00 25 optimism 0.67 0.41 0.51 193 pride 0.00 0.00 0.00 18 realization 0.75 0.06 0.11 145 relief 0.00 0.00 0.00 17 remorse 0.61 0.68 0.64 59 sadness 0.70 0.48 0.57 173 surprise 0.61 0.49 0.54 143 neutral 0.74 0.53 0.62 1787 micro avg 0.70 0.50 0.58 6488 macro avg 0.57 0.39 0.44 6488 weighted avg 0.68 0.50 0.56 6488 samples avg 0.56 0.52 0.53 6488