| | --- |
| | language: en |
| | license: mit |
| | pipeline_tag: text-classification |
| | tags: |
| | - text-classification |
| | - transformers |
| | - pytorch |
| | - multi-label-classification |
| | - multi-class-classification |
| | - emotion |
| | - bert |
| | - go_emotions |
| | - emotion-classification |
| | datasets: |
| | - google-research-datasets/go_emotions |
| | metrics: |
| | - f1 |
| | - precision |
| | - recall |
| | widget: |
| | - text: I’m just chilling today. |
| | example_title: Neutral Example |
| | - text: Thank you for saving my life! |
| | example_title: Gratitude Example |
| | - text: I’m nervous about my exam tomorrow. |
| | example_title: Nervousness Example |
| | base_model: |
| | - google-bert/bert-base-uncased |
| | --- |
| | |
| | # GoEmotions BERT Classifier |
| |
|
| | Fine-tuned [BERT-base-uncased](https://huggingface.co/bert-base-uncased) on [go_emotions](https://huggingface.co/datasets/go_emotions) for multi-label classification (28 emotions). |
| |
|
| | ## Model Details |
| |
|
| | - **Architecture**: BERT-base-uncased (110M parameters) |
| | - **Training Data**: [GoEmotions](https://huggingface.co/datasets/google-research-datasets/go_emotions) (58k Reddit comments, 28 emotions) |
| | - **Loss Function**: Focal Loss (gamma=2) |
| | - **Optimizer**: AdamW (lr=2e-5, weight_decay=0.01) |
| | - **Epochs**: 5 |
| | - **Hardware**: Kaggle T4 x2 GPUs |
| | |
| | ## Try It Out |
| | For accurate predictions with optimized thresholds, use the [Gradio demo](https://logasanjeev-goemotions-bert-demo.hf.space). |
| | |
| | ## Performance |
| | |
| | - **Micro F1**: 0.6025 (optimized thresholds) |
| | - **Macro F1**: 0.5266 |
| | - **Precision**: 0.5425 |
| | - **Recall**: 0.6775 |
| | - **Hamming Loss**: 0.0372 |
| | - **Avg Positive Predictions**: 1.4564 |
| | |
| | ### Class-Wise Performance |
| | The following table shows per-class metrics on the test set using optimized thresholds (see `thresholds.json`): |
| | |
| | | Emotion | F1 Score | Precision | Recall | Support | |
| | |----------------|----------|-----------|--------|---------| |
| | | admiration | 0.7022 | 0.6980 | 0.7063 | 504 | |
| | | amusement | 0.8171 | 0.7692 | 0.8712 | 264 | |
| | | anger | 0.5123 | 0.5000 | 0.5253 | 198 | |
| | | annoyance | 0.3820 | 0.2908 | 0.5563 | 320 | |
| | | approval | 0.4112 | 0.3485 | 0.5014 | 351 | |
| | | caring | 0.4601 | 0.4045 | 0.5333 | 135 | |
| | | confusion | 0.4488 | 0.4533 | 0.4444 | 153 | |
| | | curiosity | 0.5721 | 0.4402 | 0.8169 | 284 | |
| | | desire | 0.4068 | 0.6857 | 0.2892 | 83 | |
| | | disappointment | 0.3476 | 0.3220 | 0.3775 | 151 | |
| | | disapproval | 0.4126 | 0.3433 | 0.5169 | 267 | |
| | | disgust | 0.4950 | 0.6329 | 0.4065 | 123 | |
| | | embarrassment | 0.5000 | 0.7368 | 0.3784 | 37 | |
| | | excitement | 0.4084 | 0.4432 | 0.3786 | 103 | |
| | | fear | 0.6311 | 0.5078 | 0.8333 | 78 | |
| | | gratitude | 0.9173 | 0.9744 | 0.8665 | 352 | |
| | | grief | 0.2500 | 0.5000 | 0.1667 | 6 | |
| | | joy | 0.6246 | 0.5798 | 0.6770 | 161 | |
| | | love | 0.8110 | 0.7630 | 0.8655 | 238 | |
| | | nervousness | 0.3830 | 0.3750 | 0.3913 | 23 | |
| | | optimism | 0.5777 | 0.5856 | 0.5699 | 186 | |
| | | pride | 0.4138 | 0.4615 | 0.3750 | 16 | |
| | | realization | 0.2421 | 0.5111 | 0.1586 | 145 | |
| | | relief | 0.5385 | 0.4667 | 0.6364 | 11 | |
| | | remorse | 0.6797 | 0.5361 | 0.9286 | 56 | |
| | | sadness | 0.5391 | 0.6900 | 0.4423 | 156 | |
| | | surprise | 0.5724 | 0.5570 | 0.5887 | 141 | |
| | | neutral | 0.6895 | 0.5826 | 0.8444 | 1787 | |
| | |
| | ## Usage |
| | |
| | The model uses optimized thresholds stored in `thresholds.json` for predictions. Example in Python: |
| | |
| | ```python |
| | from transformers import BertForSequenceClassification, BertTokenizer |
| | import torch |
| | import json |
| | import requests |
| | |
| | # Load model and tokenizer |
| | repo_id = "logasanjeev/goemotions-bert" |
| | model = BertForSequenceClassification.from_pretrained(repo_id) |
| | tokenizer = BertTokenizer.from_pretrained(repo_id) |
| |
|
| | # Load thresholds |
| | thresholds_url = f"https://huggingface.co/{repo_id}/raw/main/thresholds.json" |
| | thresholds_data = json.loads(requests.get(thresholds_url).text) |
| | emotion_labels = thresholds_data["emotion_labels"] |
| | thresholds = thresholds_data["thresholds"] |
| |
|
| | # Predict |
| | text = "I’m just chilling today." |
| | encodings = tokenizer(text, padding='max_length', truncation=True, max_length=128, return_tensors='pt') |
| | with torch.no_grad(): |
| | logits = torch.sigmoid(model(**encodings).logits).numpy()[0] |
| | predictions = [(emotion_labels[i], logit) for i, (logit, thresh) in enumerate(zip(logits, thresholds)) if logit >= thresh] |
| | print(sorted(predictions, key=lambda x: x[1], reverse=True)) |
| | # Output: [('neutral', 0.8147)] |
