File size: 4,961 Bytes
34d5932 d61f8c6 34d5932 d61f8c6 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 |
---
license: mit
tags:
- emotion-classification
- mental-health
- multi-label
- transformers
- distilbert
- goemotions
language:
- en
metrics:
- f1
- precision
- recall
pipeline_tag: text-classification
base_model: distilbert-base-uncased
---
# Mental Health Emotion Detection - Enhanced DistilBERT
This model is a fine-tuned DistilBERT for multi-label emotion classification in mental health applications, detecting 28 different emotions from text input with enhanced architecture and advanced training techniques.
## Model Description
- **Model Type:** Enhanced DistilBERT (Fine-tuned)
- **Base Model:** distilbert-base-uncased
- **Task:** Multi-label emotion classification
- **Dataset:** GoEmotions (balanced and enhanced)
- **Languages:** English
- **Architecture:** Enhanced with additional layers, focal loss, and class balancing
## Performance
| Metric | Score |
|--------|-------|
| F1-Score | 0.298 |
| Precision | 0.459 |
| Recall | 0.260 |
| Accuracy | 89.5% |
| Improvement | 7.6x over baseline |
## Emotions Detected
The model can detect 28 emotions: admiration, amusement, anger, annoyance, approval, caring, confusion, curiosity, desire, disappointment, disapproval, disgust, embarrassment, excitement, fear, gratitude, grief, joy, love, nervousness, optimism, pride, realization, relief, remorse, sadness, surprise, neutral.
## Usage
```python
from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch
# Load model and tokenizer
tokenizer = AutoTokenizer.from_pretrained("YOUR_USERNAME/mental-health-enhanced-distilbert")
model = AutoModelForSequenceClassification.from_pretrained("YOUR_USERNAME/mental-health-enhanced-distilbert")
# Example usage
text = "I'm feeling really anxious about tomorrow"
inputs = tokenizer(text, return_tensors="pt", truncation=True, padding=True, max_length=128)
with torch.no_grad():
outputs = model(**inputs)
predictions = torch.sigmoid(outputs.logits)
# Get emotion labels
emotions = []
for i, score in enumerate(predictions[0]):
if score > 0.4: # Threshold
emotion = model.config.id2label[i]
emotions.append((emotion, score.item()))
print(emotions)
```
## Training Details
### Enhanced Architecture
- **Base:** DistilBERT with additional hidden layers
- **Enhancements:**
- Layer normalization
- Dropout regularization
- Enhanced forward pass with ReLU activations
- Multi-layer classification head (768 → 512 → 256 → 128 → 28)
### Advanced Training Techniques
- **Loss Function:** Focal Loss for class imbalance handling
- **Class Weighting:** Advanced weighting for rare emotions
- **Data Balancing:** Oversampling rare emotions, undersampling common ones
- **Optimization:** AdamW with cosine scheduling
- **Early Stopping:** Patience-based with best model saving
### Training Data
- **Dataset:** GoEmotions (balanced subset)
- **Training Samples:** ~12,750
- **Validation Samples:** ~2,250
- **Preprocessing:** Contraction expansion, lowercase normalization
- **Balancing:** Advanced sampling for 28 emotion categories
## Model Architecture
```
Input Text → DistilBERT Encoder → Enhanced Classification Head
↓
Hidden Layer 1 (768→512)
↓
Hidden Layer 2 (512→256)
↓
Hidden Layer 3 (256→128)
↓
Output Layer (128→28)
```
## Intended Use
This model is designed for:
- Mental health chatbots and companions
- Emotion-aware dialogue systems
- Mental health screening tools
- Research in computational psychology
- Empathetic AI applications
## Limitations
- Trained primarily on English text
- Performance may vary with very informal language
- Should not be used as sole diagnostic tool for mental health
- Requires context for optimal performance
## Training Metrics by Epoch
| Epoch | F1-Score | Precision | Recall |
|-------|----------|-----------|--------|
| 1 | 0.0145 | 0.0419 | 0.0089 |
| 2 | 0.1430 | 0.2797 | 0.1211 |
| 3 | 0.2141 | 0.4751 | 0.1804 |
| 4 | 0.2749 | 0.4317 | 0.2340 |
| 5 | 0.2897 | 0.4524 | 0.2533 |
| 6 | 0.2981 | 0.4592 | 0.2597 |
## Citation
If you use this model, please cite:
```
@misc{mental-health-emotion-distilbert,
title={Mental Health Emotion Detection - Enhanced DistilBERT},
author={Your Name},
year={2024},
publisher={Hugging Face},
url={https://huggingface.co/YOUR_USERNAME/mental-health-enhanced-distilbert}
}
```
## Acknowledgments
- Built on DistilBERT by Hugging Face
- Trained on GoEmotions dataset
- Enhanced with advanced ML techniques for mental health applications
|