README.md

---
license: mit
language:
  - en
  - hi
library_name: transformers
pipeline_tag: text-classification
tags:
  - emotion-detection
  - distilbert
  - sentiment-analysis
  - mental-health
  - emotion-classification
  - text-classification
  - transformers
  - pytorch
  - hinglish
base_model: distilbert-base-uncased
datasets:
  - google-research-datasets/go_emotions
metrics:
  - accuracy
  - f1
  - precision
  - recall
model-index:
  - name: raven-emotion-distilbert
    results:
      - task:
          type: text-classification
          name: Emotion Classification
        dataset:
          name: Custom Indian + International Dataset
          type: custom
        metrics:
          - name: Accuracy
            type: accuracy
            value: 0.9762
          - name: F1
            type: f1
            value: 0.9762
          - name: Precision
            type: precision
            value: 0.9762
          - name: Recall
            type: recall
            value: 0.9762
      - task:
          type: text-classification
          name: Emotion Classification
        dataset:
          name: GoEmotions (Balanced 300 samples)
          type: google-research-datasets/go_emotions
        metrics:
          - name: Accuracy
            type: accuracy
            value: 0.7733
          - name: F1
            type: f1
            value: 0.7724
widget:
  - text: "I'm so stressed about my exam tomorrow, I can't sleep"
    example_title: Anxious
  - text: "Just got promoted at work, feeling on top of the world!"
    example_title: Happy
  - text: "I don't understand why this code keeps throwing errors"
    example_title: Confused
  - text: "I lost my best friend over a stupid argument"
    example_title: Sad
  - text: "This is absolutely unacceptable, I'm furious right now"
    example_title: Angry
  - text: "Nothing much going on today, just chilling at home"
    example_title: Neutral
---

# Raven Emotion DistilBERT

A fine-tuned **DistilBERT** model for 6-class emotion classification, built for [Raven AI](https://raven-ai-new.streamlit.app) — an emotionally aware AI assistant.

This model classifies text into **6 emotions**: `happy`, `sad`, `anxious`, `angry`, `confused`, `neutral`.

## Performance

| Model / Method | Dataset | Accuracy | F1 Score |
|---|---|---|---|
| Zero-Shot LLM (LLama 3.3 70B) | GoEmotions | 66.67% | 0.6691 |
| Few-Shot LLM (LLama 3.3 70B) | GoEmotions | 73.00% | 0.7331 |
| **This model** (initial training) | GoEmotions | **77.33%** | **0.7724** |
| **This model** (after domain adaptation) | Custom Dataset | **97.62%** | **0.9762** |

**Key result**: This 67M parameter model outperforms a 70B parameter LLM by +4.33% on emotion classification, proving that task-specific fine-tuning beats general-purpose prompting.

## Quick Start

```python
from transformers import pipeline

classifier = pipeline("text-classification", model="Fynman-stack/raven-emotion-distilbert", top_k=None)

result = classifier("I'm so stressed about my exam tomorrow")
print(result)
# [[{'label': 'anxious', 'score': 0.95}, {'label': 'sad', 'score': 0.02}, ...]]
```

Or load the model directly:

```python
from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

tokenizer = AutoTokenizer.from_pretrained("Fynman-stack/raven-emotion-distilbert")
model = AutoModelForSequenceClassification.from_pretrained("Fynman-stack/raven-emotion-distilbert")

EMOTIONS = ["happy", "sad", "anxious", "angry", "confused", "neutral"]

def detect_emotion(text):
    inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=128, padding=True)
    with torch.no_grad():
        outputs = model(**inputs)
    return EMOTIONS[torch.argmax(outputs.logits, dim=1).item()]

print(detect_emotion("I just cleared my exam!"))  # happy
print(detect_emotion("I'm furious at this situation"))  # angry
```

## Labels

| ID | Label | Description |
|---|---|---|
| 0 | `happy` | Joy, excitement, gratitude, love, pride, amusement |
| 1 | `sad` | Sadness, grief, disappointment, remorse |
| 2 | `anxious` | Fear, nervousness, worry, stress |
| 3 | `angry` | Anger, annoyance, frustration, disgust |
| 4 | `confused` | Confusion, surprise, curiosity, realization |
| 5 | `neutral` | Neutral, calm, indifferent |

## Training Details

### Phase 1: Initial Training on GoEmotions

- **Base model**: `distilbert-base-uncased` (67M parameters)
- **Dataset**: [GoEmotions](https://huggingface.co/datasets/google-research-datasets/go_emotions) — Google's 28-emotion dataset, mapped to 6 categories
- **Epochs**: 3 | **Batch size**: 16 | **Learning rate**: 2e-5 | **Optimizer**: AdamW (weight decay 0.01)

| Epoch | Train Loss | Val Accuracy | Val F1 |
|---|---|---|---|
| 1 | 1.1599 | 66.93% | 0.6671 |
| 2 | 0.8031 | 67.37% | 0.6737 |
| 3 | 0.6494 | 67.64% | 0.6747 |

### Phase 2: Domain Adaptation on Custom Dataset

The model was further trained on ~12,343 samples of Indian English, Hinglish (Hindi-English), American English, and British English conversational text to adapt it for real-world student conversations.

- **Learning rate**: 5e-6 (reduced to prevent catastrophic forgetting)
- **Early stopping**: Patience of 2 epochs
- **Warmup**: 10% of total training steps
- **Gradient clipping**: 1.0

| Epoch | Train Loss | Val Accuracy | Val F1 |
|---|---|---|---|
| 1 | 0.6765 | 90.99% | 0.9093 |
| 2 | 0.2549 | 93.15% | 0.9311 |
| 3 | 0.1625 | 94.08% | 0.9406 |
| 4 | 0.1147 | 94.46% | 0.9444 |
| 5 | 0.0940 | 94.65% | 0.9463 |

**Domain adaptation impact**: Accuracy jumped from 64.38% to 97.62% (+33.24%) on the target domain.

## GoEmotions Label Mapping

The original 28 GoEmotions labels were mapped to 6 categories:

| Raven Label | GoEmotions Labels |
|---|---|
| `happy` | joy, amusement, excitement, gratitude, love, optimism, pride, relief, admiration, approval, caring |
| `sad` | sadness, grief, disappointment, remorse, embarrassment |
| `anxious` | fear, nervousness |
| `angry` | anger, annoyance, disgust |
| `confused` | confusion, surprise, realization, curiosity |
| `neutral` | neutral, desire |

## Use Cases

- **Emotionally aware chatbots** — Adjust response tone based on user emotion
- **Mental health applications** — Detect distress, anxiety, or anger in user messages
- **Customer support** — Route frustrated or confused customers to appropriate agents
- **Social media monitoring** — Track emotional sentiment across conversations
- **Education platforms** — Detect student frustration or confusion in real-time

## About Raven AI

This model powers [Raven AI](https://raven-ai-new.streamlit.app), an emotionally aware AI assistant that adapts its tone, persona, and response style based on detected user emotion. Raven includes crisis detection, multi-chat management, image understanding, voice input, document processing, and 20+ other features.

- **Try it live (HuggingFace Space)**: [huggingface.co/spaces/Fynman-stack/raven-ai](https://huggingface.co/spaces/Fynman-stack/raven-ai)
- **Streamlit Cloud**: [raven-ai-new.streamlit.app](https://raven-ai-new.streamlit.app)
- **GitHub**: [github.com/Fynman-stack/raven-ai](https://github.com/Fynman-stack/raven-ai)

## Model Architecture

- **Base**: DistilBERT (6 layers, 12 attention heads, 768 hidden dim)
- **Parameters**: 67M
- **Task head**: Sequence classification (6 classes)
- **Max sequence length**: 128 tokens
- **Format**: Safetensors (FP32)

## Limitations

- Trained primarily on English and Hinglish text — may not generalize well to other languages
- Emotion categories are coarse-grained (6 classes) — may miss nuanced emotional states
- Performance on formal/academic text may differ from conversational text
- Not a diagnostic tool — should not be used as a substitute for professional mental health assessment

## Citation

```bibtex
@misc{raha2026raven,
  title={Raven AI: An Emotionally Aware AI Assistant with Fine-tuned DistilBERT},
  author={Soumyadip Raha},
  year={2026},
  url={https://huggingface.co/Fynman-stack/raven-emotion-distilbert}
}
```

## License

MIT