Update links to new Fynman-stack username

093fd17 verified about 2 months ago

8.06 kB

license: mit
language:
  - en
  - hi
library_name: transformers
pipeline_tag: text-classification
tags:
  - emotion-detection
  - distilbert
  - sentiment-analysis
  - mental-health
  - emotion-classification
  - text-classification
  - transformers
  - pytorch
  - hinglish
base_model: distilbert-base-uncased
datasets:
  - google-research-datasets/go_emotions
metrics:
  - accuracy
  - f1
  - precision
  - recall
model-index:
  - name: raven-emotion-distilbert
    results:
      - task:
          type: text-classification
          name: Emotion Classification
        dataset:
          name: Custom Indian + International Dataset
          type: custom
        metrics:
          - name: Accuracy
            type: accuracy
            value: 0.9762
          - name: F1
            type: f1
            value: 0.9762
          - name: Precision
            type: precision
            value: 0.9762
          - name: Recall
            type: recall
            value: 0.9762
      - task:
          type: text-classification
          name: Emotion Classification
        dataset:
          name: GoEmotions (Balanced 300 samples)
          type: google-research-datasets/go_emotions
        metrics:
          - name: Accuracy
            type: accuracy
            value: 0.7733
          - name: F1
            type: f1
            value: 0.7724
widget:
  - text: I'm so stressed about my exam tomorrow, I can't sleep
    example_title: Anxious
  - text: Just got promoted at work, feeling on top of the world!
    example_title: Happy
  - text: I don't understand why this code keeps throwing errors
    example_title: Confused
  - text: I lost my best friend over a stupid argument
    example_title: Sad
  - text: This is absolutely unacceptable, I'm furious right now
    example_title: Angry
  - text: Nothing much going on today, just chilling at home
    example_title: Neutral

Raven Emotion DistilBERT

A fine-tuned DistilBERT model for 6-class emotion classification, built for Raven AI — an emotionally aware AI assistant.

This model classifies text into 6 emotions: happy, sad, anxious, angry, confused, neutral.

Performance

Model / Method	Dataset	Accuracy	F1 Score
Zero-Shot LLM (LLama 3.3 70B)	GoEmotions	66.67%	0.6691
Few-Shot LLM (LLama 3.3 70B)	GoEmotions	73.00%	0.7331
This model (initial training)	GoEmotions	77.33%	0.7724
This model (after domain adaptation)	Custom Dataset	97.62%	0.9762

Key result: This 67M parameter model outperforms a 70B parameter LLM by +4.33% on emotion classification, proving that task-specific fine-tuning beats general-purpose prompting.

Quick Start

from transformers import pipeline

classifier = pipeline("text-classification", model="Fynman-stack/raven-emotion-distilbert", top_k=None)

result = classifier("I'm so stressed about my exam tomorrow")
print(result)
# [[{'label': 'anxious', 'score': 0.95}, {'label': 'sad', 'score': 0.02}, ...]]

Or load the model directly:

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

tokenizer = AutoTokenizer.from_pretrained("Fynman-stack/raven-emotion-distilbert")
model = AutoModelForSequenceClassification.from_pretrained("Fynman-stack/raven-emotion-distilbert")

EMOTIONS = ["happy", "sad", "anxious", "angry", "confused", "neutral"]

def detect_emotion(text):
    inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=128, padding=True)
    with torch.no_grad():
        outputs = model(**inputs)
    return EMOTIONS[torch.argmax(outputs.logits, dim=1).item()]

print(detect_emotion("I just cleared my exam!"))  # happy
print(detect_emotion("I'm furious at this situation"))  # angry

Labels

ID	Label	Description
0	`happy`	Joy, excitement, gratitude, love, pride, amusement
1	`sad`	Sadness, grief, disappointment, remorse
2	`anxious`	Fear, nervousness, worry, stress
3	`angry`	Anger, annoyance, frustration, disgust
4	`confused`	Confusion, surprise, curiosity, realization
5	`neutral`	Neutral, calm, indifferent

Training Details

Phase 1: Initial Training on GoEmotions

Base model: distilbert-base-uncased (67M parameters)
Dataset: GoEmotions — Google's 28-emotion dataset, mapped to 6 categories
Epochs: 3 | Batch size: 16 | Learning rate: 2e-5 | Optimizer: AdamW (weight decay 0.01)

Epoch	Train Loss	Val Accuracy	Val F1
1	1.1599	66.93%	0.6671
2	0.8031	67.37%	0.6737
3	0.6494	67.64%	0.6747

Phase 2: Domain Adaptation on Custom Dataset

The model was further trained on ~12,343 samples of Indian English, Hinglish (Hindi-English), American English, and British English conversational text to adapt it for real-world student conversations.

Learning rate: 5e-6 (reduced to prevent catastrophic forgetting)
Early stopping: Patience of 2 epochs
Warmup: 10% of total training steps
Gradient clipping: 1.0

Epoch	Train Loss	Val Accuracy	Val F1
1	0.6765	90.99%	0.9093
2	0.2549	93.15%	0.9311
3	0.1625	94.08%	0.9406
4	0.1147	94.46%	0.9444
5	0.0940	94.65%	0.9463

Domain adaptation impact: Accuracy jumped from 64.38% to 97.62% (+33.24%) on the target domain.

GoEmotions Label Mapping

The original 28 GoEmotions labels were mapped to 6 categories:

Raven Label	GoEmotions Labels
`happy`	joy, amusement, excitement, gratitude, love, optimism, pride, relief, admiration, approval, caring
`sad`	sadness, grief, disappointment, remorse, embarrassment
`anxious`	fear, nervousness
`angry`	anger, annoyance, disgust
`confused`	confusion, surprise, realization, curiosity
`neutral`	neutral, desire

Use Cases

Emotionally aware chatbots — Adjust response tone based on user emotion
Mental health applications — Detect distress, anxiety, or anger in user messages
Customer support — Route frustrated or confused customers to appropriate agents
Social media monitoring — Track emotional sentiment across conversations
Education platforms — Detect student frustration or confusion in real-time

About Raven AI

This model powers Raven AI, an emotionally aware AI assistant that adapts its tone, persona, and response style based on detected user emotion. Raven includes crisis detection, multi-chat management, image understanding, voice input, document processing, and 20+ other features.

Try it live (HuggingFace Space): huggingface.co/spaces/Fynman-stack/raven-ai
Streamlit Cloud: raven-ai-new.streamlit.app
GitHub: github.com/Fynman-stack/raven-ai

Model Architecture

Base: DistilBERT (6 layers, 12 attention heads, 768 hidden dim)
Parameters: 67M
Task head: Sequence classification (6 classes)
Max sequence length: 128 tokens
Format: Safetensors (FP32)

Limitations

Trained primarily on English and Hinglish text — may not generalize well to other languages
Emotion categories are coarse-grained (6 classes) — may miss nuanced emotional states
Performance on formal/academic text may differ from conversational text
Not a diagnostic tool — should not be used as a substitute for professional mental health assessment

Citation

@misc{raha2026raven,
  title={Raven AI: An Emotionally Aware AI Assistant with Fine-tuned DistilBERT},
  author={Soumyadip Raha},
  year={2026},
  url={https://huggingface.co/Fynman-stack/raven-emotion-distilbert}
}

License

MIT