README.md

metadata

license: apache-2.0
datasets:
  - google-research-datasets/go_emotions
language:
  - ar
  - fr
  - en
metrics:
  - accuracy
  - f1
pipeline_tag: text-classification
base_model:
  - AnasAlokla/multilingual_go_emotions_V1.2
tags:
  - emotion-analysis
  - multilingual
  - fine-tuned
  - text-classification

Model Card — Samra02/multilingual_emotion_analysis

A multilingual emotion classifier fine-tuned to detect 5 core emotions (Joy, Sadness, Anger, Fear, Disgust) in English, Arabic, and French, fine-tuned from a 27-class GoEmotions base model.

---

Model Details

Model Description

This model fine-tunes AnasAlokla/multilingual_go_emotions_V1.2 — a multilingual XLM-RoBERTa model originally trained on 27 GoEmotions classes — by collapsing the label space to 5 psychologically grounded emotion categories. The 27-class classification head is replaced with a new 5-class head and trained using a two-phase strategy: head warmup (backbone frozen) followed by full end-to-end fine-tuning.

The model supports three languages. English coverage comes directly from GoEmotions training data. Arabic and French are supported via zero-shot cross-lingual transfer from the XLM-RoBERTa multilingual backbone.

• Developed by: Samra02
• Model type: Transformer encoder — sequence classification (XLM-RoBERTa base)
• Language(s) (NLP): English (en), Arabic (ar), French (fr)
• License: Apache 2.0
• Finetuned from model: AnasAlokla/multilingual_go_emotions_V1.2

Model Sources

• Repository: https://huggingface.co/Samra02/multilingual_emotion_analysis
• Demo: Available via the Hugging Face Inference API on this model page

---

Uses

Direct Use

This model can be used directly for single-label emotion classification on short texts (tweets, comments, reviews, chat messages) without any additional fine-tuning. It accepts text in English, Arabic, or French and returns one of five emotion labels with a confidence score.

from transformers import pipeline

clf = pipeline(
    "text-classification",
    model="Samra02/multilingual_emotion_analysis",
    top_k=1,
)

# English
clf("I am so happy and grateful for everything!")
# → [{'label': 'joy', 'score': 0.9731}]

# Arabic
clf("أنا خائف جداً مما قد يحدث")
# → [{'label': 'fear', 'score': 0.9412}]

# French
clf("C'est absolument dégoûtant, je n'arrive pas à y croire.")
# → [{'label': 'disgust', 'score': 0.9187}]

Downstream Use

This model can be plugged into larger NLP pipelines for tasks such as:

• Customer feedback analysis — tagging support tickets or reviews by emotional tone
• Social media monitoring — detecting emotional trends in multilingual posts
• Chatbot / dialogue systems — adapting responses based on detected user emotion
• Content moderation — flagging emotionally charged content (anger, disgust) for review
• Mental health research — detecting emotional signals in user-generated text (with appropriate ethical oversight)

It can be further fine-tuned on domain-specific labeled data to improve performance for a particular use case or language.

Out-of-Scope Use

• Clinical or medical diagnosis — this model is not validated for mental health assessment and must not be used as a diagnostic tool
• High-stakes automated decision-making — predictions should not be used without human review in consequential settings (hiring, legal, healthcare)
• Languages beyond English, Arabic, and French — performance has not been evaluated on other languages
• Fine-grained emotion detection — the 5-class output intentionally omits nuanced emotions (e.g., admiration, curiosity, relief); use the 27-class base model if finer granularity is required
• Formal or literary text — the model was trained on social media (Reddit); performance may degrade on formal documents, literature, or academic writing

---

Bias, Risks, and Limitations

Training data bias: GoEmotions is sourced from English Reddit comments, a platform with known demographic skews (predominantly English-speaking, Western, younger users). Emotion expressions and norms vary significantly across cultures and languages — labels assigned by English-speaking annotators may not transfer equally to Arabic or French emotional contexts.

Label collapse: Merging 27 GoEmotions labels into 5 categories introduces ambiguity. For example, gratitude and excitement are both mapped to joy, which may not be appropriate for all downstream tasks. The mapping is documented in the Training Details section.

Cross-lingual transfer gap: Arabic and French performance relies entirely on zero-shot cross-lingual transfer from the multilingual backbone. No native-language emotion-labeled data was used. Performance on Arabic and French is expected to be lower than on English, especially for dialectal Arabic.

Dropped labels: Samples labeled with admiration, confusion, curiosity, desire, realization, surprise, neutral, or approval were excluded from training. The model cannot predict these categories and may misclassify such inputs.

Confidence miscalibration: Softmax scores should not be interpreted as calibrated probabilities. High confidence scores do not guarantee correctness, particularly for short or ambiguous text.

Recommendations

• Always present model predictions as suggestions, not ground truth — include confidence scores and allow human review for important decisions.
• Evaluate the model on a representative sample of your target domain and language before deploying in production.
• For Arabic, evaluate separately on Modern Standard Arabic (MSA) vs. dialectal Arabic — performance may differ significantly.
• Do not use this model as the sole signal in any mental health, moderation, or safety-critical system.
• Monitor for distributional shift if deploying on data that differs from social media text (e.g., formal documents, speech transcripts).

---

How to Get Started with the Model

Install dependencies:

pip install transformers torch

Basic usage:

from transformers import pipeline

clf = pipeline(
    "text-classification",
    model="Samra02/multilingual_emotion_analysis",
    top_k=5,   # returns all 5 emotion scores
)

text = "He was furious and couldn't calm down."
results = clf(text)
for r in results[0]:
    print(f"{r['label']:10s}  {r['score']:.4f}")

Load model and tokenizer manually:

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

tokenizer = AutoTokenizer.from_pretrained("Samra02/multilingual_emotion_analysis")
model     = AutoModelForSequenceClassification.from_pretrained("Samra02/multilingual_emotion_analysis")

inputs  = tokenizer("Je suis tellement heureux!", return_tensors="pt", truncation=True, max_length=128)
outputs = model(**inputs)
probs   = torch.softmax(outputs.logits, dim=-1)

id2label = model.config.id2label
for idx, score in enumerate(probs[0]):
    print(f"{id2label[idx]:10s}  {score.item():.4f}")

---

Training Details

Training Data

Dataset: google-research-datasets/go_emotions — simplified split Original size: ~58,000 English Reddit comments annotated for 27 emotion categories After filtering: Multi-label samples and samples belonging to unmapped classes were removed

Label mapping (27 → 5):

Target label	Source GoEmotions labels
joy	joy, amusement, excitement, gratitude, love, optimism, pride, relief, caring
sadness	sadness, grief, disappointment, remorse
anger	anger, annoyance, disapproval
fear	fear, nervousness
disgust	disgust, embarrassment
(dropped)	admiration, confusion, curiosity, desire, realization, surprise, neutral, approval

Effective dataset sizes after filtering:

Split	Samples (approx.)
Train	~30,000
Validation	~3,700
Test	~3,700

Training Procedure

Training used a two-phase strategy to safely replace the 27-class head with a new 5-class head without destabilizing the pretrained multilingual representations.

Preprocessing

• Tokenization with the XLM-RoBERTa sentencepiece tokenizer
• Truncation and padding to max_length = 128 tokens
• Multi-label samples (more than one label per example) excluded
• Samples with unmapped labels excluded

Critical fix applied during model loading:

model = AutoModelForSequenceClassification.from_pretrained(
    "AnasAlokla/multilingual_go_emotions_V1.2",
    num_labels=5,
    ignore_mismatched_sizes=True,
    problem_type="single_label_classification",  # switches loss to CrossEntropyLoss
)

Without problem_type="single_label_classification", the model inherits the base model's BCEWithLogitsLoss (multi-label loss), causing a shape mismatch error at training time.

Training Hyperparameters

Parameter	Phase 1 — head warmup	Phase 2 — full fine-tuning
Epochs	1	up to 5 (early stopping)
Learning rate	1e-3	2e-5
LR scheduler	—	cosine
Warmup ratio	—	0.1
Batch size	32	32
Weight decay	0.01	0.01
Backbone frozen	Yes	No
Optimizer	AdamW	AdamW
Training regime	fp16 mixed precision	fp16 mixed precision
Early stopping patience	—	2 epochs

Speeds, Sizes, Times

• Hardware: Google Colab — NVIDIA T4 GPU (16 GB VRAM)
• Approximate training time: ~25–40 minutes (Phase 1 + Phase 2 combined on T4 GPU)
• Model size: ~1.1 GB (model.safetensors)
• Total parameters: ~278M | Trainable in Phase 1 (head only): ~594K

---

Evaluation

Testing Data, Factors & Metrics

Testing Data

Evaluation was performed on the GoEmotions test set (simplified split) after applying the same label mapping and filtering used during training. Only single-label samples belonging to the 5 mapped emotion categories were included.

Dataset card: google-research-datasets/go_emotions

Factors

Evaluation is disaggregated by:

• Emotion class — per-class precision, recall, and F1
• Language — English (from GoEmotions test set); Arabic and French evaluated manually on a small set of translated examples (no standard benchmark dataset used)

Metrics

• Accuracy — overall fraction of correctly classified samples
• F1 Macro — unweighted mean of per-class F1; preferred due to class imbalance (used for model selection)
• F1 Weighted — F1 weighted by class support
• Per-class F1 — precision, recall, and F1 for each of the 5 emotion labels individually

Results

Metric	Score
Accuracy	0.92
F1 Macro	0.89
F1 Weighted	0.92

Per-class results on GoEmotions test set:

Emotion	Precision	Recall	F1	Support
joy	0.96	0.98	0.97	963
sadness	0.91	0.81	0.85	236
anger	0.87	0.91	0.89	520
fear	1.00	0.88	0.94	77
disgust	0.81	0.81	0.81	99

Summary

The model collapses a 27-class multilingual emotion space to 5 core emotions while preserving cross-lingual capabilities from the XLM-RoBERTa backbone. English performance is expected to be strong given the training data source. Arabic and French performance depends on the quality of multilingual transfer from the base model and may be lower, particularly for dialectal Arabic.

---

Model Examination

The two-phase training strategy was chosen specifically to address the mismatch between the base model's multi-label training objective (BCEWithLogitsLoss) and the single-label classification objective of this model (CrossEntropyLoss). Training end-to-end from a random head without a warmup phase risks large, destabilizing gradients in early steps.

The most critical configuration change is problem_type="single_label_classification" — it overrides the inherited value from the base model's config.json and ensures the correct loss function is used throughout training. Without it, a ValueError: Target size ([32]) must be the same as input size ([32, 5]) will occur on the first training batch.

---

Environmental Impact

Carbon emissions estimated using the ML Impact Calculator (Lacoste et al., 2019).

• Hardware type: NVIDIA T4 GPU (Google Colab)
• Hours used: ~1 hour (estimated)
• Cloud provider: Google (Colab)
• Compute region: us-central1 (estimated)
• Carbon emitted: ~0.03 kg CO₂eq (estimated)

---

Technical Specifications

Model Architecture and Objective

• Base architecture: XLM-RoBERTa (transformer encoder, 12 layers, 768 hidden size, 12 attention heads)
• Classification head: Linear(768 → 5) + softmax
• Objective: Cross-entropy loss (single_label_classification)
• Vocabulary: 250,000 sentencepiece tokens (multilingual)
• Max sequence length: 128 tokens

Compute Infrastructure

Hardware

• Google Colab — NVIDIA T4 GPU, 16 GB VRAM
• 12 GB system RAM (Colab standard tier)

Software

Library	Version
Python	3.10+
PyTorch	2.x
Transformers	4.x
Datasets	2.x
Evaluate	0.4.x
Accelerate	0.x
scikit-learn	1.x

---

Citation

BibTeX:

@misc{samra02-emotion5-2025,
  author    = {Samra02},
  title     = {Multilingual Emotion Analysis — 5-class (Joy, Sadness, Anger, Fear, Disgust)},
  year      = {2025},
  publisher = {Hugging Face},
  url       = {https://huggingface.co/Samra02/multilingual_emotion_analysis},
}

@inproceedings{demszky-etal-2020-goemotions,
  title     = {{GoEmotions}: A Dataset of Fine-Grained Emotions},
  author    = {Demszky, Dorottya and Movshovitz-Attias, Dana and Ko, Jeongwoo
               and Cowen, Alan and Nemade, Gaurav and Ravi, Sujith},
  booktitle = {Proceedings of the 58th Annual Meeting of the Association
               for Computational Linguistics},
  year      = {2020},
  url       = {https://aclanthology.org/2020.acl-main.372},
}

APA:

Samra02. (2025). Multilingual Emotion Analysis — 5-class. Hugging Face. https://huggingface.co/Samra02/multilingual_emotion_analysis

Demszky, D., Movshovitz-Attias, D., Ko, J., Cowen, A., Nemade, G., & Ravi, S. (2020). GoEmotions: A dataset of fine-grained emotions. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. https://aclanthology.org/2020.acl-main.372

---

Glossary

• GoEmotions: A large-scale emotion dataset of ~58K English Reddit comments annotated for 27 emotion categories, published by Google Research (2020).
• XLM-RoBERTa: A multilingual transformer pretrained on 100+ languages using masked language modeling; the backbone of both the base model and this fine-tuned model.
• Cross-lingual transfer: The ability of a multilingual model to perform a task in a language it was not explicitly fine-tuned on, by leveraging shared representations learned during multilingual pretraining.
• Single-label classification: Each input is assigned exactly one label. Uses CrossEntropyLoss. Contrasted with multi-label classification, where inputs can have multiple labels simultaneously (BCEWithLogitsLoss).
• F1 Macro: The unweighted average F1 score across all classes. Penalizes poor performance on minority classes equally to majority classes — the preferred metric when class distribution is imbalanced.
• problem_type: A Hugging Face Transformers config argument that determines which loss function is used during training. single_label_classification → CrossEntropyLoss; multi_label_classification → BCEWithLogitsLoss.

---

More Information

• Base model: AnasAlokla/multilingual_go_emotions_V1.2
• Training dataset: google-research-datasets/go_emotions
• GoEmotions paper: ACL Anthology — ACL 2020
• XLM-RoBERTa paper: arXiv:1911.02116

---

Model Card Authors

Samra02

---

Model Card Contact

Open an issue or start a discussion on the model repository page.