README.md

---
license: apache-2.0
datasets:
- google-research-datasets/go_emotions
language:
- ar
- fr
- en
metrics:
- accuracy
- f1
pipeline_tag: text-classification
base_model:
- AnasAlokla/multilingual_go_emotions_V1.2
tags:
- emotion-analysis
- multilingual
- fine-tuned
- text-classification
---

# Model Card — Samra02/multilingual_emotion_analysis

> A multilingual emotion classifier fine-tuned to detect **5 core emotions** (Joy, Sadness, Anger, Fear, Disgust) in **English, Arabic, and French**, fine-tuned from a 27-class GoEmotions base model.

---

## Model Details

### Model Description

This model fine-tunes [`AnasAlokla/multilingual_go_emotions_V1.2`](https://huggingface.co/AnasAlokla/multilingual_go_emotions_V1.2) — a multilingual XLM-RoBERTa model originally trained on 27 GoEmotions classes — by collapsing the label space to 5 psychologically grounded emotion categories. The 27-class classification head is replaced with a new 5-class head and trained using a two-phase strategy: head warmup (backbone frozen) followed by full end-to-end fine-tuning.

The model supports three languages. English coverage comes directly from GoEmotions training data. Arabic and French are supported via zero-shot cross-lingual transfer from the XLM-RoBERTa multilingual backbone.

- **Developed by:** Samra02
- **Model type:** Transformer encoder — sequence classification (XLM-RoBERTa base)
- **Language(s) (NLP):** English (`en`), Arabic (`ar`), French (`fr`)
- **License:** Apache 2.0
- **Finetuned from model:** [`AnasAlokla/multilingual_go_emotions_V1.2`](https://huggingface.co/AnasAlokla/multilingual_go_emotions_V1.2)

### Model Sources

- **Repository:** https://huggingface.co/Samra02/multilingual_emotion_analysis
- **Demo:** Available via the Hugging Face Inference API on this model page

---

## Uses

### Direct Use

This model can be used directly for single-label emotion classification on short texts (tweets, comments, reviews, chat messages) without any additional fine-tuning. It accepts text in English, Arabic, or French and returns one of five emotion labels with a confidence score.

```python
from transformers import pipeline

clf = pipeline(
    "text-classification",
    model="Samra02/multilingual_emotion_analysis",
    top_k=1,
)

# English
clf("I am so happy and grateful for everything!")
# → [{'label': 'joy', 'score': 0.9731}]

# Arabic
clf("أنا خائف جداً مما قد يحدث")
# → [{'label': 'fear', 'score': 0.9412}]

# French
clf("C'est absolument dégoûtant, je n'arrive pas à y croire.")
# → [{'label': 'disgust', 'score': 0.9187}]
```

### Downstream Use

This model can be plugged into larger NLP pipelines for tasks such as:

- **Customer feedback analysis** — tagging support tickets or reviews by emotional tone
- **Social media monitoring** — detecting emotional trends in multilingual posts
- **Chatbot / dialogue systems** — adapting responses based on detected user emotion
- **Content moderation** — flagging emotionally charged content (anger, disgust) for review
- **Mental health research** — detecting emotional signals in user-generated text (with appropriate ethical oversight)

It can be further fine-tuned on domain-specific labeled data to improve performance for a particular use case or language.

### Out-of-Scope Use

- **Clinical or medical diagnosis** — this model is not validated for mental health assessment and must not be used as a diagnostic tool
- **High-stakes automated decision-making** — predictions should not be used without human review in consequential settings (hiring, legal, healthcare)
- **Languages beyond English, Arabic, and French** — performance has not been evaluated on other languages
- **Fine-grained emotion detection** — the 5-class output intentionally omits nuanced emotions (e.g., admiration, curiosity, relief); use the 27-class base model if finer granularity is required
- **Formal or literary text** — the model was trained on social media (Reddit); performance may degrade on formal documents, literature, or academic writing

---

## Bias, Risks, and Limitations

**Training data bias:** GoEmotions is sourced from English Reddit comments, a platform with known demographic skews (predominantly English-speaking, Western, younger users). Emotion expressions and norms vary significantly across cultures and languages — labels assigned by English-speaking annotators may not transfer equally to Arabic or French emotional contexts.

**Label collapse:** Merging 27 GoEmotions labels into 5 categories introduces ambiguity. For example, `gratitude` and `excitement` are both mapped to `joy`, which may not be appropriate for all downstream tasks. The mapping is documented in the Training Details section.

**Cross-lingual transfer gap:** Arabic and French performance relies entirely on zero-shot cross-lingual transfer from the multilingual backbone. No native-language emotion-labeled data was used. Performance on Arabic and French is expected to be lower than on English, especially for dialectal Arabic.

**Dropped labels:** Samples labeled with admiration, confusion, curiosity, desire, realization, surprise, neutral, or approval were excluded from training. The model cannot predict these categories and may misclassify such inputs.

**Confidence miscalibration:** Softmax scores should not be interpreted as calibrated probabilities. High confidence scores do not guarantee correctness, particularly for short or ambiguous text.

### Recommendations

- Always present model predictions as suggestions, not ground truth — include confidence scores and allow human review for important decisions.
- Evaluate the model on a representative sample of your target domain and language before deploying in production.
- For Arabic, evaluate separately on Modern Standard Arabic (MSA) vs. dialectal Arabic — performance may differ significantly.
- Do not use this model as the sole signal in any mental health, moderation, or safety-critical system.
- Monitor for distributional shift if deploying on data that differs from social media text (e.g., formal documents, speech transcripts).

---

## How to Get Started with the Model

**Install dependencies:**

```bash
pip install transformers torch
```

**Basic usage:**

```python
from transformers import pipeline

clf = pipeline(
    "text-classification",
    model="Samra02/multilingual_emotion_analysis",
    top_k=5,   # returns all 5 emotion scores
)

text = "He was furious and couldn't calm down."
results = clf(text)
for r in results[0]:
    print(f"{r['label']:10s}  {r['score']:.4f}")
```

**Load model and tokenizer manually:**

```python
from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

tokenizer = AutoTokenizer.from_pretrained("Samra02/multilingual_emotion_analysis")
model     = AutoModelForSequenceClassification.from_pretrained("Samra02/multilingual_emotion_analysis")

inputs  = tokenizer("Je suis tellement heureux!", return_tensors="pt", truncation=True, max_length=128)
outputs = model(**inputs)
probs   = torch.softmax(outputs.logits, dim=-1)

id2label = model.config.id2label
for idx, score in enumerate(probs[0]):
    print(f"{id2label[idx]:10s}  {score.item():.4f}")
```

---

## Training Details

### Training Data

**Dataset:** [`google-research-datasets/go_emotions`](https://huggingface.co/datasets/google-research-datasets/go_emotions) — `simplified` split
**Original size:** ~58,000 English Reddit comments annotated for 27 emotion categories
**After filtering:** Multi-label samples and samples belonging to unmapped classes were removed

**Label mapping (27 → 5):**

| Target label | Source GoEmotions labels |
|---|---|
| `joy` | joy, amusement, excitement, gratitude, love, optimism, pride, relief, caring |
| `sadness` | sadness, grief, disappointment, remorse |
| `anger` | anger, annoyance, disapproval |
| `fear` | fear, nervousness |
| `disgust` | disgust, embarrassment |
| *(dropped)* | admiration, confusion, curiosity, desire, realization, surprise, neutral, approval |

**Effective dataset sizes after filtering:**

| Split | Samples (approx.) |
|---|---|
| Train | ~30,000 |
| Validation | ~3,700 |
| Test | ~3,700 |

### Training Procedure

Training used a **two-phase strategy** to safely replace the 27-class head with a new 5-class head without destabilizing the pretrained multilingual representations.

#### Preprocessing

- Tokenization with the XLM-RoBERTa sentencepiece tokenizer
- Truncation and padding to `max_length = 128` tokens
- Multi-label samples (more than one label per example) excluded
- Samples with unmapped labels excluded

**Critical fix applied during model loading:**

```python
model = AutoModelForSequenceClassification.from_pretrained(
    "AnasAlokla/multilingual_go_emotions_V1.2",
    num_labels=5,
    ignore_mismatched_sizes=True,
    problem_type="single_label_classification",  # switches loss to CrossEntropyLoss
)
```

Without `problem_type="single_label_classification"`, the model inherits the base model's `BCEWithLogitsLoss` (multi-label loss), causing a shape mismatch error at training time.

#### Training Hyperparameters

| Parameter | Phase 1 — head warmup | Phase 2 — full fine-tuning |
|---|---|---|
| Epochs | 1 | up to 5 (early stopping) |
| Learning rate | `1e-3` | `2e-5` |
| LR scheduler | — | cosine |
| Warmup ratio | — | 0.1 |
| Batch size | 32 | 32 |
| Weight decay | 0.01 | 0.01 |
| Backbone frozen | Yes | No |
| Optimizer | AdamW | AdamW |
| **Training regime** | **fp16 mixed precision** | **fp16 mixed precision** |
| Early stopping patience | — | 2 epochs |

#### Speeds, Sizes, Times

- **Hardware:** Google Colab — NVIDIA T4 GPU (16 GB VRAM)
- **Approximate training time:** ~25–40 minutes (Phase 1 + Phase 2 combined on T4 GPU)
- **Model size:** ~1.1 GB (`model.safetensors`)
- **Total parameters:** ~278M | **Trainable in Phase 1 (head only):** ~594K

---

## Evaluation

### Testing Data, Factors & Metrics

#### Testing Data

Evaluation was performed on the **GoEmotions test set** (`simplified` split) after applying the same label mapping and filtering used during training. Only single-label samples belonging to the 5 mapped emotion categories were included.

Dataset card: [`google-research-datasets/go_emotions`](https://huggingface.co/datasets/google-research-datasets/go_emotions)

#### Factors

Evaluation is disaggregated by:

- **Emotion class** — per-class precision, recall, and F1
- **Language** — English (from GoEmotions test set); Arabic and French evaluated manually on a small set of translated examples (no standard benchmark dataset used)

#### Metrics

- **Accuracy** — overall fraction of correctly classified samples
- **F1 Macro** — unweighted mean of per-class F1; preferred due to class imbalance (used for model selection)
- **F1 Weighted** — F1 weighted by class support
- **Per-class F1** — precision, recall, and F1 for each of the 5 emotion labels individually

### Results

| Metric | Score |
|--------|-------|
| Accuracy | 0.92 |
| F1 Macro | 0.89 |
| F1 Weighted | 0.92 |

**Per-class results on GoEmotions test set:**

| Emotion | Precision | Recall | F1 | Support |
| :--- | :---: | :---: | :---: | :---: |
| **joy** | 0.96 | 0.98 | 0.97 | 963 |
| **sadness** | 0.91 | 0.81 | 0.85 | 236 |
| **anger** | 0.87 | 0.91 | 0.89 | 520 |
| **fear** | 1.00 | 0.88 | 0.94 | 77 |
| **disgust** | 0.81 | 0.81 | 0.81 | 99 |


#### Summary

The model collapses a 27-class multilingual emotion space to 5 core emotions while preserving cross-lingual capabilities from the XLM-RoBERTa backbone. English performance is expected to be strong given the training data source. Arabic and French performance depends on the quality of multilingual transfer from the base model and may be lower, particularly for dialectal Arabic.

---

## Model Examination

The two-phase training strategy was chosen specifically to address the mismatch between the base model's multi-label training objective (`BCEWithLogitsLoss`) and the single-label classification objective of this model (`CrossEntropyLoss`). Training end-to-end from a random head without a warmup phase risks large, destabilizing gradients in early steps.

The most critical configuration change is `problem_type="single_label_classification"` — it overrides the inherited value from the base model's `config.json` and ensures the correct loss function is used throughout training. Without it, a `ValueError: Target size ([32]) must be the same as input size ([32, 5])` will occur on the first training batch.

---

## Environmental Impact

Carbon emissions estimated using the [ML Impact Calculator](https://mlco2.github.io/impact#compute) (Lacoste et al., 2019).

- **Hardware type:** NVIDIA T4 GPU (Google Colab)
- **Hours used:** ~1 hour (estimated)
- **Cloud provider:** Google (Colab)
- **Compute region:** `us-central1` (estimated)
- **Carbon emitted:** ~0.03 kg CO₂eq (estimated)

---

## Technical Specifications

### Model Architecture and Objective

- **Base architecture:** XLM-RoBERTa (transformer encoder, 12 layers, 768 hidden size, 12 attention heads)
- **Classification head:** Linear(768 → 5) + softmax
- **Objective:** Cross-entropy loss (`single_label_classification`)
- **Vocabulary:** 250,000 sentencepiece tokens (multilingual)
- **Max sequence length:** 128 tokens

### Compute Infrastructure

#### Hardware

- Google Colab — NVIDIA T4 GPU, 16 GB VRAM
- 12 GB system RAM (Colab standard tier)

#### Software

| Library | Version |
|---|---|
| Python | 3.10+ |
| PyTorch | 2.x |
| Transformers | 4.x |
| Datasets | 2.x |
| Evaluate | 0.4.x |
| Accelerate | 0.x |
| scikit-learn | 1.x |

---

## Citation

**BibTeX:**

```bibtex
@misc{samra02-emotion5-2025,
  author    = {Samra02},
  title     = {Multilingual Emotion Analysis — 5-class (Joy, Sadness, Anger, Fear, Disgust)},
  year      = {2025},
  publisher = {Hugging Face},
  url       = {https://huggingface.co/Samra02/multilingual_emotion_analysis},
}

@inproceedings{demszky-etal-2020-goemotions,
  title     = {{GoEmotions}: A Dataset of Fine-Grained Emotions},
  author    = {Demszky, Dorottya and Movshovitz-Attias, Dana and Ko, Jeongwoo
               and Cowen, Alan and Nemade, Gaurav and Ravi, Sujith},
  booktitle = {Proceedings of the 58th Annual Meeting of the Association
               for Computational Linguistics},
  year      = {2020},
  url       = {https://aclanthology.org/2020.acl-main.372},
}
```

**APA:**

Samra02. (2025). *Multilingual Emotion Analysis — 5-class*. Hugging Face. https://huggingface.co/Samra02/multilingual_emotion_analysis

Demszky, D., Movshovitz-Attias, D., Ko, J., Cowen, A., Nemade, G., & Ravi, S. (2020). GoEmotions: A dataset of fine-grained emotions. *Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics*. https://aclanthology.org/2020.acl-main.372

---

## Glossary

- **GoEmotions:** A large-scale emotion dataset of ~58K English Reddit comments annotated for 27 emotion categories, published by Google Research (2020).
- **XLM-RoBERTa:** A multilingual transformer pretrained on 100+ languages using masked language modeling; the backbone of both the base model and this fine-tuned model.
- **Cross-lingual transfer:** The ability of a multilingual model to perform a task in a language it was not explicitly fine-tuned on, by leveraging shared representations learned during multilingual pretraining.
- **Single-label classification:** Each input is assigned exactly one label. Uses `CrossEntropyLoss`. Contrasted with multi-label classification, where inputs can have multiple labels simultaneously (`BCEWithLogitsLoss`).
- **F1 Macro:** The unweighted average F1 score across all classes. Penalizes poor performance on minority classes equally to majority classes — the preferred metric when class distribution is imbalanced.
- **problem_type:** A Hugging Face Transformers config argument that determines which loss function is used during training. `single_label_classification` → CrossEntropyLoss; `multi_label_classification` → BCEWithLogitsLoss.

---

## More Information

- Base model: [`AnasAlokla/multilingual_go_emotions_V1.2`](https://huggingface.co/AnasAlokla/multilingual_go_emotions_V1.2)
- Training dataset: [`google-research-datasets/go_emotions`](https://huggingface.co/datasets/google-research-datasets/go_emotions)
- GoEmotions paper: [ACL Anthology — ACL 2020](https://aclanthology.org/2020.acl-main.372)
- XLM-RoBERTa paper: [arXiv:1911.02116](https://arxiv.org/abs/1911.02116)

---

## Model Card Authors

Samra02

---

## Model Card Contact

Open an issue or start a discussion on the [model repository page](https://huggingface.co/Samra02/multilingual_emotion_analysis).