|
|
--- |
|
|
license: mit |
|
|
datasets: |
|
|
- google-research-datasets/go_emotions |
|
|
language: |
|
|
- en |
|
|
base_model: |
|
|
- google/mobilebert-uncased |
|
|
pipeline_tag: text-classification |
|
|
library_name: transformers |
|
|
--- |
|
|
|
|
|
#### Overview |
|
|
|
|
|
|
|
|
Model trained from [mobileBert](https://huggingface.co/google/mobilebert-uncased) on the [go_emotions](https://huggingface.co/datasets/google-research-datasets/go_emotions) dataset for multi-label classification. |
|
|
<div> |
|
|
<a href=https://github.com/04AR/Senti target="_blank"><img src=https://img.shields.io/badge/Code-black.svg?logo=github height=22px></a> |
|
|
<a href=https://huggingface.co/AR04/Senti target="_blank"><img src=https://img.shields.io/badge/%F0%9F%A4%97%20Models-d96902.svg height=22px></a> |
|
|
<a href=https://github.com/04AR/Senti/blob/main/Senti.ipynb target="_blank"><img src="https://img.shields.io/badge/Training%20Code-%20jupyter-orange"></a> |
|
|
<a href=https://huggingface.co/spaces/AR04/SentiDemo target="_blank"><img src="https://img.shields.io/badge/Demo-%20HuggingFaceDemo-orange"></a> |
|
|
</div> |
|
|
|
|
|
#### Dataset used for the model |
|
|
|
|
|
[go_emotions](https://huggingface.co/datasets/google-research-datasets/go_emotions) is based on Reddit data and has 28 labels. It is a multi-label dataset where one or multiple labels may apply for any given input text, hence this model is a multi-label classification model with 28 'probability' float outputs for any given input text. Typically a threshold of 0.5 is applied to the probabilities for the prediction for each label. |
|
|
|
|
|
## 🏷️ Emotion Labels |
|
|
|
|
|
| ID | Emotion | ID | Emotion | ID | Emotion | ID | Emotion | |
|
|
|-----|----------------|-----|----------------|-----|----------------|-----|----------------| |
|
|
| 0 | admiration | 1 | amusement | 2 | anger | 3 | annoyance | |
|
|
| 4 | approval | 5 | caring | 6 | confusion | 7 | curiosity | |
|
|
| 8 | desire | 9 | disappointment | 10 | disapproval | 11 | disgust | |
|
|
| 12 | embarrassment | 13 | excitement | 14 | fear | 15 | gratitude | |
|
|
| 16 | grief | 17 | joy | 18 | love | 19 | nervousness | |
|
|
| 20 | optimism | 21 | pride | 22 | realization | 23 | relief | |
|
|
| 24 | remorse | 25 | sadness | 26 | surprise | 27 | neutral | |
|
|
|
|
|
#### How the model was created |
|
|
|
|
|
The model was trained using `AutoModelForSequenceClassification.from_pretrained` with `problem_type="multi_label_classification"` for 3 epochs with a learning rate of 2e-5 and weight decay of 0.01. |
|
|
|
|
|
#### Inference |
|
|
|
|
|
There are multiple ways to use this model in Huggingface Transformers. Possibly the simplest is using a pipeline: |
|
|
|
|
|
### 1. Install dependencies |
|
|
|
|
|
```bash |
|
|
pip install torch transformers |
|
|
``` |
|
|
|
|
|
|
|
|
```python |
|
|
from transformers import pipeline |
|
|
classifier = pipeline(task="text-classification", model="AR04/Senti", top_k=None) |
|
|
sentences = ["hi! u r looking beautiful today dear"] |
|
|
model_outputs = classifier(sentences) |
|
|
print(model_outputs[0]) |
|
|
# produces a list of dicts for each of the labels |
|
|
``` |
|
|
Output: |
|
|
|
|
|
```bash |
|
|
[{'label': 'admiration', 'score': 0.9517803192138672}, {'label': 'love', 'score': 0.18317067623138428}, {'label': 'joy', 'score': 0.03131399303674698}, {'label': 'neutral', 'score': 0.01567094214260578}, {'label': 'surprise', 'score': 0.009232419542968273}, {'label': 'approval', 'score': 0.007308646105229855}, {'label': 'excitement', 'score': 0.006345656234771013}, {'label': 'pride', 'score': 0.004945244640111923}, {'label': 'caring', 'score': 0.0038624939043074846}, {'label': 'realization', 'score': 0.0023580112028867006}, {'label': 'desire', 'score': 0.0017759536858648062}, {'label': 'optimism', 'score': 0.0013220690889284015}, {'label': 'sadness', 'score': 0.001188945840112865}, {'label': 'disappointment', 'score': 0.0009136834414675832}, {'label': 'gratitude', 'score': 0.0008250900427810848}, {'label': 'relief', 'score': 0.0005154621903784573}, {'label': 'amusement', 'score': 0.0004376845608931035}, {'label': 'fear', 'score': 0.00038696840056218207}, {'label': 'embarrassment', 'score': 0.0003084330528508872}, {'label': 'grief', 'score': 0.00019462488126009703}, {'label': 'confusion', 'score': 0.00018893269589170814}, {'label': 'annoyance', 'score': 0.0001587819424457848}, {'label': 'curiosity', 'score': 0.0001355114800389856}, {'label': 'remorse', 'score': 0.00011744408402591944}, {'label': 'anger', 'score': 0.00010586195276118815}, {'label': 'disgust', 'score': 9.386352030560374e-05}, {'label': 'nervousness', 'score': 7.547048153355718e-05}, {'label': 'disapproval', 'score': 3.7117086321813986e-05}] |
|
|
``` |
|
|
|
|
|
#### Evaluation / metrics |
|
|
|
|
|
Here are the evaluation results of **Senti** on the GoEmotions validation set: |
|
|
|
|
|
| Metric | Value | |
|
|
|--------------|-------| |
|
|
| Loss | 0.085 | |
|
|
| F1-score | 0.586 | |
|
|
| ROC AUC | 0.752 | |
|
|
| Accuracy | 0.460 | |
