metadata
language: en
license: mit
base_model: declare-lab/segue-w2v2-base
datasets:
- declare-lab/MELD
tags:
- audio
- speech
- sentiment-analysis
- emotion-recognition
- multitask
SEGUE fine-tuned on MELD (multitask sentiment + emotion)
This model is a fine-tuned version of declare-lab/segue-w2v2-base trained jointly on sentiment (3-class) and emotion (7-class) recognition on the MELD dataset (Friends TV show dialogues).
Labels
Sentiment: neutral, positive, negative
Emotion: neutral, surprise, fear, sadness, joy, disgust, anger
Performance (test set)
| Task | Weighted F1 | Macro F1 |
|---|---|---|
| Sentiment | 0.558 | 0.519 |
| Emotion | 0.475 | 0.273 |
Requirements
This model depends on the declare-lab/segue repository, which is not pip-installable. You need to clone it and add it to your path:
git clone https://github.com/declare-lab/segue
# run your scripts from inside the segue/ directory, or:
import sys; sys.path.append('/path/to/segue')
Usage
Download model.pt and inference.py from this repository, then:
from inference import load_segue_multitask, segue_predict
model, processor = load_segue_multitask("model.pt")
sent_probs, emo_probs = segue_predict(model, processor, audio_array, sampling_rate=16000)
Training details
- Base model:
declare-lab/segue-w2v2-base - Dataset: MELD (9989 train / 1109 dev / 2610 test utterances)
- Learning rate: 3e-5, warmup ratio: 0.3, 3 epochs
- Checkpoint averaging: last 10 checkpoints (every 100 steps)
- Multitask loss: 0.5 × sentiment + 0.5 × emotion cross-entropy