| --- |
| language: en |
| license: mit |
| base_model: declare-lab/segue-w2v2-base |
| datasets: |
| - declare-lab/MELD |
| tags: |
| - audio |
| - speech |
| - sentiment-analysis |
| - emotion-recognition |
| - multitask |
| --- |
| |
| # SEGUE fine-tuned on MELD (multitask sentiment + emotion) |
|
|
| This model is a fine-tuned version of [declare-lab/segue-w2v2-base](https://huggingface.co/declare-lab/segue-w2v2-base) |
| trained jointly on sentiment (3-class) and emotion (7-class) recognition |
| on the [MELD dataset](https://github.com/declare-lab/MELD) (Friends TV show dialogues). |
|
|
| ## Labels |
|
|
| **Sentiment:** neutral, positive, negative |
| **Emotion:** neutral, surprise, fear, sadness, joy, disgust, anger |
|
|
| ## Performance (test set) |
|
|
| | Task | Weighted F1 | Macro F1 | |
| |-----------|-------------|----------| |
| | Sentiment | 0.558 | 0.519 | |
| | Emotion | 0.475 | 0.273 | |
|
|
| ## Requirements |
|
|
| This model depends on the [declare-lab/segue](https://github.com/declare-lab/segue) |
| repository, which is not pip-installable. You need to clone it and add it to your path: |
|
|
| git clone https://github.com/declare-lab/segue |
| # run your scripts from inside the segue/ directory, or: |
| import sys; sys.path.append('/path/to/segue') |
| |
| ## Usage |
|
|
| Download `model.pt` and `inference.py` from this repository, then: |
|
|
| from inference import load_segue_multitask, segue_predict |
| model, processor = load_segue_multitask("model.pt") |
| sent_probs, emo_probs = segue_predict(model, processor, audio_array, sampling_rate=16000) |
| |
| ## Training details |
|
|
| - Base model: `declare-lab/segue-w2v2-base` |
| - Dataset: MELD (9989 train / 1109 dev / 2610 test utterances) |
| - Learning rate: 3e-5, warmup ratio: 0.3, 3 epochs |
| - Checkpoint averaging: last 10 checkpoints (every 100 steps) |
| - Multitask loss: 0.5 × sentiment + 0.5 × emotion cross-entropy |
|
|
