---
language: en
license: mit
base_model: declare-lab/segue-w2v2-base
datasets:
  - declare-lab/MELD
tags:
  - audio
  - speech
  - sentiment-analysis
  - emotion-recognition
  - multitask
---

# SEGUE fine-tuned on MELD (multitask sentiment + emotion)

This model is a fine-tuned version of [declare-lab/segue-w2v2-base](https://huggingface.co/declare-lab/segue-w2v2-base)
trained jointly on sentiment (3-class) and emotion (7-class) recognition
on the [MELD dataset](https://github.com/declare-lab/MELD) (Friends TV show dialogues).

## Labels

**Sentiment:** neutral, positive, negative  
**Emotion:** neutral, surprise, fear, sadness, joy, disgust, anger

## Performance (test set)

| Task      | Weighted F1 | Macro F1 |
|-----------|-------------|----------|
| Sentiment | 0.558       | 0.519    |
| Emotion   | 0.475       | 0.273    |

## Requirements

This model depends on the [declare-lab/segue](https://github.com/declare-lab/segue)
repository, which is not pip-installable. You need to clone it and add it to your path:

    git clone https://github.com/declare-lab/segue
    # run your scripts from inside the segue/ directory, or:
    import sys; sys.path.append('/path/to/segue')

## Usage

Download `model.pt` and `inference.py` from this repository, then:

    from inference import load_segue_multitask, segue_predict
    model, processor = load_segue_multitask("model.pt")
    sent_probs, emo_probs = segue_predict(model, processor, audio_array, sampling_rate=16000)

## Training details

- Base model: `declare-lab/segue-w2v2-base`
- Dataset: MELD (9989 train / 1109 dev / 2610 test utterances)
- Learning rate: 3e-5, warmup ratio: 0.3, 3 epochs
- Checkpoint averaging: last 10 checkpoints (every 100 steps)
- Multitask loss: 0.5 × sentiment + 0.5 × emotion cross-entropy