---
tags:
- audio-classification
- speech-emotion-recognition
- tensorflow
- keras
- emotion2vec
language:
- en
license: apache-2.0
metrics:
- accuracy
---

# Speech Emotion Recognition (SER) System

## Overview
Production-quality Speech Emotion Recognition detecting **6 core emotions** from voice/audio:
- **Angry** | **Disgust** | **Fear** | **Happy** | **Neutral** | **Sad**

## Architecture
**Fusion Model**: CNN + BiLSTM + Multi-Head Self-Attention (spectrogram features) + emotion2vec embeddings

### Feature Pipeline
| Feature | Dimensions |
|---------|-----------|
| Mel Spectrogram | 128 bands |
| MFCC | 40 coefficients |
| Zero Crossing Rate | 1 |
| RMS Energy | 1 |
| **Total** | **170 × 200 → (170, 200, 1)** |
| emotion2vec embedding | 768-dim |

### Training Data
- **CREMA-D**: 7,442 clips, 91 actors (train/val/test split provided)
- **RAVDESS**: 1,056 speech clips, 24 actors (70/15/15 split)
- **Augmentation**: pitch shift, time stretch, Gaussian noise, SpecAugment

## Results

| Model | Val Accuracy | Test Accuracy |
|-------|-------------|---------------|
| CNN+BiLSTM+Attention | 56.0% | 59.2% |
| **Fusion (CNN + emotion2vec)** | **53.2%** | **54.9%** |
| Human baseline (audio-only) | - | 40.9% |

**Best: Model 1 — 59.2% test accuracy (+18.3pp over human baseline)**

## Quick Start

```bash
pip install tensorflow librosa numpy funasr modelscope
```

```python
from predict import predict_emotion

label, confidence, probs = predict_emotion("audio.wav", model_dir="./outputs")
# Prints: Predicted Emotion: HAPPY, Confidence: 87.3%
```

## Download & Use Locally

```bash
# Clone the repo
git lfs install
git clone https://huggingface.co/SamOp224/speech-emotion-recognition
cd speech-emotion-recognition

# Run prediction
python outputs/predict.py your_audio.wav outputs
```

## Files
- `outputs/fusion_model.keras` — Fusion model (best)
- `outputs/model1_cnn_bilstm_attn.keras` — CNN+BiLSTM+Attention standalone
- `outputs/predict.py` — Prediction script with visualization
- `outputs/config.json` — Configuration and results