---
library_name: transformers
license: bsd-3-clause
base_model: Simon-Kotchou/ssast-tiny-patch-audioset-16-16
tags:
- audio-classification
- speech-emotion-recognition
- yoruba
- code-switching
- multilingual
- ssast
- nigeria
datasets:
- Professor/YORENG100
metrics:
- accuracy
model-index:
- name: YORENG100-SSAST-Emotion-Recognition
  results:
  - task:
      type: audio-classification
      name: Speech Emotion Recognition
    dataset:
      name: YORENG100 (Yoruba-English Code-Switching)
      type: custom
    metrics:
    - type: accuracy
      value: 0.9645
      name: Test Accuracy
pipeline_tag: audio-classification
---

# YORENG100-SSAST-Emotion-Recognition

This model is a specialized Speech Emotion Recognition (SER) system fine-tuned from the **SSAST (Self-Supervised Audio Spectrogram Transformer)**. It is specifically designed to handle the linguistic complexities of **Yoruba-English code-switching**, common in Nigerian urban centers.

## 🌟 Model Highlights
- **Architecture:** SSAST-Tiny (16x16 Patches)
- **Dataset:** YORENG100 (100 hours of curated Yoruba-English bilingual speech)
- **Top Accuracy:** 96.45% on validation set.
- **Innovation:** Captures emotional prosody across language boundaries where traditional monolingual models often fail.



## 🚀 How to Use (Manual Inference)

Since SSAST requires specific tensor dimension handling for its patch-based architecture, we recommend the following manual inference style over the standard `pipeline`.

```python
import torch
import librosa
from transformers import AutoFeatureExtractor, ASTForAudioClassification

# 1. Load Model & Processor
model_id = "Professor/YORENG100-SSAST-Emotion-Recognition"
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

feature_extractor = AutoFeatureExtractor.from_pretrained(model_id)
model = ASTForAudioClassification.from_pretrained(model_id).to(device)
model.eval()

# 2. Prepare Audio (Resample to 16kHz)
audio_path = "path_to_your_audio.wav"
speech, _ = librosa.load(audio_path, sr=16000)

# 3. Preprocess & Predict
inputs = feature_extractor(speech, sampling_rate=16000, return_tensors="pt")
# SSAST Fix: Ensure [Batch, Time, Freq] shape
input_values = inputs.input_values.squeeze().unsqueeze(0).to(device)

with torch.no_grad():
    logits = model(input_values).logits
    prediction = torch.argmax(logits, dim=-1).item()

print(f"Predicted Emotion: {model.config.id2label[prediction]}")

```

## 📊 Training Results

The model was trained for 3 epochs on a TPU-v3-8. Note the high stability in accuracy even as loss continued to decrease.

| Epoch | Step | Training Loss | Validation Loss | Accuracy |
| --- | --- | --- | --- | --- |
| 1.0 | 2501 | 0.2156 | 0.2080 | 0.9645 |
| 2.0 | 5002 | 0.1943 | 0.2044 | 0.9645 |
| 3.0 | 7503 | 0.1842 | 0.2023 | 0.9645 |

## 🌍 Background: Yoruba-English Emotion Recognition

Emotion recognition in tonal languages like **Yoruba** is uniquely challenging because pitch (F0) is used to distinguish word meanings (lexical tone) as well as emotional state. When speakers code-switch into English, the "emotional acoustic profile" shifts mid-sentence.

This model uses the patch-based attention mechanism of SSAST to focus on local time-frequency features that are invariant to these language shifts.

## 🛠 Limitations

* **Background Noise:** Not yet robust to high-noise environments (e.g., market or traffic noise).
* **Sampling Rate:** Only supports 16kHz audio input.
* **Labels:** Performance is optimized for [Angry, Happy, Sad, Neutral, etc.].

## 📜 Citation

If you use this model or the YORENG100 dataset, please cite us