DistilHuBERT SER (Speech Emotion Recognition)
Arousal-valence dimensional emotion recognition from speech, built on ntu-spml/distilhubert.
Model Details
- Backbone: DistilHuBERT (23.5M params)
- Head: Linear(768 → 256) → GELU → Dropout(0.3) → Linear(256 → 2) → Tanh
- Output: arousal ∈ [-1, 1], valence ∈ [-1, 1]
- Input: Raw 16kHz waveform, variable length
- Training data: CREMA-D (7,442 samples, 91 speakers)
- Loss: Concordance Correlation Coefficient (CCC)
- Best CCC: arousal=0.783, valence=0.737, avg=0.760
Files
| File | Size | Description |
|---|---|---|
distilhubert_ser.onnx |
90 MB | fp32 ONNX model |
distilhubert_ser_int8.onnx |
48 MB | INT8 dynamic quantized (deployment) |
Usage (ONNX Runtime)
import onnxruntime as ort
import numpy as np
session = ort.InferenceSession(
"distilhubert_ser_int8.onnx", providers=["CPUExecutionProvider"]
)
# audio: raw 16 kHz mono waveform, float32, peak-normalized, shape [1, N]
audio = waveform.astype(np.float32)[None, :]
arousal, valence = session.run(["arousal", "valence"], {"audio": audio})
# arousal, valence: float32 [1], each in [-1, 1]
Training
Trained with CCC loss on CREMA-D categorical emotions mapped to dimensional centroids:
- ANG → arousal=0.8, valence=-0.6
- DIS → arousal=0.3, valence=-0.7
- FEA → arousal=0.7, valence=-0.5
- HAP → arousal=0.6, valence=0.7
- NEU → arousal=0.0, valence=0.0
- SAD → arousal=-0.5, valence=-0.4
30 epochs, batch size 16, lr=1e-4, AdamW.
Provenance and License
Full provenance: provenance.json in this repo.
Apache-2.0, matching the ntu-spml/distilhubert backbone this model fine-tunes. (This card previously said MIT; corrected 2026-07-25 to match the backbone's license and the Omote model registry.)