Voice Turn Detection
This audio classifier estimates whether a speaker has completed a conversational turn. It is trained for English, Hindi and Hinglish speech and does not require transcription.
Model
The model keeps the Whisper Tiny encoder, trims its positional context to eight seconds, pools the 400 encoder frames with learned scalar attention and predicts one completion logit through a small MLP. The exported ONNX graph includes log-Mel extraction and returns a sigmoid probability.
Input and output
- Input name:
waveform - Input shape:
[batch, 128000] - Input format: mono float32 PCM at 16 kHz
- Output name:
p_complete - Output shape:
[batch]
Keep the latest eight seconds. Left-pad short audio with zeros so the newest sample stays on the right edge.
Evaluation
The frozen model was evaluated on all 9,104 English and Hindi clips in pipecat-ai/smart-turn-data-v3.2-test.
| Slice | Clips | Accuracy at 0.5 | ROC AUC |
|---|---|---|---|
| Overall | 9,104 | 93.70% | 0.9826 |
| Hindi | 1,284 | 93.93% | 0.9853 |
| English | 7,820 | 93.66% | 0.9820 |
| Real speech | 5,367 | 94.37% | 0.9862 |
| Synthetic speech | 3,737 | 92.72% | 0.9766 |
Limits
The training data is synthetic-heavy and the model is limited to the speech conditions represented in that dataset. Calibrate the threshold on deployment traffic. A higher threshold reduces interruptions but increases response delay.