---
license: cc-by-4.0
tags: [audio-classification, vocal-bursts, non-verbal, voiceclap, multi-label]
pretty_name: VocalBurst Classifier (multi-label)
---

# VocalBurst Classifier — multi-label

A **presence classifier** for non-speech **vocal bursts** (laughs, sighs, gasps, coughs, groans, throat
sounds, …) over the **82-class** LAION VocalBurst taxonomy + `no_burst`. **Multi-label** (independent
sigmoid per class) — a clip can contain several bursts. Works on pure bursts and bursts embedded in speech.

## Architecture
`laion/voiceclap-commercial` (768-d, frozen) → **MLP 2048-wide × 4 deep** (LayerNorm+GELU+dropout) → 83 sigmoids. ~14.4M trainable params.

## Training
- **Positives:** Gemini-confirmed single bursts (DACVAE, 82 classes) + **real speech-with-bursts** (gemini_pro_25, multi-label).
- **Negatives / speech carriers:** Emilia clips filtered burst-free by `laion/vocalburst-locator`.
- **~97k samples**: pure bursts + composites (burst + speech carriers + a 2nd burst) + 25% overlays (burst mixed over speech) + neg+neg. The speech carriers carry no label, so the head must localise the burst, not memorise audio. Balanced (≤1000/class); 50 epochs, AdamW, BCE with per-class pos-weight, best-mAP checkpoint.

> **v2 (current):** retrained with **9,000 multilingual burst-free negatives** (FLEURS: Chinese, Hindi, Bengali, Arabic, Persian, Urdu, Tamil, Telugu, Vietnamese, Thai, Indonesian, Japanese, Korean, Swahili, Yoruba, Zulu, Turkish, Russian) — fixes hallucinated bursts on non-European speech.

## Results (Gemini-mixed validation)
| metric | fine (82) | coarse (16 families) |
|---|---|---|
| macro mAP | 0.39 | **0.64** |
| top-1 exact | 49% | – |
| a true label in top-3 | 77% | – |
Best when you take the **top-1** prediction (which the two-stage combo demo uses). Much stronger at the coarse "which family of sound" level than the exact fine subtype.

## How it works
Two stages: a **frozen** `laion/voiceclap-commercial` audio encoder turns a clip into a **768-d embedding**
(`encode_waveform`, auto-downloaded — the repo needs no extra setup), then this small trained **MLP head**
maps it to **83 outputs = 82 VocalBurst classes + `no_burst`** (taxonomy: [LAION-AI/voice-taxonomies · vocalburst](https://github.com/LAION-AI/voice-taxonomies/tree/main/vocalburst)).
A **no-burst gate**: if `P(no_burst) ≥ 0.5` the clip is declared burst-free (no false alarm); otherwise the top classes are returned. Clips are truncated to the first 30 s (the encoder's window).

## Usage
```python
from inference import VocalBurstClassifier
clf = VocalBurstClassifier("laion/vocalburst-classifier-multilabel")      # HF repo id, or a local checkout dir
print(clf.predict("clip.wav"))        # -> {no_burst, p_no_burst, top1, predictions:[(class,prob)], group}
```

## Files
`model.pt` (MLP weights) · `config.json` (arch) · `classes.json` (83 labels, index order) ·
`class_to_group.json` (fine→16 coarse families) · `inference.py` · `example.py` · `requirements.txt`.

## Interactive demos (audio + predictions)
- Multi-label predictions vs ground truth: https://projects.laion.ai/procedural-voice-captions/vocalburst-predictions/
- Single-burst classifier: https://projects.laion.ai/procedural-voice-captions/vocalburst-single/
- Two-stage detect→classify combo: https://projects.laion.ai/procedural-voice-captions/vocalburst-combo/
- On MOSS character voices: https://projects.laion.ai/procedural-voice-captions/vocalburst-character/

Training data + embeddings: [laion/vocalburst-classification](https://huggingface.co/datasets/laion/vocalburst-classification).
Embedder: [laion/voiceclap-commercial](https://huggingface.co/laion/voiceclap-commercial). License: CC-BY-4.0.