README.md

---
license: cc-by-4.0
tags: [audio-classification, vocal-bursts, non-verbal, voiceclap]
pretty_name: VocalBurst Classifier (single-burst)
---

# VocalBurst Classifier — single-burst

A sharper classifier specialised for clips that contain **exactly one isolated vocal burst** (no speech).
Same taxonomy (82 classes + `no_burst`). **Softmax** (single most-likely type). Use it as the "name the
burst" stage after a detector has isolated a segment.

> ⚠️ **Taxonomy patch (label mapping only, weights unchanged):** the three classes originally trained at
> indices **79 `Hand Scratching Head`**, **80 `Hand Slaps`** and **81 `Slap Face`** are now labelled
> **`No burst`** because of excessive false positives in production. All 83 logits and indices are
> intact. See [Taxonomy patch](#taxonomy-patch-indices-79--80--81-are-now-labelled-no-burst) before
> using raw logits.

## Architecture
`laion/voiceclap-commercial` (768-d, frozen) → **MLP 1024-wide × 3 deep** → 83-way softmax. ~3M params.

## Training
Trained ONLY on **clean single-burst clips**: Gemini-confirmed DACVAE positives (82 classes, exactly one
label, no speech) + pure Emilia `no_burst` clips (~15.9k). Cleaner labels, no speech confusion → sharper.
50 epochs, AdamW, cross-entropy, best-mAP checkpoint.

> **v2 (current):** retrained with **9,000 multilingual burst-free negatives** (FLEURS: Chinese, Hindi, Bengali, Arabic, Persian, Urdu, Tamil, Telugu, Vietnamese, Thai, Indonesian, Japanese, Korean, Swahili, Yoruba, Zulu, Turkish, Russian) — fixes hallucinated bursts on non-European speech.

## Results (held-out pure single-burst clips)
| metric | fine (82) | coarse (16 families) |
|---|---|---|
| macro mAP | **0.68** | **0.87** |
| top-1 exact | 60% | – |
| a true label in top-3 | 88% | – |
On isolated bursts it names the coarse family very reliably (~0.87) and the fine subtype well.

## How it works
Two stages: a **frozen** `laion/voiceclap-commercial` audio encoder turns a clip into a **768-d embedding**
(`encode_waveform`, auto-downloaded — the repo needs no extra setup), then this small trained **MLP head**
maps it to **83 outputs = 82 VocalBurst classes + `no_burst`** (taxonomy: [LAION-AI/voice-taxonomies · vocalburst](https://github.com/LAION-AI/voice-taxonomies/tree/main/vocalburst)).
A **no-burst gate**: if `P(no_burst) ≥ 0.5` the clip is declared burst-free (no false alarm); otherwise the top classes are returned. Clips are truncated to the first 30 s (the encoder's window).

## Usage
```python
from inference import VocalBurstClassifier
clf = VocalBurstClassifier("laion/vocalburst-classifier-single")      # HF repo id, or a local checkout dir
print(clf.predict("clip.wav"))        # -> {no_burst, p_no_burst, top1, predictions:[(class,prob)], group}
```

## Taxonomy patch: indices 79 / 80 / 81 are now labelled `No burst`

> **This section supersedes the earlier "Known caveat: `Slap Face` false positives" note.**
> That note recommended skipping `Slap Face` as top-1 and falling through to the runner-up.
> The problem turned out to be broader than one class and is now fixed in the label mapping
> itself, so the manual skip rule is **no longer needed** — do not apply it on top of this.

### What changed

The model was **originally trained** with three fine classes in the
`hand_and_body_sounds` family:

| index | original class name (as trained) | original group |
|---|---|---|
| **79** | `Hand Scratching Head` | `hand_and_body_sounds` |
| **80** | `Hand Slaps`           | `hand_and_body_sounds` |
| **81** | `Slap Face`            | `hand_and_body_sounds` |

In large-scale production use — re-classifying ~1.5 M CrisperWhisper-detected burst
timestamps from in-the-wild expressive speech — all three fired as top-1 far more often
than is plausible, and manual spot-checks showed the great majority were **false
positives**: the true event was usually a laugh, gasp, sigh, grunt, or simply no burst at
all. `Slap Face` was the worst offender, but `Hand Slaps` and `Hand Scratching Head`
behaved the same way. These three are percussive, broadband, very short events, and the
frozen embedder appears to map a lot of unrelated transient energy (mouth clicks, mic
bumps, plosives, clipping) into that corner of the space.

Accordingly, in `classes.json` **all three labels have been replaced with `"No burst"`**,
and `class_to_group.json` maps `"No burst"` to the group `no_burst`.

### What did *not* change

**The weights were not retrained and not altered.** `model.pt` is bit-identical to the
previous revision. The head still emits **83 logits** and every index keeps its original
position, so **existing checkpoints, cached embeddings, cached logits and stored
`predicted_index` values all stay valid**. This is purely a relabelling of the output
mapping.

### What this means if you use the raw logits

* `argmax` still returns an integer in `[0, 82]`. Indices 79, 80 and 81 now *decode* to
  the string `"No burst"` instead of the three hand-sound names.
* `"No burst"` therefore appears **three times** in `classes.json`. Do not use
  `classes.index("No burst")` to recover an index — it will always give 79. Always keep
  the integer index as the identity and use the list only for display.
* Index **82** is still the model's own dedicated `no_burst` class, spelled
  `no_burst` (lowercase, underscore) to keep the pre-existing gate code working:
  `P(no_burst) ≥ 0.5` still refers to index 82 alone.
* The full set of "this is not a burst" outcomes is therefore
  `{79, 80, 81, 82}`. If you want a single boolean, test membership in that set
  (or test `class_to_group[label] == "no_burst"`), not equality against one index.
* If you need the original three-way hand-sound distinction back (e.g. your domain really
  does contain slaps and you have evaluated it), the information is still in the logits —
  just restore the names at indices 79/80/81 in your own copy of `classes.json`.
* The coarse family `hand_and_body_sounds` now contains only `Finger Snaps` (index 78).

```python
probs = model(emb).softmax(-1)
idx   = int(probs.argmax(-1))          # keep the INDEX as the identity
label = classes[idx]                   # 79/80/81 -> "No burst"
is_burst = idx not in (79, 80, 81, 82)
```

This is an observation from downstream use on in-the-wild speech, not a formal error
analysis on the model's own held-out split — the reported metrics above were computed
**before** this relabelling and still refer to the original 83-way taxonomy.

## Files
`model.pt` (MLP weights, unchanged) · `config.json` (arch) · `classes.json` (83 labels, index order;
79/80/81 relabelled `No burst`) · `class_to_group.json` (fine→coarse families) · `inference.py` ·
`example.py` · `requirements.txt`.

## Interactive demos (audio + predictions)
- Multi-label predictions vs ground truth: https://projects.laion.ai/procedural-voice-captions/vocalburst-predictions/
- Single-burst classifier: https://projects.laion.ai/procedural-voice-captions/vocalburst-single/
- Two-stage detect→classify combo: https://projects.laion.ai/procedural-voice-captions/vocalburst-combo/
- On MOSS character voices: https://projects.laion.ai/procedural-voice-captions/vocalburst-character/

Training data + embeddings: [laion/vocalburst-classification](https://huggingface.co/datasets/laion/vocalburst-classification).
Embedder: [laion/voiceclap-commercial](https://huggingface.co/laion/voiceclap-commercial). License: CC-BY-4.0.