Hayasuki
/

ChuckleNet-Genki

@@ -8,88 +8,48 @@ sdk_version: 5.34.0
 app_file: app.py
 pinned: false
 license: mit
 ---
-# 🎭 ChuckleNet — Audio Laughter Detection Benchmark
-**The first model that detects and predicts laughter beyond words or language.**
-> Laughter is universal — it transcends language, culture, and geography. ChuckleNet uses audio prosody (pitch, energy, voice quality) to detect laughter, enabling AI systems to understand human joy **regardless of what language is being spoken**.
-## Why Laughter Detection Matters for AI
-Current AI systems (ASR, TTS, conversation AI) struggle with non-verbal emotional signals. Laughter is one of the most important social signals in human conversation — yet it leaves no textual trace.
-**Key use cases:**
-- **Conversational AI** — detect when users are laughing to adapt response tone
-- **Meeting summarization** — mark moments of audience reaction in presentations
-- **Content moderation** — identify comedic or positive content at scale
-- **Dubbing/localization** — detect laughter regions that don't translate across languages
-## How It Works
-```
-Audio → Prosody Features (F0, RMS, ZCR, spectral) → ChuckleNet → Laughter Detection
-```
-No text. No language model. Pure audio signal.
-## Quick Facts
-| Property | Value |
-|---|---|
-| **Input** | 791-dim (WavLM 768-dim + prosody 23-dim) |
-| **Task** | Binary: laugh vs non-laugh per 5-second chunk |
-| **Training data** | 221 English stand-up comedy videos |
-| **Validation F1** | 0.879 ± 0.022 (5-fold GroupKFold) |
-| **Prosody-only F1** | 0.715 (23-dim prosody, no text) |
-| **Language agnostic** | Works on any language — no text required |
-## Key Finding: Prosody Rivals WavLM
-| Model | Input | F1 | Notes |
-|---|---|---|---|
-| Prosody MLP | 23-dim | 0.715 | Simple spectral + F0 features |
-| WavLM MLP | 768-dim | 0.220 | Frozen transformer |
-| **Fusion** | **791-dim** | **0.879** | WavLM + prosody |
-**Insight:** Prosody alone captures 81% of the fusion performance while being **33× smaller** and **language-agnostic**. A frozen WavLM encoder adds only +16% F1 over simple prosody features.
-## Why No Text?
-1. **Laughter is universal** — a laugh sounds the same in English, Mandarin, or Arabic
-2. **Cross-lingual** — works on languages you've never trained on
-3. **Real-time** — no ASR needed, can run on raw audio stream
-4. **Emotion signal** — captures the *how* not the *what* of communication
 ## Limitations
-⚠️ **Chunk-level only:** 5-second fixed windows — not word-level timestamps
-⚠️ **English comedy domain:** Trained on stand-up; generalization varies by domain
-⚠️ **No temporal context:** Each chunk classified independently
-⚠️ **F1 ≠ IoU-F1:** Our F1=0.88 and StandUp4AI's IoU-F1=0.51 are NOT comparable
-## Comparison with Prior Work
-| Work | Metric | Dataset | Notes |
-|---|---|---|---|
-| StandUp4AI (EMNLP 2025) | IoU-F1=0.51 | 330hr, 7 lang | Word-level, requires text |
-| Gillick et al. 2021 | F1=0.75 | Switchboard | Utterance-level |
-| **ChuckleNet (ours)** | **F1=0.88** | **221 videos** | **Audio-only, language-agnostic** |
 ## Citation
 ```bibtex
-@article{chuckleNet,
-  title={ChuckleNet: Audio Laughter Detection Beyond Language},
-  author={Subhajit Das},
-  year={2026},
-  url={https://huggingface.co/Hayasuki/chuckleNet-v2}
 }
 ```
-## Contact
-- **Model:** [Hayasuki/chuckleNet-v2](https://huggingface.co/Hayasuki/chuckleNet-v2)
-- **Demo:** [Hayasuki/a3m-router](https://huggingface.co/spaces/Hayasuki/a3m-router)
-- **Code:** [Das-rebel/autonomous_laughter_prediction](https://github.com/Das-rebel/autonomous_laughter_prediction)

 app_file: app.py
 pinned: false
 license: mit
+short_description: First benchmark for laughter detection beyond words — evaluates on held-out comedians.
 ---
+# 🎭 ChuckleNet — Laughter Detection Benchmark
+**What makes ChuckleNet unique:** The first laughter detection model evaluated on **held-out comedians** (unseen performers), which is the real deployment scenario. Previous works report strong results on seen comedians but don't assess cross-comedian generalization.
+## Benchmark Results
+| Method | Metric | Dataset | Generalization? |
+|--------|---------|---------|------------------|
+| **ChuckleNet (prosody)** | Word F1=0.21 | 40 EMNLP videos | ✅ held-out |
+| **ChuckleNet (fusion)** | IoU-F1=0.30 @ IoU≥0.2 | 40 EMNLP videos | ✅ held-out |
+| StandUp4AI | IoU-F1=0.51 @ IoU≥0.2 | ~255 EMNLP videos | ❌ seen comedians |
+| MTLLFM | Clip F1=0.99 | Proprietary | ❌ seen comedians |
+**Key insight:** Simple prosody features (F0, RMS, ZCR) transfer better across comedians than deep transformer embeddings (WavLM), even though embeddings are 50x higher dimensional.
+## Key Findings
+1. **Prosody > WavLM for cross-comedian transfer**: 5-dim pitch features outperform 768-dim WavLM on held-out comedian evaluation
+2. **Segment-level is easier than word-level**: Our segment-level F1=0.952 vs word-level IoU-F1=0.30 reveals the task difficulty gap
+3. **Positive rate matters**: Models collapse when positive rate < 15%
 ## Limitations
+- **Small evaluation set**: 40 EMNLP videos with word-level labels (only ~22% of 221 available had correct timestamps)
+- **No word-level IoU breakthrough yet**: Best word-level IoU-F1=0.30 vs StandUp4AI's 0.51
+- **Resource-constrained extraction**: Word-level WavLM features couldn't complete on available GPU
+## How It Works
+1. **Prosody extraction** (5-dim): F0 mean/std/max/min/voiced_rate from librosa.pyin()
+2. **Fusion MLP**: 791-dim (WavLM 768 + prosody 23) → 512 → 256 → 64 → 1
+3. **Training**: Class-weighted BCE, GroupKFold by video for proper comedian separation
 ## Citation
 ```bibtex
+@article{chucklenet,
+  title={ChuckleNet: Laughter Detection Beyond Words},
+  author={Das, Subhajit},
+  year={2026}
 }
 ```