README.md

metadata

title: SpotSound Temporal Grounding
emoji: 🔍
colorFrom: indigo
colorTo: pink
sdk: gradio
sdk_version: 6.24.0
app_file: app.py
short_description: Find when a described sound happens in a recording
python_version: '3.12'
startup_duration_timeout: 1h

SpotSound — fine-grained audio temporal grounding

Demo of SpotSound (Sun et al., 2026), a LoRA on top of NVIDIA Audio Flamingo 3 that gives large audio–language models fine-grained temporal grounding: given a long, untrimmed recording and a natural-language query, it answers with the start/end timestamps of the described sound event.

• Paper: https://huggingface.co/papers/2604.13023
• Project page: https://loiesun.github.io/spotsound/
• Code: https://github.com/LoieSun/SpotSound
• Benchmark: https://huggingface.co/datasets/Loie/SpotSound-Bench

How it works

spotsound.py ports the two pieces of the official implementation onto current transformers:

1. Timestamp-interleaved audio tokens — the <sound> placeholder is expanded to "timestamp: t seconds; feature: " + <sound> × 25 for every second of audio (AF3 produces exactly 25 post-pool audio tokens per second), so the language model can read absolute time off the audio stream.
2. Tolerant audio scatter — the audio features are scattered into the audio-token slots without the base class's strict #tokens == #features check, since the interleaving keeps floor(N / 25) × 25 of the N features.

The published LoRA adapter is merged into the base model at startup and served on ZeroGPU.

Examples

The four example clips come from Loie/SpotSound-Bench (MIT licensed), the paper's own "needle-in-a-haystack" benchmark.

Licensing

The SpotSound adapter and benchmark are MIT licensed. The base model, nvidia/audio-flamingo-3-hf, is released under the NVIDIA OneWay Noncommercial License — this demo is for research/non-commercial use.