README.md

---
title: SpotSound Temporal Grounding
emoji: 🔍
colorFrom: indigo
colorTo: pink
sdk: gradio
sdk_version: 6.24.0
app_file: app.py
short_description: Find when a described sound happens in a recording
python_version: '3.12'
startup_duration_timeout: 1h
---

# SpotSound — fine-grained audio temporal grounding

Demo of **[SpotSound](https://huggingface.co/Loie/SpotSound)** (Sun et al., 2026), a LoRA on top of
[NVIDIA Audio Flamingo 3](https://huggingface.co/nvidia/audio-flamingo-3-hf) that gives large
audio–language models fine-grained temporal grounding: given a long, untrimmed recording and a
natural-language query, it answers with the start/end timestamps of the described sound event.

* Paper: <https://huggingface.co/papers/2604.13023>
* Project page: <https://loiesun.github.io/spotsound/>
* Code: <https://github.com/LoieSun/SpotSound>
* Benchmark: <https://huggingface.co/datasets/Loie/SpotSound-Bench>

## How it works

`spotsound.py` ports the two pieces of the official implementation onto current `transformers`:

1. **Timestamp-interleaved audio tokens** — the `<sound>` placeholder is expanded to
   `"timestamp: t seconds; feature: " + <sound> × 25` for every second of audio (AF3 produces
   exactly 25 post-pool audio tokens per second), so the language model can read absolute time
   off the audio stream.
2. **Tolerant audio scatter** — the audio features are scattered into the audio-token slots
   without the base class's strict `#tokens == #features` check, since the interleaving keeps
   `floor(N / 25) × 25` of the `N` features.

The published LoRA adapter is merged into the base model at startup and served on ZeroGPU.

## Examples

The four example clips come from
[Loie/SpotSound-Bench](https://huggingface.co/datasets/Loie/SpotSound-Bench) (MIT licensed),
the paper's own "needle-in-a-haystack" benchmark.

## Licensing

The SpotSound adapter and benchmark are MIT licensed. The base model,
`nvidia/audio-flamingo-3-hf`, is released under the NVIDIA OneWay Noncommercial License — this
demo is for research/non-commercial use.