The detector, its normalisation, and an honest card

README.md

@@ -0,0 +1,130 @@
+---
+license: apache-2.0
+tags:
+  - audio-classification
+  - voice-activity-detection
+  - radio
+  - onnx
+library_name: onnx
+---
+
+# skywave-host-detect
+
+Tells you when a radio station is talking over the music.
+
+Audio in, a verdict every 80 ms out: `music`, or `speech`. It never sees words,
+so nothing about it is French — but it has only ever heard one station, and that
+matters more than the architecture does. Read the limitations before using it on
+anything else.
+
+620,002 parameters, 2.4 MB, 690× real time on one laptop core.
+
+## What it is for
+
+A music station's metadata says which record is playing. It never says that a
+human has just started speaking over it. If you want to do anything at that
+moment — switch to a lossless copy of the record, subtitle the announcement,
+cut an advert, index a show — you have to detect it from the audio.
+
+The hard part is not speech versus music. It is a **voice over a bed** versus a
+**voice inside the record**. Both have someone talking and music at the same
+time, which is why the usual features do not separate them:
+
+| | margin |
+|---|---|
+| 4 Hz modulation energy (Scheirer & Slaney) | negative — hosts 0.244–0.278, music 0.171–0.282 |
+| spectral flux variance | negative |
+| pitch stability | negative |
+| envelope dynamic range | negative |
+| speaking rate from a transcript | +3.2% |
+| speaker embedding | +5.5% |
+| best AudioSet-derived score | −2.1% |
+
+Every one of those was measured on the same labelled clips, not assumed.
+
+## How it was trained
+
+By distillation, from a pipeline that cannot ship.
+
+That pipeline segments the audio, transcribes every speech region and reasons
+about the words — how fast they arrive, whether they name the station, whether
+they repeat the track's title. It works. It also costs seconds per window, needs
+the language named in advance, and a station whose announcements the transcriber
+mishears becomes a station with no announcements.
+
+So it was used as a teacher. It labelled 28 clean hours of FIP (Paris), and this
+model learned to reach the same verdicts from the sound alone. No hand labelling
+was involved, which is the point: **the ASR you already have is a good enough
+teacher to bootstrap a detector for your own station in an afternoon.** The
+recipe is in the repository.
+
+The student then corrected the teacher on a class of error the teacher cannot
+see. FIP puns on its own name, and the transcriber writes "flippe" or "Philippe";
+the rules reject those for naming no station, and the model catches them without
+needing the word.
+
+## Measured
+
+Two four-hour stretches held out entirely, three seeds:
+
+| | |
+|---|---|
+| announcements found | 32 or 33 of 33 |
+| songs mistaken for the station | 1 of 15 |
+| ordinary music wrongly flagged | 0 of 40 windows, every seed |
+| boundaries | within 0.4 s of the teacher's |
+
+The single miss is the same every seed: a 1.3-second station ident.
+
+These are **end-to-end** numbers, through the segmenter as it runs, with
+smoothing and span merging. The bare network scores better — 27 of 28, 0.93% of
+music windows misread — and quoting that would be quoting a component rather
+than the thing built from it. That mistake was made once here already.
+
+## Limitations
+
+**It has only heard FIP.** The architecture is language-agnostic; the weights
+are not station-agnostic. It learned one production chain, one set of voices,
+one way of mixing a voice over a bed. Expect it to need retraining on your
+station, which is what the recipe is for.
+
+**It cannot tell an announcer from a station ident.** It used to answer three
+classes and was bad at the third. Relabelling the corpus by content rather than
+duration took station dressing from 1.4 minutes to 8, the training windows from
+88 to 1056 — and ident recall moved from 3 of 22 to 6. Almost nothing. What
+separates them is *what is said*, and mel does not carry that. Dropping to two
+classes moved per-window recall from 96.6% to 98.1%.
+
+**Confidence does not predict error.** A floor looks tempting and does not work:
+the corpus's one false positive scores 0.982, above the tenth percentile of real
+announcements (0.939). A 0.95 floor loses four announcements and blocks nothing.
+
+**55 minutes of speech.** That is what it learned from — 19 hours of music and
+55 minutes of someone talking. It works because the problem is narrow, not
+because the data is plentiful.
+
+## Running it
+
+```python
+from radio_host_detect.backends.skywave import SkywaveSegmenter
+
+for label, start, end, confidence in SkywaveSegmenter().detail("chunk.wav"):
+    print(label, start, end, confidence)
+```
+
+Input is log-mel: 16 kHz mono, 400-sample window, 320 hop, 64 bands, 50–7600 Hz.
+The shape is `(batch, 1, time, 64)` and the output `(batch, 2, time/4)` — one
+verdict per four input frames, so every 80 ms. `melspec.py` computes it in plain
+numpy so training and inference cannot drift apart.
+
+## Training data
+
+28 hours of FIP, a French public music station, captured from its public HLS
+stream in August 2026. The audio is Radio France's and is not distributed. The
+labels, the recipe and the tooling are.
+
+## Citation
+
+Built for [Skywave](https://github.com/vdeturckheim/radio-host-detect), an app
+that plays a station's records from Apple Music and hands you back to the
+broadcast when the station speaks.

melspec.py

@@ -0,0 +1,76 @@
+"""Log-mel spectrograms, in one place.
+
+Training and inference must compute this identically. When they do not, the
+model sees a slightly different picture in production than it was taught on and
+degrades in a way that looks like a bad model rather than a bad pipeline — so
+there is one implementation and both sides import it.
+
+Plain numpy, on purpose. The alternative is librosa, which pulls in a large
+dependency tree to compute forty lines of arithmetic, and this has to install
+cleanly on a small droplet.
+"""
+
+from __future__ import annotations
+
+import subprocess
+
+import numpy as np
+
+SR = 16_000
+N_FFT = 400          # 25 ms
+HOP = 320            # 20 ms, so 50 frames a second
+N_MELS = 64
+FMIN, FMAX = 50.0, 7600.0
+FPS = SR / HOP
+
+
+def _filterbank(sr=SR, n_fft=N_FFT, n_mels=N_MELS, fmin=FMIN, fmax=FMAX):
+    def to_mel(f):
+        return 2595.0 * np.log10(1.0 + f / 700.0)
+
+    def to_hz(m):
+        return 700.0 * (10.0 ** (m / 2595.0) - 1.0)
+
+    points = to_hz(np.linspace(to_mel(fmin), to_mel(fmax), n_mels + 2))
+    bins = np.floor((n_fft + 1) * points / sr).astype(int)
+    bank = np.zeros((n_mels, n_fft // 2 + 1), dtype=np.float32)
+    for i in range(n_mels):
+        left, centre, right = bins[i], bins[i + 1], bins[i + 2]
+        centre = max(centre, left + 1)
+        right = max(right, centre + 1)
+        bank[i, left:centre] = np.linspace(0, 1, centre - left, endpoint=False)
+        bank[i, centre:right] = np.linspace(1, 0, right - centre, endpoint=False)
+    return bank
+
+
+BANK = _filterbank()
+
+
+def logmel(samples: np.ndarray) -> np.ndarray:
+    """(frames, 64) log-mel from mono float samples at 16 kHz."""
+    samples = np.ascontiguousarray(samples, dtype=np.float32)
+    if len(samples) < N_FFT:
+        return np.zeros((0, N_MELS), dtype=np.float32)
+    frames = 1 + (len(samples) - N_FFT) // HOP
+    window = np.hanning(N_FFT).astype(np.float32)
+    strided = np.lib.stride_tricks.as_strided(
+        samples, shape=(frames, N_FFT),
+        strides=(samples.strides[0] * HOP, samples.strides[0]))
+    spectrum = np.fft.rfft(strided * window, axis=1)
+    power = (spectrum.real ** 2 + spectrum.imag ** 2).astype(np.float32)
+    # The float32 matmul raises divide-by-zero, overflow and invalid on macOS
+    # regardless of the values — Accelerate's vectorised kernel sets the FP
+    # status flags from lanes past the end of the data. Checked on real audio:
+    # the largest power in a chunk is around 27 and every output is finite.
+    with np.errstate(all="ignore"):
+        energy = power @ BANK.T
+    return np.log(energy + 1e-6).astype(np.float32)
+
+
+def decode(path: str) -> np.ndarray:
+    """Any audio file to mono float32 at 16 kHz, via ffmpeg."""
+    raw = subprocess.run(
+        ["ffmpeg", "-v", "error", "-i", str(path), "-f", "f32le",
+         "-ac", "1", "-ar", str(SR), "-"],
+        capture_output=True, check=True).stdout
+    return np.frombuffer(raw, dtype=np.float32)

skywave-segmenter-norm.npz

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1fb448256ab233da2ea0b495048d7ed0b4035324195b24ceac26f5bc09865e91
+size 1012

skywave-segmenter.json

@@ -0,0 +1,17 @@
+{
+ "classes": [
+  "music",
+  "speech"
+ ],
+ "mode": "stream",
+ "stride": 4,
+ "sampleRate": 16000,
+ "nFft": 400,
+ "hop": 320,
+ "nMels": 64,
+ "heldOut": [
+  "h20",
+  "h12"
+ ],
+ "balancedRecall": 0.9983
+}

skywave-segmenter.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9a2654a39e5622da13ac774cd25d7c0ad19d5acdf02e37facdb8561a6a499c68
+size 2478851