README.md

---
license: apache-2.0
library_name: coreml
tags:
  - audio-classification
  - voice-activity-detection
  - radio
  - coreml
  - mlx
  - swift
pipeline_tag: audio-classification
---

# Skywave Segmenter — small

**Is a person speaking over the music right now?**

Nothing in a radio station's metadata answers that. It says which record is playing,
never that a presenter has started talking over it — and no public dataset labels it
either, which is why this exists.

159,058 parameters. **192 KB** as CoreML at int8. On an M5 Max it takes **3.1 ms** to
label a forty-second window end to end, features included: about 13,000× faster than
the audio arrives.

## What it does

Log-mel in, two classes out — `music` and `speech` — at **12.5 verdicts a second**,
for audio of any length. It is fully convolutional over time, so the same weights
handle a three-second clip and a three-minute one.

"Speech" here means *a human talking over or between records*, which is a narrower
question than voice activity. A sung vocal is music. That distinction is the whole
difficulty: the two things that most resemble a presenter are a rapper and a singer.

## Files

| | |
|---|---|
| `SkywaveSegmenter.mlpackage` | CoreML, int8, what the app runs |
| `skywave-segmenter.safetensors` | the trained weights, MLX layout |
| `skywave-segmenter-norm.safetensors` | per-bin mean and standard deviation |
| `skywave-segmenter.json` | the model card the runtime reads: classes, stride, front end |

The normalisation is folded into the CoreML graph, so a caller feeds **raw log-mel**
and the model normalises internally. The safetensors weights are unnormalised and
expect it applied.

## The front end is part of the model

```
16 kHz mono · 400-sample window · 320-sample hop · 64 mel bands · 50–7600 Hz
log(energy + 1e-6) · 50 frames per second
```

These are not suggestions. The model learned on frames computed exactly this way, and
frames computed any other way make it degrade quietly — which looks like a mediocre
model rather than a mismatch. The reference implementation is
[`SkywaveCore`](https://github.com/vdeturckheim/skywave-model), which is checked
against committed fixtures to 1.5e-04 on every build.

One detail that has cost real accuracy: decode to mono the way `ffmpeg -ac 1` does,
which divides a stereo pair by **√2** rather than by 2 because it normalises to
preserve power. The model reads absolute level, and averaging the channels instead is
three decibels quieter — measured to be the difference between finding an
announcement and reporting silence.

## Results

Held out from training (four-hour chunks `h20` and `h12`), scored **end to end** —
through the smoothing and the span merging, counting announcements a listener would
notice rather than frames:

| | |
|---|---|
| Announcements found | **33 of 33** |
| Sung vocals wrongly flagged | 1 of 15 |
| Ordinary music wrongly flagged | **0 of 40** |

The last row is the one that matters: a false positive pulls a listener off a lossless
track mid-song, and it does not happen.

**Do not quote a per-frame number for this model.** They disagree with the end-to-end
result badly enough to invert a ranking. A 4-bit quantization of this network agrees
with its own float32 self on **81% of individual verdicts** while losing six spans of
eight — it reports 1.36 s of speech where there is 7.28 s. A model can look four
fifths right and detect almost nothing, because the frames it loses are the ones
carrying the announcements.

## Size, and why this one

Twelve architectures were trained, 41k to 2.5M parameters. **None of them can be told
apart**: every one lands at 32–33 of 33. With 33 announcements to be right about, the
benchmark cannot separate an architecture from a seed — three seeds of one
architecture gave 32–33 found and 1–3 sung false positives.

So `small` was chosen on what does not swing: it is small, and its sung false
positives were 1 of 15 in all three seeds where the 41k variant gave 1 to 3. A `tiny`
variant at 41,610 parameters measured equal and is published separately for anything
where 41 KB matters more than margin.

int8 is free here — identical spans to float16, verified through the app's own
inference path. Below eight bits nothing survives.

## Training data, and its limits

28 hours of FIP (Radio France), labelled by an earlier transcribe-and-reason pipeline
whose verdicts this model distils. The audio is not published and the derived corpus is
private: it is a broadcaster's material, not ours.

**This is a distillation of a teacher, not ground truth.** It inherits that pipeline's
judgement, including its mistakes — three of eighteen "hard negatives" in the original
labelling were real station dressing rejected for the wrong reason, because FIP puns
on its own name and the transcriber wrote "flippe" and "Philippe".

**It knows one station.** Trained on French-language presenters over predominantly
anglophone music. On a station where the presenter and the songs share a language, the
harder case, it has not been measured. The held-out split is also thin in one class:
`h12` contains no presenter frames at all, only music and station idents.

**It cannot report a station ident.** The corpus carries three classes — music, host,
ident — and this model collapses the last two, so a jingle and a presenter both come
back as `speech`. A three-class variant trained directly reaches only 25% ident
recall: idents are 0.48% of frames, too thin for the class weighting used.

## Using it

Swift, via [`skywave-model`](https://github.com/vdeturckheim/skywave-model):

```swift
let segmenter = try Segmenter()            // reads the card beside the model
let spans = try segmenter.detail(samples)  // mono 16 kHz Float
for span in spans where span.label != "music" {
    print(span.start, span.end, span.confidence)
}
```

Verdicts become final **2.8 s** after the audio: the receptive field is ±2.16 s,
measured by perturbing one input frame and watching which outputs move, plus half the
1.25 s smoothing vote. Nothing beyond that window changes a verdict, so there is no
reason to hand it more than about twelve seconds at a time.