README.md

---
license: mit
tags:
  - coreml
  - apple-silicon
  - ios
  - beat-this
  - beat-tracking
  - downbeat-tracking
  - music-information-retrieval
  - schism-coreml
---

# Beat This! — Core ML

[CPJKU/beat_this](https://github.com/CPJKU/beat_this) (`final0` checkpoint; "Beat This! Accurate Beat Tracking Without DBN Postprocessing", [Foscarin, Schlüter & Widmer, ISMIR 2024](https://arxiv.org/abs/2407.21658), **MIT licensed**) for Core ML on Apple devices. Frame-wise beat and downbeat tracking at 50 fps with minimal peak-picking postprocessing — no DBN.

Converted from the same verified reference used by the
[schism-mlx](https://github.com/schism-audio/schism-mlx) MLX ports. Two variants per model:

| File | Precision | Compute units | max logit diff |
|---|---|---|---|
| `BeatThis_fp16.mlpackage` | FLOAT16 | ALL (ANE) | 6.7e-2, peak-picked beat/downbeat times identical on tested real drum audio |
| `BeatThis_fp32.mlpackage` | FLOAT32 | CPU+GPU | 5.2e-5, peak-picked beat/downbeat times identical on all tested clips |

Verified on-device-equivalently via coremltools on an M5 Max, against the
reference implementation on real audio. fp16 is ANE-eligible and recommended
for iPhone / iPad; fp32 is the tight-parity fallback.

## Download

`.mlpackage` bundles must be materialized as real files — the Core ML
compiler rejects the symlinks that a default `snapshot_download` creates in
the Hugging Face cache:

```python
from huggingface_hub import snapshot_download
path = snapshot_download("schism-audio/beat-this-coreml", local_dir="./beat-this-coreml")
```

(or `hf download schism-audio/beat-this-coreml --local-dir ./beat-this-coreml`). Swift hosts
downloading files directly are unaffected.

## I/O contract

- input `spect`: `(1, 1500, 128)` float32 — one 30 s chunk of the log-mel frontend (22.05 kHz mono, n_fft 1024, hop 441, 128 slaney mels 30–11000 Hz **without** area norm, magnitude STFT normalized `1/sqrt(n_fft)`, `log1p(1000 x)`), 50 fps
- outputs `beat_logits` / `downbeat_logits`: `(1, 1500)` float32 — frame-wise logits at 50 fps; sigmoid for probabilities, peak-pick for times (local maxima within ±3 frames with logit > 0, adjacent peaks merged by averaging, downbeats snapped to the nearest beat — the reference "minimal" postprocessor)
- longer audio: 1500-frame chunks starting at `-6 + k*1488`, last start shifted left to `n - 1494` so the last chunk ends at the piece end; chunk borders that fall outside the piece are zero-padded on the **spectrogram**; discard the 6 border frames of every chunk, earlier chunks winning on overlap ("keep_first"); frames no chunk covers keep the filler -1000 (see `test_vectors_beatmel.npz` params)
- pieces shorter than 1488 frames (~29.8 s): the reference runs one shorter window that this fixed-shape graph cannot reproduce — attention is global, so zero-padding to 1500 frames changes all logits slightly (beat times on tested real audio were still identical; pad with zeros and drop the padded frames, or prefer the MLX port for exactness)

## DSP frontend (host-side)

The Core ML graph contains the network only. The host implements the log-mel frontend and must match `schism_mlx.analyze.beat_this.model.logmel_beat_this` numerically — `test_vectors_beatmel.npz` in this repo holds deterministic input/output pairs plus the exact slaney filterbank matrix (float32; match within ~1e-4 relative). Its params json documents the full 30 s chunk/stitching contract and the peak-picking recipe. A validated Swift implementation (Accelerate) is available at [schism-audio/schism-dsp](https://github.com/schism-audio/schism-dsp), tested against these exact vectors.

## License

MIT, inherited from the source repository (code and released weights). Model by the [Institute of Computational Perception, JKU Linz](https://github.com/CPJKU); Core ML conversion by [schism-audio](https://huggingface.co/schism-audio).