Upload folder using huggingface_hub
Browse files- BeatThis_fp16.mlpackage/Data/com.apple.CoreML/model.mlmodel +3 -0
- BeatThis_fp16.mlpackage/Data/com.apple.CoreML/weights/weight.bin +3 -0
- BeatThis_fp16.mlpackage/Manifest.json +18 -0
- BeatThis_fp32.mlpackage/Data/com.apple.CoreML/model.mlmodel +3 -0
- BeatThis_fp32.mlpackage/Data/com.apple.CoreML/weights/weight.bin +3 -0
- BeatThis_fp32.mlpackage/Manifest.json +18 -0
- README.md +88 -0
- test_vectors_beatmel.npz +3 -0
BeatThis_fp16.mlpackage/Data/com.apple.CoreML/model.mlmodel
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:db6e5a91cdd5798e6627ebc0cf54505eb79652010c17cab18ceaa11524e8c0a3
|
| 3 |
+
size 235611
|
BeatThis_fp16.mlpackage/Data/com.apple.CoreML/weights/weight.bin
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:0bd7336ed1c7b9cd746edea148bb64530c2e3a4f7c7b1ea780ffc110fdc84d4b
|
| 3 |
+
size 40718144
|
BeatThis_fp16.mlpackage/Manifest.json
ADDED
|
@@ -0,0 +1,18 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"fileFormatVersion": "1.0.0",
|
| 3 |
+
"itemInfoEntries": {
|
| 4 |
+
"97F48BF4-AE1D-48C1-91BA-1964DEBD9544": {
|
| 5 |
+
"author": "com.apple.CoreML",
|
| 6 |
+
"description": "CoreML Model Weights",
|
| 7 |
+
"name": "weights",
|
| 8 |
+
"path": "com.apple.CoreML/weights"
|
| 9 |
+
},
|
| 10 |
+
"9FBC2379-00F8-4BFB-9258-1AF917D67E30": {
|
| 11 |
+
"author": "com.apple.CoreML",
|
| 12 |
+
"description": "CoreML Model Specification",
|
| 13 |
+
"name": "model.mlmodel",
|
| 14 |
+
"path": "com.apple.CoreML/model.mlmodel"
|
| 15 |
+
}
|
| 16 |
+
},
|
| 17 |
+
"rootModelIdentifier": "9FBC2379-00F8-4BFB-9258-1AF917D67E30"
|
| 18 |
+
}
|
BeatThis_fp32.mlpackage/Data/com.apple.CoreML/model.mlmodel
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:0fa10d898241fa400201bfdec0b420b1cac1ac7fd2644eaf29850a6105800775
|
| 3 |
+
size 207858
|
BeatThis_fp32.mlpackage/Data/com.apple.CoreML/weights/weight.bin
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:fd175cdec237a272da9804b09be7a4267886b733c61b904c680d02809f8496f6
|
| 3 |
+
size 81426304
|
BeatThis_fp32.mlpackage/Manifest.json
ADDED
|
@@ -0,0 +1,18 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"fileFormatVersion": "1.0.0",
|
| 3 |
+
"itemInfoEntries": {
|
| 4 |
+
"2EA572E6-9CD9-4D98-83CB-4523A6DE302B": {
|
| 5 |
+
"author": "com.apple.CoreML",
|
| 6 |
+
"description": "CoreML Model Specification",
|
| 7 |
+
"name": "model.mlmodel",
|
| 8 |
+
"path": "com.apple.CoreML/model.mlmodel"
|
| 9 |
+
},
|
| 10 |
+
"5EEA07D7-D290-4F63-B160-059DF953724A": {
|
| 11 |
+
"author": "com.apple.CoreML",
|
| 12 |
+
"description": "CoreML Model Weights",
|
| 13 |
+
"name": "weights",
|
| 14 |
+
"path": "com.apple.CoreML/weights"
|
| 15 |
+
}
|
| 16 |
+
},
|
| 17 |
+
"rootModelIdentifier": "2EA572E6-9CD9-4D98-83CB-4523A6DE302B"
|
| 18 |
+
}
|
README.md
ADDED
|
@@ -0,0 +1,88 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: mit
|
| 3 |
+
tags:
|
| 4 |
+
- coreml
|
| 5 |
+
- apple-silicon
|
| 6 |
+
- ios
|
| 7 |
+
- beat-this
|
| 8 |
+
- beat-tracking
|
| 9 |
+
- downbeat-tracking
|
| 10 |
+
- music-information-retrieval
|
| 11 |
+
- schism-coreml
|
| 12 |
+
---
|
| 13 |
+
|
| 14 |
+
# Beat This! — Core ML
|
| 15 |
+
|
| 16 |
+
[CPJKU/beat_this](https://github.com/CPJKU/beat_this) (`final0` checkpoint;
|
| 17 |
+
"Beat This! Accurate Beat Tracking Without DBN Postprocessing",
|
| 18 |
+
[Foscarin, Schlüter & Widmer, ISMIR 2024](https://arxiv.org/abs/2407.21658),
|
| 19 |
+
**MIT licensed**) for Core ML on Apple devices. Frame-wise beat and downbeat
|
| 20 |
+
tracking at 50 fps with minimal peak-picking postprocessing — no DBN.
|
| 21 |
+
|
| 22 |
+
Converted from the same verified reference used by the
|
| 23 |
+
[schism-mlx](https://github.com/schism-audio/schism-mlx) MLX ports. Two variants per model:
|
| 24 |
+
|
| 25 |
+
| File | Precision | Compute units | max logit diff |
|
| 26 |
+
|---|---|---|---|
|
| 27 |
+
| `BeatThis_fp16.mlpackage` | FLOAT16 | ALL (ANE) | 6.7e-2, peak-picked beat/downbeat times identical on tested real drum audio |
|
| 28 |
+
| `BeatThis_fp32.mlpackage` | FLOAT32 | CPU+GPU | 5.2e-5, peak-picked beat/downbeat times identical on all tested clips |
|
| 29 |
+
|
| 30 |
+
Verified on-device-equivalently via coremltools on an M5 Max, against the
|
| 31 |
+
reference implementation on real audio. fp16 is ANE-eligible and recommended
|
| 32 |
+
for iPhone / iPad; fp32 is the tight-parity fallback.
|
| 33 |
+
|
| 34 |
+
## Download
|
| 35 |
+
|
| 36 |
+
`.mlpackage` bundles must be materialized as real files — the Core ML
|
| 37 |
+
compiler rejects the symlinks that a default `snapshot_download` creates in
|
| 38 |
+
the Hugging Face cache:
|
| 39 |
+
|
| 40 |
+
```python
|
| 41 |
+
from huggingface_hub import snapshot_download
|
| 42 |
+
path = snapshot_download("schism-audio/beat-this-coreml", local_dir="./beat-this-coreml")
|
| 43 |
+
```
|
| 44 |
+
|
| 45 |
+
(or `hf download schism-audio/beat-this-coreml --local-dir ./beat-this-coreml`). Swift hosts
|
| 46 |
+
downloading files directly are unaffected.
|
| 47 |
+
|
| 48 |
+
## I/O contract
|
| 49 |
+
|
| 50 |
+
- input `spect`: `(1, 1500, 128)` float32 — one 30 s chunk of the log-mel
|
| 51 |
+
frontend (22.05 kHz mono, n_fft 1024, hop 441, 128 slaney mels
|
| 52 |
+
30–11000 Hz **without** area norm, magnitude STFT normalized
|
| 53 |
+
`1/sqrt(n_fft)`, `log1p(1000 x)`), 50 fps
|
| 54 |
+
- outputs `beat_logits` / `downbeat_logits`: `(1, 1500)` float32 —
|
| 55 |
+
frame-wise logits at 50 fps; sigmoid for probabilities, peak-pick for
|
| 56 |
+
times (local maxima within ±3 frames with logit > 0, adjacent peaks
|
| 57 |
+
merged by averaging, downbeats snapped to the nearest beat — the
|
| 58 |
+
reference "minimal" postprocessor)
|
| 59 |
+
- longer audio: 1500-frame chunks starting at `-6 + k·1488`, last start
|
| 60 |
+
shifted left to `n − 1494` so the last chunk ends at the piece end;
|
| 61 |
+
chunk borders that fall outside the piece are zero-padded on the
|
| 62 |
+
**spectrogram**; discard the 6 border frames of every chunk, earlier
|
| 63 |
+
chunks winning on overlap ("keep_first"); frames no chunk covers keep
|
| 64 |
+
the filler −1000 (see `test_vectors_beatmel.npz` params)
|
| 65 |
+
- pieces shorter than 1488 frames (~29.8 s): the reference runs one
|
| 66 |
+
shorter window that this fixed-shape graph cannot reproduce — attention
|
| 67 |
+
is global, so zero-padding to 1500 frames changes all logits slightly
|
| 68 |
+
(beat times on tested real audio were still identical; pad with zeros
|
| 69 |
+
and drop the padded frames, or prefer the MLX port for exactness)
|
| 70 |
+
|
| 71 |
+
## DSP frontend (host-side)
|
| 72 |
+
|
| 73 |
+
The Core ML graph contains the network only. The host implements the audio
|
| 74 |
+
frontend and must match `schism_mlx.analyze.beat_this.model.logmel_beat_this`
|
| 75 |
+
numerically — `test_vectors_beatmel.npz` in this repo holds a deterministic
|
| 76 |
+
waveform → log-mel pair plus the exact slaney filterbank matrix (float32;
|
| 77 |
+
match within ~1e-4 relative to be interchangeable with what this model was
|
| 78 |
+
verified against), and its params json documents the frontend and the full
|
| 79 |
+
chunk/stitching contract. A validated Swift implementation (Accelerate;
|
| 80 |
+
modules `SchismDSP` and `SchismPipeline`) is available at
|
| 81 |
+
[schism-audio/schism-dsp](https://github.com/schism-audio/schism-dsp),
|
| 82 |
+
tested against these exact vectors.
|
| 83 |
+
|
| 84 |
+
## License
|
| 85 |
+
|
| 86 |
+
MIT, inherited from the source repository (code and released weights).
|
| 87 |
+
Model by the [Institute of Computational Perception, JKU Linz](https://github.com/CPJKU);
|
| 88 |
+
Core ML conversion by [schism-audio](https://huggingface.co/schism-audio).
|
test_vectors_beatmel.npz
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:95fc73dd86ec2c81eab1a9589843ca16a3713438ff64c9aac103c8ae8a29602a
|
| 3 |
+
size 498806
|