zkeown commited on
Commit
93cf237
·
verified ·
1 Parent(s): 00125a2

Upload folder using huggingface_hub

Browse files
BeatThis_fp16.mlpackage/Data/com.apple.CoreML/model.mlmodel ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:db6e5a91cdd5798e6627ebc0cf54505eb79652010c17cab18ceaa11524e8c0a3
3
+ size 235611
BeatThis_fp16.mlpackage/Data/com.apple.CoreML/weights/weight.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0bd7336ed1c7b9cd746edea148bb64530c2e3a4f7c7b1ea780ffc110fdc84d4b
3
+ size 40718144
BeatThis_fp16.mlpackage/Manifest.json ADDED
@@ -0,0 +1,18 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "fileFormatVersion": "1.0.0",
3
+ "itemInfoEntries": {
4
+ "97F48BF4-AE1D-48C1-91BA-1964DEBD9544": {
5
+ "author": "com.apple.CoreML",
6
+ "description": "CoreML Model Weights",
7
+ "name": "weights",
8
+ "path": "com.apple.CoreML/weights"
9
+ },
10
+ "9FBC2379-00F8-4BFB-9258-1AF917D67E30": {
11
+ "author": "com.apple.CoreML",
12
+ "description": "CoreML Model Specification",
13
+ "name": "model.mlmodel",
14
+ "path": "com.apple.CoreML/model.mlmodel"
15
+ }
16
+ },
17
+ "rootModelIdentifier": "9FBC2379-00F8-4BFB-9258-1AF917D67E30"
18
+ }
BeatThis_fp32.mlpackage/Data/com.apple.CoreML/model.mlmodel ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0fa10d898241fa400201bfdec0b420b1cac1ac7fd2644eaf29850a6105800775
3
+ size 207858
BeatThis_fp32.mlpackage/Data/com.apple.CoreML/weights/weight.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fd175cdec237a272da9804b09be7a4267886b733c61b904c680d02809f8496f6
3
+ size 81426304
BeatThis_fp32.mlpackage/Manifest.json ADDED
@@ -0,0 +1,18 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "fileFormatVersion": "1.0.0",
3
+ "itemInfoEntries": {
4
+ "2EA572E6-9CD9-4D98-83CB-4523A6DE302B": {
5
+ "author": "com.apple.CoreML",
6
+ "description": "CoreML Model Specification",
7
+ "name": "model.mlmodel",
8
+ "path": "com.apple.CoreML/model.mlmodel"
9
+ },
10
+ "5EEA07D7-D290-4F63-B160-059DF953724A": {
11
+ "author": "com.apple.CoreML",
12
+ "description": "CoreML Model Weights",
13
+ "name": "weights",
14
+ "path": "com.apple.CoreML/weights"
15
+ }
16
+ },
17
+ "rootModelIdentifier": "2EA572E6-9CD9-4D98-83CB-4523A6DE302B"
18
+ }
README.md ADDED
@@ -0,0 +1,88 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: mit
3
+ tags:
4
+ - coreml
5
+ - apple-silicon
6
+ - ios
7
+ - beat-this
8
+ - beat-tracking
9
+ - downbeat-tracking
10
+ - music-information-retrieval
11
+ - schism-coreml
12
+ ---
13
+
14
+ # Beat This! — Core ML
15
+
16
+ [CPJKU/beat_this](https://github.com/CPJKU/beat_this) (`final0` checkpoint;
17
+ "Beat This! Accurate Beat Tracking Without DBN Postprocessing",
18
+ [Foscarin, Schlüter & Widmer, ISMIR 2024](https://arxiv.org/abs/2407.21658),
19
+ **MIT licensed**) for Core ML on Apple devices. Frame-wise beat and downbeat
20
+ tracking at 50 fps with minimal peak-picking postprocessing — no DBN.
21
+
22
+ Converted from the same verified reference used by the
23
+ [schism-mlx](https://github.com/schism-audio/schism-mlx) MLX ports. Two variants per model:
24
+
25
+ | File | Precision | Compute units | max logit diff |
26
+ |---|---|---|---|
27
+ | `BeatThis_fp16.mlpackage` | FLOAT16 | ALL (ANE) | 6.7e-2, peak-picked beat/downbeat times identical on tested real drum audio |
28
+ | `BeatThis_fp32.mlpackage` | FLOAT32 | CPU+GPU | 5.2e-5, peak-picked beat/downbeat times identical on all tested clips |
29
+
30
+ Verified on-device-equivalently via coremltools on an M5 Max, against the
31
+ reference implementation on real audio. fp16 is ANE-eligible and recommended
32
+ for iPhone / iPad; fp32 is the tight-parity fallback.
33
+
34
+ ## Download
35
+
36
+ `.mlpackage` bundles must be materialized as real files — the Core ML
37
+ compiler rejects the symlinks that a default `snapshot_download` creates in
38
+ the Hugging Face cache:
39
+
40
+ ```python
41
+ from huggingface_hub import snapshot_download
42
+ path = snapshot_download("schism-audio/beat-this-coreml", local_dir="./beat-this-coreml")
43
+ ```
44
+
45
+ (or `hf download schism-audio/beat-this-coreml --local-dir ./beat-this-coreml`). Swift hosts
46
+ downloading files directly are unaffected.
47
+
48
+ ## I/O contract
49
+
50
+ - input `spect`: `(1, 1500, 128)` float32 — one 30 s chunk of the log-mel
51
+ frontend (22.05 kHz mono, n_fft 1024, hop 441, 128 slaney mels
52
+ 30–11000 Hz **without** area norm, magnitude STFT normalized
53
+ `1/sqrt(n_fft)`, `log1p(1000 x)`), 50 fps
54
+ - outputs `beat_logits` / `downbeat_logits`: `(1, 1500)` float32 —
55
+ frame-wise logits at 50 fps; sigmoid for probabilities, peak-pick for
56
+ times (local maxima within ±3 frames with logit > 0, adjacent peaks
57
+ merged by averaging, downbeats snapped to the nearest beat — the
58
+ reference "minimal" postprocessor)
59
+ - longer audio: 1500-frame chunks starting at `-6 + k·1488`, last start
60
+ shifted left to `n − 1494` so the last chunk ends at the piece end;
61
+ chunk borders that fall outside the piece are zero-padded on the
62
+ **spectrogram**; discard the 6 border frames of every chunk, earlier
63
+ chunks winning on overlap ("keep_first"); frames no chunk covers keep
64
+ the filler −1000 (see `test_vectors_beatmel.npz` params)
65
+ - pieces shorter than 1488 frames (~29.8 s): the reference runs one
66
+ shorter window that this fixed-shape graph cannot reproduce — attention
67
+ is global, so zero-padding to 1500 frames changes all logits slightly
68
+ (beat times on tested real audio were still identical; pad with zeros
69
+ and drop the padded frames, or prefer the MLX port for exactness)
70
+
71
+ ## DSP frontend (host-side)
72
+
73
+ The Core ML graph contains the network only. The host implements the audio
74
+ frontend and must match `schism_mlx.analyze.beat_this.model.logmel_beat_this`
75
+ numerically — `test_vectors_beatmel.npz` in this repo holds a deterministic
76
+ waveform → log-mel pair plus the exact slaney filterbank matrix (float32;
77
+ match within ~1e-4 relative to be interchangeable with what this model was
78
+ verified against), and its params json documents the frontend and the full
79
+ chunk/stitching contract. A validated Swift implementation (Accelerate;
80
+ modules `SchismDSP` and `SchismPipeline`) is available at
81
+ [schism-audio/schism-dsp](https://github.com/schism-audio/schism-dsp),
82
+ tested against these exact vectors.
83
+
84
+ ## License
85
+
86
+ MIT, inherited from the source repository (code and released weights).
87
+ Model by the [Institute of Computational Perception, JKU Linz](https://github.com/CPJKU);
88
+ Core ML conversion by [schism-audio](https://huggingface.co/schism-audio).
test_vectors_beatmel.npz ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:95fc73dd86ec2c81eab1a9589843ca16a3713438ff64c9aac103c8ae8a29602a
3
+ size 498806