zkeown commited on
Commit
531a90e
·
verified ·
1 Parent(s): 52eb6cd

Upload folder using huggingface_hub

Browse files
BasicPitch_fp16.mlpackage/Data/com.apple.CoreML/model.mlmodel ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:722b4d2ee30e763cf868f5eff4c6cda22b0ae95f3d54bb9e8deb9824ce8ef510
3
+ size 129228
BasicPitch_fp16.mlpackage/Data/com.apple.CoreML/weights/weight.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:aa0f4253ef6bdb7ff7171c4cd13f1f9168eee1e535d23b4759b2126348b65652
3
+ size 112594
BasicPitch_fp16.mlpackage/Manifest.json ADDED
@@ -0,0 +1,18 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "fileFormatVersion": "1.0.0",
3
+ "itemInfoEntries": {
4
+ "203000E8-75EC-44B2-81C2-E370FE788AC2": {
5
+ "author": "com.apple.CoreML",
6
+ "description": "CoreML Model Weights",
7
+ "name": "weights",
8
+ "path": "com.apple.CoreML/weights"
9
+ },
10
+ "7015EF96-273F-475E-B8B6-329B11A31AC5": {
11
+ "author": "com.apple.CoreML",
12
+ "description": "CoreML Model Specification",
13
+ "name": "model.mlmodel",
14
+ "path": "com.apple.CoreML/model.mlmodel"
15
+ }
16
+ },
17
+ "rootModelIdentifier": "7015EF96-273F-475E-B8B6-329B11A31AC5"
18
+ }
BasicPitch_fp32.mlpackage/Data/com.apple.CoreML/model.mlmodel ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:af7bf7d49bc167e0bf0c30aa2ca6b432c3e10df048d2dd4173ff3a738c020858
3
+ size 123027
BasicPitch_fp32.mlpackage/Data/com.apple.CoreML/weights/weight.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:691a6b63c7ddcdde0ee131ff3986dcb1250df47cd738612efde966ba9b4c99cd
3
+ size 145956
BasicPitch_fp32.mlpackage/Manifest.json ADDED
@@ -0,0 +1,18 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "fileFormatVersion": "1.0.0",
3
+ "itemInfoEntries": {
4
+ "7EB69391-53BD-4281-BFA1-831180579ABC": {
5
+ "author": "com.apple.CoreML",
6
+ "description": "CoreML Model Specification",
7
+ "name": "model.mlmodel",
8
+ "path": "com.apple.CoreML/model.mlmodel"
9
+ },
10
+ "C784FB56-1BA8-4A0E-AC2A-F0388FE1A2DB": {
11
+ "author": "com.apple.CoreML",
12
+ "description": "CoreML Model Weights",
13
+ "name": "weights",
14
+ "path": "com.apple.CoreML/weights"
15
+ }
16
+ },
17
+ "rootModelIdentifier": "7EB69391-53BD-4281-BFA1-831180579ABC"
18
+ }
README.md ADDED
@@ -0,0 +1,59 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ tags:
4
+ - coreml
5
+ - apple-silicon
6
+ - ios
7
+ - basic-pitch
8
+ - audio-to-midi
9
+ - music-transcription
10
+ - multipitch-estimation
11
+ - schism-coreml
12
+ ---
13
+
14
+ # Basic Pitch — Core ML
15
+
16
+ [Spotify's Basic Pitch](https://github.com/spotify/basic-pitch) (Bittner et al., ICASSP 2022 — a ~17k-parameter instrument-agnostic model for polyphonic note transcription and multipitch estimation, audio → MIDI) for Core ML on Apple devices. `BasicPitch_fp32.mlpackage` **is Spotify's official Core ML artifact** (repo commit `fa5997af`, sha256-pinned, redistributed bit-identical); the fp16 variant is derived from it by recasting the CNN body while keeping the CQT frontend in FLOAT32.
17
+
18
+ Verified against the official `nmp.onnx` reference — the same oracle the [schism-mlx](https://github.com/schism-audio/schism-mlx) MLX port is verified against (posteriorgram parity and note-event decode agreement on synthetic melodies and an E-GMD drum clip). Two variants per model:
19
+
20
+ | File | Precision | Compute units | max posteriorgram diff vs nmp.onnx |
21
+ |---|---|---|---|
22
+ | `BasicPitch_fp16.mlpackage` | FLOAT16 body, FLOAT32 CQT frontend | ALL (ANE) | 1.3e-3, decoded note events identical on tested clips |
23
+ | `BasicPitch_fp32.mlpackage` | FLOAT32 (official artifact, iOS 15+) | CPU+GPU | 1.3e-6 |
24
+
25
+ Verified on-device-equivalently via coremltools on an M5 Max, against the
26
+ reference implementation on real audio. fp16 is ANE-eligible and recommended
27
+ for iPhone / iPad; fp32 is the tight-parity fallback.
28
+
29
+ ## Download
30
+
31
+ `.mlpackage` bundles must be materialized as real files — the Core ML
32
+ compiler rejects the symlinks that a default `snapshot_download` creates in
33
+ the Hugging Face cache:
34
+
35
+ ```python
36
+ from huggingface_hub import snapshot_download
37
+ path = snapshot_download("schism-audio/basic-pitch-coreml", local_dir="./basic-pitch-coreml")
38
+ ```
39
+
40
+ (or `hf download schism-audio/basic-pitch-coreml --local-dir ./basic-pitch-coreml`). Swift hosts
41
+ downloading files directly are unaffected.
42
+
43
+ ## I/O contract
44
+
45
+ - input `input_2`: `(1, 43844, 1)` float32 — one raw ~2 s window of 22050 Hz mono audio (43844 = 2·22050 − 256)
46
+ - output `Identity`: `(1, 172, 264)` float32 — pitch-contour posteriorgram (3 bins/semitone from A0)
47
+ - output `Identity_1`: `(1, 172, 88)` float32 — note posteriorgram (semitones A0–C8)
48
+ - output `Identity_2`: `(1, 172, 88)` float32 — onset posteriorgram
49
+ - frames are ~86.13 fps (22050/256); all outputs are sigmoid probabilities
50
+ - full tracks: front-pad 3840 zeros, windows every 36164 samples, drop 15 frames per side of each window's 172 output frames, concatenate, trim to `int(n_samples/36164*142)`; frame→seconds includes a per-window offset with the reference's 0.0018 s magic alignment constant — the exact recipe (plus note-event decoding defaults) is in `test_vectors_cqt.npz` params and implemented in schism-mlx `transcribe/basic_pitch/model.py`
51
+ - **caveat**: skip the model for windows that are *entirely* zero and emit zero posteriorgram frames instead — the official artifact's normalization diverges from `nmp.onnx` on exact digital silence (≥ 2 s of zero samples) and can hallucinate notes; any nonzero sample in the window is fine (verified to 7e-7 on a mostly-zero window)
52
+
53
+ ## DSP frontend (host-side)
54
+
55
+ Unlike the other models in this org, the Core ML graph **embeds the whole DSP frontend** (multirate CQT, log normalization, harmonic stacking) — the host feeds raw audio windows and implements only windowing, output unwrapping, frame-time mapping and note-event decoding, all documented in the `test_vectors_cqt.npz` params json. The vectors also pin every internal frontend stage (CQT magnitude, normalized log-CQT, harmonic stack — float32, computed by the verified schism-mlx port, 1e-6 parity vs `nmp.onnx`) alongside the posteriorgrams the shipped fp32 package produces for the same window, for hosts that want a native frontend or to validate the pipeline end to end. Note-event decoding is pure post-processing (peak-picking + note segmentation); the schism-mlx numpy transliteration is verified exactly equal to `basic_pitch.note_creation`.
56
+
57
+ ## License
58
+
59
+ Apache-2.0, same as the upstream code and weights. `BasicPitch_fp32.mlpackage` is the unmodified official artifact from [spotify/basic-pitch](https://github.com/spotify/basic-pitch); `BasicPitch_fp16.mlpackage` is derived from it. Cite the [ICASSP 2022 paper](https://arxiv.org/abs/2203.09893) if you use this model in research. Core ML packaging and verification by [schism-audio](https://huggingface.co/schism-audio).
test_vectors_cqt.npz ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c64b3f2a12b86311d6149882d40e5e6660f68b2c87f6e68751807ece9a3d4855
3
+ size 2378760