Upload folder using huggingface_hub
Browse files- BasicPitch_fp16.mlpackage/Data/com.apple.CoreML/model.mlmodel +3 -0
- BasicPitch_fp16.mlpackage/Data/com.apple.CoreML/weights/weight.bin +3 -0
- BasicPitch_fp16.mlpackage/Manifest.json +18 -0
- BasicPitch_fp32.mlpackage/Data/com.apple.CoreML/model.mlmodel +3 -0
- BasicPitch_fp32.mlpackage/Data/com.apple.CoreML/weights/weight.bin +3 -0
- BasicPitch_fp32.mlpackage/Manifest.json +18 -0
- README.md +59 -0
- test_vectors_cqt.npz +3 -0
BasicPitch_fp16.mlpackage/Data/com.apple.CoreML/model.mlmodel
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:722b4d2ee30e763cf868f5eff4c6cda22b0ae95f3d54bb9e8deb9824ce8ef510
|
| 3 |
+
size 129228
|
BasicPitch_fp16.mlpackage/Data/com.apple.CoreML/weights/weight.bin
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:aa0f4253ef6bdb7ff7171c4cd13f1f9168eee1e535d23b4759b2126348b65652
|
| 3 |
+
size 112594
|
BasicPitch_fp16.mlpackage/Manifest.json
ADDED
|
@@ -0,0 +1,18 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"fileFormatVersion": "1.0.0",
|
| 3 |
+
"itemInfoEntries": {
|
| 4 |
+
"203000E8-75EC-44B2-81C2-E370FE788AC2": {
|
| 5 |
+
"author": "com.apple.CoreML",
|
| 6 |
+
"description": "CoreML Model Weights",
|
| 7 |
+
"name": "weights",
|
| 8 |
+
"path": "com.apple.CoreML/weights"
|
| 9 |
+
},
|
| 10 |
+
"7015EF96-273F-475E-B8B6-329B11A31AC5": {
|
| 11 |
+
"author": "com.apple.CoreML",
|
| 12 |
+
"description": "CoreML Model Specification",
|
| 13 |
+
"name": "model.mlmodel",
|
| 14 |
+
"path": "com.apple.CoreML/model.mlmodel"
|
| 15 |
+
}
|
| 16 |
+
},
|
| 17 |
+
"rootModelIdentifier": "7015EF96-273F-475E-B8B6-329B11A31AC5"
|
| 18 |
+
}
|
BasicPitch_fp32.mlpackage/Data/com.apple.CoreML/model.mlmodel
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:af7bf7d49bc167e0bf0c30aa2ca6b432c3e10df048d2dd4173ff3a738c020858
|
| 3 |
+
size 123027
|
BasicPitch_fp32.mlpackage/Data/com.apple.CoreML/weights/weight.bin
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:691a6b63c7ddcdde0ee131ff3986dcb1250df47cd738612efde966ba9b4c99cd
|
| 3 |
+
size 145956
|
BasicPitch_fp32.mlpackage/Manifest.json
ADDED
|
@@ -0,0 +1,18 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"fileFormatVersion": "1.0.0",
|
| 3 |
+
"itemInfoEntries": {
|
| 4 |
+
"7EB69391-53BD-4281-BFA1-831180579ABC": {
|
| 5 |
+
"author": "com.apple.CoreML",
|
| 6 |
+
"description": "CoreML Model Specification",
|
| 7 |
+
"name": "model.mlmodel",
|
| 8 |
+
"path": "com.apple.CoreML/model.mlmodel"
|
| 9 |
+
},
|
| 10 |
+
"C784FB56-1BA8-4A0E-AC2A-F0388FE1A2DB": {
|
| 11 |
+
"author": "com.apple.CoreML",
|
| 12 |
+
"description": "CoreML Model Weights",
|
| 13 |
+
"name": "weights",
|
| 14 |
+
"path": "com.apple.CoreML/weights"
|
| 15 |
+
}
|
| 16 |
+
},
|
| 17 |
+
"rootModelIdentifier": "7EB69391-53BD-4281-BFA1-831180579ABC"
|
| 18 |
+
}
|
README.md
ADDED
|
@@ -0,0 +1,59 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
tags:
|
| 4 |
+
- coreml
|
| 5 |
+
- apple-silicon
|
| 6 |
+
- ios
|
| 7 |
+
- basic-pitch
|
| 8 |
+
- audio-to-midi
|
| 9 |
+
- music-transcription
|
| 10 |
+
- multipitch-estimation
|
| 11 |
+
- schism-coreml
|
| 12 |
+
---
|
| 13 |
+
|
| 14 |
+
# Basic Pitch — Core ML
|
| 15 |
+
|
| 16 |
+
[Spotify's Basic Pitch](https://github.com/spotify/basic-pitch) (Bittner et al., ICASSP 2022 — a ~17k-parameter instrument-agnostic model for polyphonic note transcription and multipitch estimation, audio → MIDI) for Core ML on Apple devices. `BasicPitch_fp32.mlpackage` **is Spotify's official Core ML artifact** (repo commit `fa5997af`, sha256-pinned, redistributed bit-identical); the fp16 variant is derived from it by recasting the CNN body while keeping the CQT frontend in FLOAT32.
|
| 17 |
+
|
| 18 |
+
Verified against the official `nmp.onnx` reference — the same oracle the [schism-mlx](https://github.com/schism-audio/schism-mlx) MLX port is verified against (posteriorgram parity and note-event decode agreement on synthetic melodies and an E-GMD drum clip). Two variants per model:
|
| 19 |
+
|
| 20 |
+
| File | Precision | Compute units | max posteriorgram diff vs nmp.onnx |
|
| 21 |
+
|---|---|---|---|
|
| 22 |
+
| `BasicPitch_fp16.mlpackage` | FLOAT16 body, FLOAT32 CQT frontend | ALL (ANE) | 1.3e-3, decoded note events identical on tested clips |
|
| 23 |
+
| `BasicPitch_fp32.mlpackage` | FLOAT32 (official artifact, iOS 15+) | CPU+GPU | 1.3e-6 |
|
| 24 |
+
|
| 25 |
+
Verified on-device-equivalently via coremltools on an M5 Max, against the
|
| 26 |
+
reference implementation on real audio. fp16 is ANE-eligible and recommended
|
| 27 |
+
for iPhone / iPad; fp32 is the tight-parity fallback.
|
| 28 |
+
|
| 29 |
+
## Download
|
| 30 |
+
|
| 31 |
+
`.mlpackage` bundles must be materialized as real files — the Core ML
|
| 32 |
+
compiler rejects the symlinks that a default `snapshot_download` creates in
|
| 33 |
+
the Hugging Face cache:
|
| 34 |
+
|
| 35 |
+
```python
|
| 36 |
+
from huggingface_hub import snapshot_download
|
| 37 |
+
path = snapshot_download("schism-audio/basic-pitch-coreml", local_dir="./basic-pitch-coreml")
|
| 38 |
+
```
|
| 39 |
+
|
| 40 |
+
(or `hf download schism-audio/basic-pitch-coreml --local-dir ./basic-pitch-coreml`). Swift hosts
|
| 41 |
+
downloading files directly are unaffected.
|
| 42 |
+
|
| 43 |
+
## I/O contract
|
| 44 |
+
|
| 45 |
+
- input `input_2`: `(1, 43844, 1)` float32 — one raw ~2 s window of 22050 Hz mono audio (43844 = 2·22050 − 256)
|
| 46 |
+
- output `Identity`: `(1, 172, 264)` float32 — pitch-contour posteriorgram (3 bins/semitone from A0)
|
| 47 |
+
- output `Identity_1`: `(1, 172, 88)` float32 — note posteriorgram (semitones A0–C8)
|
| 48 |
+
- output `Identity_2`: `(1, 172, 88)` float32 — onset posteriorgram
|
| 49 |
+
- frames are ~86.13 fps (22050/256); all outputs are sigmoid probabilities
|
| 50 |
+
- full tracks: front-pad 3840 zeros, windows every 36164 samples, drop 15 frames per side of each window's 172 output frames, concatenate, trim to `int(n_samples/36164*142)`; frame→seconds includes a per-window offset with the reference's 0.0018 s magic alignment constant — the exact recipe (plus note-event decoding defaults) is in `test_vectors_cqt.npz` params and implemented in schism-mlx `transcribe/basic_pitch/model.py`
|
| 51 |
+
- **caveat**: skip the model for windows that are *entirely* zero and emit zero posteriorgram frames instead — the official artifact's normalization diverges from `nmp.onnx` on exact digital silence (≥ 2 s of zero samples) and can hallucinate notes; any nonzero sample in the window is fine (verified to 7e-7 on a mostly-zero window)
|
| 52 |
+
|
| 53 |
+
## DSP frontend (host-side)
|
| 54 |
+
|
| 55 |
+
Unlike the other models in this org, the Core ML graph **embeds the whole DSP frontend** (multirate CQT, log normalization, harmonic stacking) — the host feeds raw audio windows and implements only windowing, output unwrapping, frame-time mapping and note-event decoding, all documented in the `test_vectors_cqt.npz` params json. The vectors also pin every internal frontend stage (CQT magnitude, normalized log-CQT, harmonic stack — float32, computed by the verified schism-mlx port, 1e-6 parity vs `nmp.onnx`) alongside the posteriorgrams the shipped fp32 package produces for the same window, for hosts that want a native frontend or to validate the pipeline end to end. Note-event decoding is pure post-processing (peak-picking + note segmentation); the schism-mlx numpy transliteration is verified exactly equal to `basic_pitch.note_creation`.
|
| 56 |
+
|
| 57 |
+
## License
|
| 58 |
+
|
| 59 |
+
Apache-2.0, same as the upstream code and weights. `BasicPitch_fp32.mlpackage` is the unmodified official artifact from [spotify/basic-pitch](https://github.com/spotify/basic-pitch); `BasicPitch_fp16.mlpackage` is derived from it. Cite the [ICASSP 2022 paper](https://arxiv.org/abs/2203.09893) if you use this model in research. Core ML packaging and verification by [schism-audio](https://huggingface.co/schism-audio).
|
test_vectors_cqt.npz
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:c64b3f2a12b86311d6149882d40e5e6660f68b2c87f6e68751807ece9a3d4855
|
| 3 |
+
size 2378760
|