File size: 2,510 Bytes
22979a4 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 | ---
license: apache-2.0
tags:
- mlx
- audio
- music
- audio-to-midi
- transcription
- basic-pitch
library_name: mlx
---
# Basic Pitch (MLX)
MLX port of [Spotify's Basic Pitch](https://github.com/spotify/basic-pitch),
a lightweight (~17k parameter) instrument-agnostic model for polyphonic
note transcription and multipitch estimation (audio → MIDI note events with
pitch bends).
- **Paper:** Bittner et al., ["A Lightweight Instrument-Agnostic Model for
Polyphonic Note Transcription and Multipitch Estimation"](https://arxiv.org/abs/2203.09893)
(ICASSP 2022)
- **Weights:** converted from the official `nmp.onnx` artifact in
[spotify/basic-pitch](https://github.com/spotify/basic-pitch)
(commit `fa5997af`, sha256 `2c3c1d14…`), Apache-2.0 code **and** weights.
- **Architecture:** fixed CQT frontend (nnAudio CQT2010v2 variant; 309 bins,
3 per semitone from A0) with 8-way harmonic stacking, then a small CNN with
three sigmoid heads: onset (88 bins), note (88 bins) and pitch-contour
(264 bins) posteriorgrams at ~86 fps. The frontend runs in numpy; the CNN
in MLX. Note-event decoding is the official basic-pitch postprocessing
(onset peak-picking, note segmentation, melodia trick, pitch bends),
transliterated to numpy.
## Usage
```python
from schism_mlx.transcribe.basic_pitch import BasicPitch
model = BasicPitch.from_pretrained("build/basic-pitch-mlx")
result = model.transcribe("song.wav") # any sample rate; resampled to 22050
for note in result.notes:
print(f"{note.start:6.2f}s {note.end:6.2f}s midi={note.pitch} amp={note.amplitude:.2f}")
result.save_midi("song.mid") # optional, needs pretty_midi (schism-mlx[midi])
```
`result` also carries the raw `note` / `onset` / `contour` posteriorgrams
and their frame times. Decoding thresholds (`onset_threshold=0.5`,
`frame_threshold=0.3`, `minimum_note_length=127.7` ms, frequency limits,
pitch bends, melodia trick) match `basic_pitch.inference.predict` and are
keyword arguments of `transcribe`.
## Parity
Verified against the official ONNX artifact via onnxruntime
(`scripts/verify_basic_pitch.py`): CQT frontend magnitude, per-window
posteriorgrams, windowed long-audio equivalence against
`basic_pitch.inference.run_inference`, and note-event agreement of the
decoded output on synthetic tonal audio and an [E-GMD](https://huggingface.co/datasets/schism-audio/e-gmd)
drum clip.
## License
Apache-2.0, same as the upstream code and weights. Cite the ICASSP 2022
paper if you use this model in research.
|