README.md

---
license: apache-2.0
tags:
- mlx
- audio
- music
- audio-to-midi
- transcription
- basic-pitch
library_name: mlx
---

# Basic Pitch (MLX)

MLX port of [Spotify's Basic Pitch](https://github.com/spotify/basic-pitch),
a lightweight (~17k parameter) instrument-agnostic model for polyphonic
note transcription and multipitch estimation (audio → MIDI note events with
pitch bends).

- **Paper:** Bittner et al., ["A Lightweight Instrument-Agnostic Model for
  Polyphonic Note Transcription and Multipitch Estimation"](https://arxiv.org/abs/2203.09893)
  (ICASSP 2022)
- **Weights:** converted from the official `nmp.onnx` artifact in
  [spotify/basic-pitch](https://github.com/spotify/basic-pitch)
  (commit `fa5997af`, sha256 `2c3c1d14…`), Apache-2.0 code **and** weights.
- **Architecture:** fixed CQT frontend (nnAudio CQT2010v2 variant; 309 bins,
  3 per semitone from A0) with 8-way harmonic stacking, then a small CNN with
  three sigmoid heads: onset (88 bins), note (88 bins) and pitch-contour
  (264 bins) posteriorgrams at ~86 fps. The frontend runs in numpy; the CNN
  in MLX. Note-event decoding is the official basic-pitch postprocessing
  (onset peak-picking, note segmentation, melodia trick, pitch bends),
  transliterated to numpy.

## Usage

```python
from schism_mlx.transcribe.basic_pitch import BasicPitch

model = BasicPitch.from_pretrained("build/basic-pitch-mlx")
result = model.transcribe("song.wav")  # any sample rate; resampled to 22050
for note in result.notes:
    print(f"{note.start:6.2f}s  {note.end:6.2f}s  midi={note.pitch}  amp={note.amplitude:.2f}")
result.save_midi("song.mid")  # optional, needs pretty_midi (schism-mlx[midi])
```

`result` also carries the raw `note` / `onset` / `contour` posteriorgrams
and their frame times. Decoding thresholds (`onset_threshold=0.5`,
`frame_threshold=0.3`, `minimum_note_length=127.7` ms, frequency limits,
pitch bends, melodia trick) match `basic_pitch.inference.predict` and are
keyword arguments of `transcribe`.

## Parity

Verified against the official ONNX artifact via onnxruntime
(`scripts/verify_basic_pitch.py`): CQT frontend magnitude, per-window
posteriorgrams, windowed long-audio equivalence against
`basic_pitch.inference.run_inference`, and note-event agreement of the
decoded output on synthetic tonal audio and an [E-GMD](https://huggingface.co/datasets/schism-audio/e-gmd)
drum clip.

## License

Apache-2.0, same as the upstream code and weights. Cite the ICASSP 2022
paper if you use this model in research.