README.md

metadata

license: apache-2.0
tags:
  - mlx
  - audio
  - music
  - audio-to-midi
  - transcription
  - basic-pitch
library_name: mlx

Basic Pitch (MLX)

MLX port of Spotify's Basic Pitch, a lightweight (~17k parameter) instrument-agnostic model for polyphonic note transcription and multipitch estimation (audio → MIDI note events with pitch bends).

• Paper: Bittner et al., "A Lightweight Instrument-Agnostic Model for Polyphonic Note Transcription and Multipitch Estimation" (ICASSP 2022)
• Weights: converted from the official nmp.onnx artifact in spotify/basic-pitch (commit fa5997af, sha256 2c3c1d14…), Apache-2.0 code and weights.
• Architecture: fixed CQT frontend (nnAudio CQT2010v2 variant; 309 bins, 3 per semitone from A0) with 8-way harmonic stacking, then a small CNN with three sigmoid heads: onset (88 bins), note (88 bins) and pitch-contour (264 bins) posteriorgrams at ~86 fps. The frontend runs in numpy; the CNN in MLX. Note-event decoding is the official basic-pitch postprocessing (onset peak-picking, note segmentation, melodia trick, pitch bends), transliterated to numpy.

Usage

from schism_mlx.transcribe.basic_pitch import BasicPitch

model = BasicPitch.from_pretrained("build/basic-pitch-mlx")
result = model.transcribe("song.wav")  # any sample rate; resampled to 22050
for note in result.notes:
    print(f"{note.start:6.2f}s  {note.end:6.2f}s  midi={note.pitch}  amp={note.amplitude:.2f}")
result.save_midi("song.mid")  # optional, needs pretty_midi (schism-mlx[midi])

result also carries the raw note / onset / contour posteriorgrams and their frame times. Decoding thresholds (onset_threshold=0.5, frame_threshold=0.3, minimum_note_length=127.7 ms, frequency limits, pitch bends, melodia trick) match basic_pitch.inference.predict and are keyword arguments of transcribe.

Parity

Verified against the official ONNX artifact via onnxruntime (scripts/verify_basic_pitch.py): CQT frontend magnitude, per-window posteriorgrams, windowed long-audio equivalence against basic_pitch.inference.run_inference, and note-event agreement of the decoded output on synthetic tonal audio and an E-GMD drum clip.

License

Apache-2.0, same as the upstream code and weights. Cite the ICASSP 2022 paper if you use this model in research.