Instructions to use litert-community/Basic-Pitch-LiteRT with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- LiteRT
How to use litert-community/Basic-Pitch-LiteRT with LiteRT:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- Notebooks
- Google Colab
- Kaggle
Upload README.md with huggingface_hub
Browse files
README.md
ADDED
|
@@ -0,0 +1,64 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
library_name: litert
|
| 4 |
+
pipeline_tag: audio-classification
|
| 5 |
+
tags: [music-transcription, audio-to-midi, basic-pitch, cqt, litert, tflite, on-device, gpu]
|
| 6 |
+
base_model: spotify/basic-pitch
|
| 7 |
+
---
|
| 8 |
+
|
| 9 |
+
# Basic Pitch — LiteRT (CompiledModel GPU) music transcription
|
| 10 |
+
|
| 11 |
+
[Basic Pitch](https://github.com/spotify/basic-pitch) (Spotify, ICASSP 2022, Apache-2.0)
|
| 12 |
+
re-authored to a **GPU-native** LiteRT `.tflite` — **including the conv-based CQT front-end**
|
| 13 |
+
(9 octaves, shared 36×256 kernel banks, lowpass downsample chain, no FFT). Bit-exact torch
|
| 14 |
+
re-implementation of the official ONNX (corr 1.000000). FP32, **0.84 MB**.
|
| 15 |
+
|
| 16 |
+
Pixel 8a (Tensor G3): **241/241 nodes LITERT_CL** (1 partition), **~4.4 ms** per 2 s window;
|
| 17 |
+
note-event F1@0.5 **0.98** vs the official model, per-frame argmax agreement 98%.
|
| 18 |
+
|
| 19 |
+
## I/O
|
| 20 |
+
- Input `[1, 43844]` float32 — 2 s @ 22 050 Hz mono, [-1, 1] (official window; overlap windows by
|
| 21 |
+
7 680 samples and keep center frames when stitching).
|
| 22 |
+
- Outputs: `contour [1,172,264]`, `note [1,172,88]`, `onset [1,172,88]` — sigmoid posteriorgrams
|
| 23 |
+
(~11.6 ms frames; note/onset bins = MIDI 21–108).
|
| 24 |
+
|
| 25 |
+
## Minimal usage
|
| 26 |
+
|
| 27 |
+
```python
|
| 28 |
+
import numpy as np, soundfile as sf
|
| 29 |
+
from ai_edge_litert.interpreter import Interpreter
|
| 30 |
+
|
| 31 |
+
wav, _ = sf.read("audio_22050.wav", dtype="float32") # mono 22.05 kHz
|
| 32 |
+
x = np.zeros(43844, np.float32); n = min(len(wav), 43844); x[:n] = wav[:n]
|
| 33 |
+
it = Interpreter(model_path="basicpitch.tflite"); it.allocate_tensors()
|
| 34 |
+
it.set_tensor(it.get_input_details()[0]["index"], x[None]); it.invoke()
|
| 35 |
+
contour, note, onset = (it.get_tensor(o["index"])[0] for o in
|
| 36 |
+
sorted(it.get_output_details(), key=lambda o: o["index"]))
|
| 37 |
+
active = np.argwhere(note > 0.5) # (frame, key); midi = key + 21, t = frame * 256/22050
|
| 38 |
+
```
|
| 39 |
+
|
| 40 |
+
### Kotlin (Android, LiteRT CompiledModel GPU)
|
| 41 |
+
|
| 42 |
+
```kotlin
|
| 43 |
+
// implementation("com.google.ai.edge.litert:litert:2.1.5")
|
| 44 |
+
val model = CompiledModel.create(File(ctx.filesDir, "basicpitch.tflite").absolutePath,
|
| 45 |
+
CompiledModel.Options(Accelerator.GPU), null)
|
| 46 |
+
val inBuf = model.createInputBuffers(); val outBuf = model.createOutputBuffers()
|
| 47 |
+
inBuf[0].writeFloat(window43844) // 2 s @ 22.05 kHz mono
|
| 48 |
+
model.run(inBuf, outBuf)
|
| 49 |
+
val note = outBuf[1].readFloat() // [172 * 88], frame-major; midi = bin + 21
|
| 50 |
+
val onset = outBuf[2].readFloat() // onset-triggered decoding -> note events
|
| 51 |
+
```
|
| 52 |
+
|
| 53 |
+
## Conversion
|
| 54 |
+
Extracted from the official `nmp.onnx` (102 constants; no TensorFlow). Reflect-pad →
|
| 55 |
+
anti-diagonal-constant `FULLY_CONNECTED`; PACK → concat + static slices. Two fp16-on-GPU fixes,
|
| 56 |
+
both exact: post-log clamp `clamp(10·log10(p+1e-10), min=-100)` (recovers `log(0)` from the
|
| 57 |
+
fp16-flushed floor; desktop no-op) and the per-bin CQT norm folded into per-octave kernel copies
|
| 58 |
+
(magnitude is linear in kernel scale). FP32 flatbuffer (fp16 weights cost ~0.005 corr on the tiny
|
| 59 |
+
CQT kernels).
|
| 60 |
+
|
| 61 |
+
## Upstream
|
| 62 |
+
[spotify/basic-pitch](https://github.com/spotify/basic-pitch) (Apache-2.0). Please cite
|
| 63 |
+
Bittner et al., *A Lightweight Instrument-Agnostic Model for Polyphonic Note Transcription and
|
| 64 |
+
Multipitch Estimation* (ICASSP 2022).
|