litert-community/CREPE-pitch-LiteRT

CREPE — LiteRT (on-device pitch detection / real-time tuner, fully-GPU)

CREPE monophonic pitch (f0) estimation, converted to LiteRT and running fully on the CompiledModel GPU (ML Drift) on Android. A 1024-sample (16 kHz) window → activations over 360 pitch bins (20 cents each, ~C1–B7); the host decodes them to a frequency and the nearest musical note. Drives a real-time mic tuner (note + cents flat/sharp).

frame[1,1024] (16 kHz, per-frame zero-mean/unit-var) →[GPU CNN]→ activations[1,360] →[host]→ Hz → note

On-device (Pixel 8a, Tensor G3 — verified)

nodes on GPU	49 / 49 LITERT_CL (full residency, single graph, 1 partition)
inference	~75 ms / frame (full model)
size	44.5 MB (fp16)
accuracy	fp16 tflite-vs-PyTorch corr 1.000000; self-test (synth 440 Hz) → A4, 440.4 Hz

How it converts — the cleanest in the zoo, zero patches

The whole network is a pure CNN: 6× {zero-pad → Conv2d → ReLU → BatchNorm → MaxPool} + permute/reshape (≤4D) + Linear + sigmoid. Converted directly with litert-torch — no rewrites:

• No banned ops — the asymmetric "same" padding is a constant zero-pad → native PAD (not GATHER); no GELU / TransposeConv / dilated conv; the head permute(0,2,1,3).reshape stays ≤4D.
• No fp16-on-Mali wall — per-frame zero-mean/unit-var normalization keeps activations ~O(1).

op-check: banned NONE, >4D 0; fp16 corr 1.000000.

Preprocessing & decode (host-side)

Mono 16 kHz. Frame into 1024-sample windows; per frame subtract the mean and divide by the std. Decode the 360 activations: peak bin ± 4, activation-weighted average → cents = 20·bin + 1997.3794… → Hz = 10·2^(cents/1200); the peak activation is the confidence. Nearest note: midi = 69 + 12·log2(Hz/440).

Minimal usage

Android (Kotlin, CompiledModel GPU)

val model = CompiledModel.create(context.assets, "crepe_full_fp16.tflite",
    CompiledModel.Options(Accelerator.GPU), null)
val inputs = model.createInputBuffers(); val outputs = model.createOutputBuffers()
inputs[0].writeFloat(frame)           // [1,1024] 16 kHz window, per-frame zero-mean/unit-var
model.run(inputs, outputs)
val act = outputs[0].readFloat()      // [360] pitch-bin activations -> host decode to Hz

Python (desktop verification)

import numpy as np, soundfile as sf
from ai_edge_litert.interpreter import Interpreter

wav, _ = sf.read("note_16k.wav", dtype="float32")               # mono 16 kHz
f = wav[:1024].copy()
f = (f - f.mean()) / max(f.std(), 1e-10)                        # per-frame normalize

it = Interpreter(model_path="crepe_full_fp16.tflite"); it.allocate_tensors()
it.set_tensor(it.get_input_details()[0]["index"], f[None]); it.invoke()
act = it.get_tensor(it.get_output_details()[0]["index"])[0]     # [360]

c = int(act.argmax()); s, e = max(0, c - 4), min(360, c + 5)    # torchcrepe weighted_argmax
w, b = act[s:e], np.arange(s, e)
cents = 20.0 * (w * b).sum() / w.sum() + 1997.3794084376191
hz = 10.0 * 2 ** (cents / 1200.0)
midi = 69 + 12 * np.log2(hz / 440.0)
names = ["C", "C#", "D", "D#", "E", "F", "F#", "G", "G#", "A", "A#", "B"]
n = int(round(midi))
print(f"{hz:.1f} Hz  {names[n % 12]}{n // 12 - 1} {round((midi - n) * 100):+d} cents  conf {act[c]:.2f}")

Files

File	What
crepe_full_fp16.tflite	the full CREPE model, fp16, frame [1,1024] → activations [1,360]
build_crepe.py	conversion + parity + 220/440/880 Hz self-test

License

MIT. Upstream: marl/crepe (Kim, Salamon, Li, Bello — "CREPE: A Convolutional Representation for Pitch Estimation", ICASSP 2018); PyTorch weights via torchcrepe (MIT).

Citation

@inproceedings{kim2018crepe,
  title={CREPE: A Convolutional Representation for Pitch Estimation},
  author={Kim, Jong Wook and Salamon, Justin and Li, Peter and Bello, Juan Pablo},
  booktitle={IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)},
  year={2018}
}