DeafBoyAudio/audio-compass-models

Audio Compass models

ONNX models used by Audio Compass, a sample-finder plugin by Deaf Boy Audio. The plugin downloads these files itself (⋯ › Download models) and checks every file against SHA256SUMS, so there is nothing to do by hand.

File	What it is	Licence
clap_audio.onnx	CLAP audio encoder: 48 kHz mono, 10 s → 512-d L2-normalised embedding. Log-mel front end built into the graph. Dynamic int8 weights (front end kept fp32).	Apache-2.0
clap_text.onnx	CLAP text encoder: token ids → 512-d L2-normalised embedding. Dynamic int8.	Apache-2.0
clap_vocab.json, clap_merges.txt	RoBERTa byte-level BPE tokenizer files	Apache-2.0
clap_config.json	Every preprocessing parameter (sample rate, crop/pad rule, mel settings, token ids)	Apache-2.0
crepe_tiny.onnx	CREPE "tiny" pitch tracker: 1024-sample frames @ 16 kHz → 360-bin salience. Per-frame normalisation built into the graph.	MIT
SHA256SUMS	Checksums of the files above	—

Where they come from (modified works)

• CLAP: converted from laion/larger_clap_music_and_speech (LAION, Apache-2.0). Changes: exported to ONNX as separate audio and text encoders with projection + L2 normalisation, mel spectrogram moved into the graph, weights dynamically quantised to int8. Licence: LICENSE-CLAP.
• CREPE tiny: the weights distributed with torchcrepe 0.0.24 (MIT, © 2020 Max Morrison), a port of CREPE (MIT, © 2018 Jong Wook Kim, Justin Salamon, Peter Li, Juan Pablo Bello). Changes: exported to ONNX with per-frame normalisation in the graph. Licence: LICENSE-CREPE.

The files are produced by tools/export_models.py in the Audio Compass source code; its parity tests check the ONNX outputs against the original PyTorch models (cosine ≥ 0.99 for CLAP, within 10 cents for CREPE).

Citations

• Wu, Chen, Zhang, Hui, Berg-Kirkpatrick, Dubnov. Large-scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to-Caption Augmentation. ICASSP 2023.
• Kim, Salamon, Li, Bello. CREPE: A Convolutional Representation for Pitch Estimation. ICASSP 2018.