metadata
license: cc-by-nc-sa-4.0
tags:
- music
- audio
- contrastive
- clap
- music-information-retrieval
library_name: amclap
AllMusicCaps: Baseline + Structured captions
Baseline plus LLM-structured captions.
Audio-text contrastive (CLAP) model over the OMAR-RQ audio encoder and an
all-mpnet-base-v2 text encoder, trained on music with album-review derived
captions. Embeddings are 512-dimensional; audio is expected at 24 kHz.
Usage
from amclap import get_model
model = get_model(model_id="mtg-upf/allmusiccaps_baseline_struct", device="cpu")
audio_emb = model.forward_audio(audio) # (B, 512), audio at 24 kHz
text_emb = model.forward_text(["warm analog synth pads"]) # (B, 512)
Training
| Text encoder | frozen (stock all-mpnet-base-v2) |
| Reported checkpoint | step 149,796 |
| Audio encoder layers | see config.gin |
The text encoder was frozen during training, so stock all-mpnet-base-v2 weights are reloaded at construction.
License
CC BY-NC-SA 4.0. See LICENSE.
Citation
@inproceedings{alonso2026allmusiccaps,
title = {{AllMusicCaps}: Album Reviews as Complementary Supervision for Music {CLAP}},
author = {Alonso-Jim{\'e}nez, Pablo and Lizarraga-Seijas, Xavier and Serra, Xavier and Bogdanov, Dmitry},
booktitle = {International Society for Music Information Retrieval Conference (ISMIR)},
year = {2026},
}