mtg-upf/allmusiccaps_te_trained

AllMusicCaps: Trainable text encoder

Joint text-encoder fine-tuning.

Audio-text contrastive (CLAP) model over the OMAR-RQ audio encoder and an all-mpnet-base-v2 text encoder, trained on music with album-review derived captions. Embeddings are 512-dimensional; audio is expected at 24 kHz.

Usage

from amclap import get_model

model = get_model(model_id="mtg-upf/allmusiccaps_te_trained", device="cpu")

audio_emb = model.forward_audio(audio)   # (B, 512), audio at 24 kHz
text_emb = model.forward_text(["warm analog synth pads"])  # (B, 512)

Training

Text encoder	fine-tuned jointly
Reported checkpoint	step 60,000 (mid-training: models with a trainable text encoder overfit past ~40-80k steps)
Audio encoder layers	all 12

This model's text encoder was fine-tuned, so its weights ship with the checkpoint rather than being loaded from the stock sentence-transformers model.

License

CC BY-NC-SA 4.0. See LICENSE.

Citation

@inproceedings{alonso2026allmusiccaps,
  title = {{AllMusicCaps}: Album Reviews as Complementary Supervision for Music {CLAP}},
  author = {Alonso-Jim{\'e}nez, Pablo and Lizarraga-Seijas, Xavier and Serra, Xavier and Bogdanov, Dmitry},
  booktitle = {International Society for Music Information Retrieval Conference (ISMIR)},
  year = {2026},
}