---
license: cc-by-nc-sa-4.0
tags:
  - music
  - audio
  - contrastive
  - clap
  - music-information-retrieval
library_name: amclap
---

# AllMusicCaps: Baseline + Structured captions

Baseline plus LLM-structured captions.

Audio-text contrastive (CLAP) model over the OMAR-RQ audio encoder and an
`all-mpnet-base-v2` text encoder, trained on music with album-review derived
captions. Embeddings are 512-dimensional; audio is expected at 24 kHz.

## Usage

```python
from amclap import get_model

model = get_model(model_id="mtg-upf/allmusiccaps_baseline_struct", device="cpu")

audio_emb = model.forward_audio(audio)   # (B, 512), audio at 24 kHz
text_emb = model.forward_text(["warm analog synth pads"])  # (B, 512)
```

## Training

| | |
|---|---|
| Text encoder | frozen (stock all-mpnet-base-v2) |
| Reported checkpoint | step 149,796 |
| Audio encoder layers | see `config.gin` |

The text encoder was frozen during training, so stock `all-mpnet-base-v2` weights are reloaded at construction.

## License

CC BY-NC-SA 4.0. See `LICENSE`.

## Citation

```bibtex
@inproceedings{alonso2026allmusiccaps,
  title = {{AllMusicCaps}: Album Reviews as Complementary Supervision for Music {CLAP}},
  author = {Alonso-Jim{\'e}nez, Pablo and Lizarraga-Seijas, Xavier and Serra, Xavier and Bogdanov, Dmitry},
  booktitle = {International Society for Music Information Retrieval Conference (ISMIR)},
  year = {2026},
}
```