README.md

---
license: mit
base_model: laion/clap-htsat-unfused
library_name: peft
tags:
  - clap
  - audio-retrieval
  - music-information-retrieval
  - lora
  - contrastive
pipeline_tag: feature-extraction
---

# Crate — CLAP LoRA for producer vocabulary

LoRA adapter over [`laion/clap-htsat-unfused`](https://huggingface.co/laion/clap-htsat-unfused) fine-tuned so producer terms base CLAP
barely knows — *boom-bap, tape-saturated, rimshot, reese bass* — pull the right
audio. Part of [Crate](https://github.com/jahnavi-yelamanchi/crate), sound-native
search for music producers (hum / drop a track / describe it → one embedding space).

## Results — producer-vocab retrieval (held-out)

| metric | base CLAP | fine-tuned |
|---|---|---|
| recall@1 | 0.210 | 0.405 |
| recall@10 | 0.746 | 0.951 |
| recall@5 | 0.580 | 0.868 |

## Usage

```python
from transformers import ClapModel, ClapProcessor
from peft import PeftModel

base = "laion/clap-htsat-unfused"
model = PeftModel.from_pretrained(ClapModel.from_pretrained(base), "jahnaviym/crate-clap-lora").merge_and_unload()
proc = ClapProcessor.from_pretrained(base)
# proc(text=[...]) / proc(audios=[...], sampling_rate=48000) → get_text/audio_features
```

## Training

- Symmetric InfoNCE on (audio, text) pairs from Freesound (CC0/CC-BY) + free packs.
- LoRA on the audio/text projection heads (r=16, alpha=32).
- Label-preserving augmentation (pitch/stretch/noise/EQ) for positive pairs.

See the repo for the full pipeline and eval.