| --- |
| tags: |
| - espnet |
| - audio |
| - self-supervised-learning |
| - beats |
| datasets: |
| - audioset |
| license: cc-by-4.0 |
| arxiv: 2212.09058 |
| --- |
| |
| # BEATs base encoder — AudioSet-2M, iteration 1 |
|
|
| BEATs base audio encoder pre-trained on AudioSet-2M with ESPnet, reproducing |
| [BEATs](https://arxiv.org/abs/2212.09058) (Table 4). Iteration 1 is trained against the |
| discrete targets of the [iteration-1 VQ tokenizer](https://huggingface.co/jaeyeonkim99/BEATs-tokenizer-AS2M-iter1). |
| Produced by the [egs2/audioset/ssl1](https://github.com/espnet/espnet/tree/master/egs2/audioset/ssl1) recipe. |
|
|
| ## Usage |
|
|
| Load into an ESPnet downstream classification recipe (e.g. `egs2/esc50/asr1`, |
| `egs2/as20k/cls1`) via `beats_ckpt_path`: |
|
|
| ```yaml |
| encoder: beats |
| encoder_conf: |
| beats_ckpt_path: beats_encoder_iter1.pt |
| fbank_mean: 15.66439 |
| fbank_std: 6.38312 |
| ``` |
|
|
| ## Downstream results |
|
|
| | | AS-20K test mAP | ESC-50 (5-fold acc) | |
| |---|---|---| |
| | iter 1 | 31.26 | 93.35 | |
|
|
| ## Citation |
|
|
| ```bibtex |
| @inproceedings{chen2022beats, |
| title={BEATs: Audio Pre-Training with Acoustic Tokenizers}, |
| author={Chen, Sanyuan and Wu, Yu and Wang, Chengyi and Liu, Shujie and |
| Tompkins, Daniel and Chen, Zhuo and Wei, Furu}, |
| booktitle={ICML}, year={2023} |
| } |
| @inproceedings{bharadwaj2025openbeats, |
| title={OpenBEATs: A Fully Open-Source General-Purpose Audio Encoder}, |
| author={Bharadwaj, Shikhar and Cornell, Samuele and Choi, Kwanghee and |
| Fukayama, Satoru and Shim, Hye-jin and Deshmukh, Soham and Watanabe, Shinji}, |
| booktitle={WASPAA}, year={2025} |
| } |
| ``` |
|
|
