Instructions to use jaeyeonkim99/BEATs-base-AS2M-iter0 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- ESPnet
How to use jaeyeonkim99/BEATs-base-AS2M-iter0 with ESPnet:
unknown model type (must be text-to-speech or automatic-speech-recognition)
- Notebooks
- Google Colab
- Kaggle
Upload README.md with huggingface_hub
Browse files
README.md
ADDED
|
@@ -0,0 +1,53 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
tags:
|
| 3 |
+
- espnet
|
| 4 |
+
- audio
|
| 5 |
+
- self-supervised-learning
|
| 6 |
+
- beats
|
| 7 |
+
datasets:
|
| 8 |
+
- audioset
|
| 9 |
+
license: cc-by-4.0
|
| 10 |
+
arxiv: 2212.09058
|
| 11 |
+
---
|
| 12 |
+
|
| 13 |
+
# BEATs base encoder — AudioSet-2M, iteration 0
|
| 14 |
+
|
| 15 |
+
BEATs base audio encoder pre-trained on AudioSet-2M with ESPnet, reproducing
|
| 16 |
+
[BEATs](https://arxiv.org/abs/2212.09058) (Table 4). Iteration 0 is trained against
|
| 17 |
+
random-projection targets. Produced by the [egs2/audioset/ssl1](https://github.com/espnet/espnet/tree/master/egs2/audioset/ssl1) recipe.
|
| 18 |
+
|
| 19 |
+
## Usage
|
| 20 |
+
|
| 21 |
+
Load into an ESPnet downstream classification recipe (e.g. `egs2/esc50/asr1`,
|
| 22 |
+
`egs2/as20k/cls1`) via `beats_ckpt_path`:
|
| 23 |
+
|
| 24 |
+
```yaml
|
| 25 |
+
encoder: beats
|
| 26 |
+
encoder_conf:
|
| 27 |
+
beats_ckpt_path: beats_encoder_iter0.pt
|
| 28 |
+
fbank_mean: 15.66439
|
| 29 |
+
fbank_std: 6.38312
|
| 30 |
+
```
|
| 31 |
+
|
| 32 |
+
## Downstream results
|
| 33 |
+
|
| 34 |
+
| | AS-20K test mAP | ESC-50 (5-fold acc) |
|
| 35 |
+
|---|---|---|
|
| 36 |
+
| iter 0 | 29.72 | 93.35 |
|
| 37 |
+
|
| 38 |
+
## Citation
|
| 39 |
+
|
| 40 |
+
```bibtex
|
| 41 |
+
@inproceedings{chen2022beats,
|
| 42 |
+
title={BEATs: Audio Pre-Training with Acoustic Tokenizers},
|
| 43 |
+
author={Chen, Sanyuan and Wu, Yu and Wang, Chengyi and Liu, Shujie and
|
| 44 |
+
Tompkins, Daniel and Chen, Zhuo and Wei, Furu},
|
| 45 |
+
booktitle={ICML}, year={2023}
|
| 46 |
+
}
|
| 47 |
+
@inproceedings{bharadwaj2025openbeats,
|
| 48 |
+
title={OpenBEATs: A Fully Open-Source General-Purpose Audio Encoder},
|
| 49 |
+
author={Bharadwaj, Shikhar and Cornell, Samuele and Choi, Kwanghee and
|
| 50 |
+
Fukayama, Satoru and Shim, Hye-jin and Deshmukh, Soham and Watanabe, Shinji},
|
| 51 |
+
booktitle={WASPAA}, year={2025}
|
| 52 |
+
}
|
| 53 |
+
```
|