ESPnet
audio
self-supervised-learning
beats
jaeyeonkim99 commited on
Commit
5e944f8
·
verified ·
1 Parent(s): cb54893

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +53 -0
README.md ADDED
@@ -0,0 +1,53 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ tags:
3
+ - espnet
4
+ - audio
5
+ - self-supervised-learning
6
+ - beats
7
+ datasets:
8
+ - audioset
9
+ license: cc-by-4.0
10
+ arxiv: 2212.09058
11
+ ---
12
+
13
+ # BEATs base encoder — AudioSet-2M, iteration 0
14
+
15
+ BEATs base audio encoder pre-trained on AudioSet-2M with ESPnet, reproducing
16
+ [BEATs](https://arxiv.org/abs/2212.09058) (Table 4). Iteration 0 is trained against
17
+ random-projection targets. Produced by the [egs2/audioset/ssl1](https://github.com/espnet/espnet/tree/master/egs2/audioset/ssl1) recipe.
18
+
19
+ ## Usage
20
+
21
+ Load into an ESPnet downstream classification recipe (e.g. `egs2/esc50/asr1`,
22
+ `egs2/as20k/cls1`) via `beats_ckpt_path`:
23
+
24
+ ```yaml
25
+ encoder: beats
26
+ encoder_conf:
27
+ beats_ckpt_path: beats_encoder_iter0.pt
28
+ fbank_mean: 15.66439
29
+ fbank_std: 6.38312
30
+ ```
31
+
32
+ ## Downstream results
33
+
34
+ | | AS-20K test mAP | ESC-50 (5-fold acc) |
35
+ |---|---|---|
36
+ | iter 0 | 29.72 | 93.35 |
37
+
38
+ ## Citation
39
+
40
+ ```bibtex
41
+ @inproceedings{chen2022beats,
42
+ title={BEATs: Audio Pre-Training with Acoustic Tokenizers},
43
+ author={Chen, Sanyuan and Wu, Yu and Wang, Chengyi and Liu, Shujie and
44
+ Tompkins, Daniel and Chen, Zhuo and Wei, Furu},
45
+ booktitle={ICML}, year={2023}
46
+ }
47
+ @inproceedings{bharadwaj2025openbeats,
48
+ title={OpenBEATs: A Fully Open-Source General-Purpose Audio Encoder},
49
+ author={Bharadwaj, Shikhar and Cornell, Samuele and Choi, Kwanghee and
50
+ Fukayama, Satoru and Shim, Hye-jin and Deshmukh, Soham and Watanabe, Shinji},
51
+ booktitle={WASPAA}, year={2025}
52
+ }
53
+ ```