Update README.md
Browse files
README.md
CHANGED
|
@@ -2,4 +2,50 @@
|
|
| 2 |
license: cc-by-nc-4.0
|
| 3 |
language:
|
| 4 |
- en
|
| 5 |
-
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 2 |
license: cc-by-nc-4.0
|
| 3 |
language:
|
| 4 |
- en
|
| 5 |
+
---
|
| 6 |
+
## Baseline model for 4th COG-MHEAR Audio-Visual Speech Enhancement Challenge
|
| 7 |
+
|
| 8 |
+
[Challenge link](https://challenge.cogmhear.org/)
|
| 9 |
+
|
| 10 |
+
## Requirements
|
| 11 |
+
* [Python >= 3.6](https://www.anaconda.com/docs/getting-started/miniconda/install)
|
| 12 |
+
* [PyTorch](https://pytorch.org/)
|
| 13 |
+
* [PyTorch Lightning](https://lightning.ai/docs/pytorch/latest/)
|
| 14 |
+
* [Decord](https://github.com/dmlc/decord)
|
| 15 |
+
* [Hydra](https://hydra.cc)
|
| 16 |
+
* [SpeechBrain](https://github.com/speechbrain/speechbrain)
|
| 17 |
+
* [TQDM](https://github.com/tqdm/tqdm)
|
| 18 |
+
|
| 19 |
+
## Usage
|
| 20 |
+
|
| 21 |
+
```bash
|
| 22 |
+
# Expected folder structure for the dataset
|
| 23 |
+
data_root
|
| 24 |
+
|-- train
|
| 25 |
+
| `-- scenes
|
| 26 |
+
|-- dev
|
| 27 |
+
| `-- scenes
|
| 28 |
+
|-- eval
|
| 29 |
+
| `-- scenes
|
| 30 |
+
```
|
| 31 |
+
|
| 32 |
+
### Clone the repo
|
| 33 |
+
```bash
|
| 34 |
+
git clone https://github.com/cogmhear/avse_challenge
|
| 35 |
+
cd avse_challenge/baseline/avse4
|
| 36 |
+
```
|
| 37 |
+
|
| 38 |
+
### Train
|
| 39 |
+
```bash
|
| 40 |
+
python train.py data.root="./avsec4" data.num_channels=2 trainer.log_dir="./logs" data.batch_size=8 trainer.accelerator gpu trainer.gpus 1
|
| 41 |
+
|
| 42 |
+
more arguments in conf/train.yaml
|
| 43 |
+
```
|
| 44 |
+
|
| 45 |
+
### Test
|
| 46 |
+
```bash
|
| 47 |
+
python test.py data.root=./avsec4 data.num_channels=2 ckpt_path=pretrained.ckpt save_dir="./eval" model_uid="./avse4"
|
| 48 |
+
|
| 49 |
+
more arguments in conf/eval.yaml
|
| 50 |
+
```
|
| 51 |
+
|