README.md

metadata

license: cc-by-nc-4.0
language:
  - en

Baseline model for 4th COG-MHEAR Audio-Visual Speech Enhancement Challenge

Challenge link

Requirements

• Python >= 3.6
• PyTorch
• PyTorch Lightning
• Decord
• Hydra
• SpeechBrain
• TQDM

Usage

# Expected folder structure for the dataset
data_root
|-- train
|   `-- scenes
|-- dev
|   `-- scenes
|-- eval
|   `-- scenes

Clone the repo

git clone https://github.com/cogmhear/avse_challenge
cd avse_challenge/baseline/avse4

Train

python train.py data.root="./avsec4" data.num_channels=2 trainer.log_dir="./logs" data.batch_size=8 trainer.accelerator gpu trainer.gpus 1

more arguments in conf/train.yaml

Test

python test.py data.root=./avsec4 data.num_channels=2 ckpt_path=pretrained.ckpt save_dir="./eval" model_uid="./avse4" 

more arguments in conf/eval.yaml