metadata
license: cc-by-nc-sa-4.0
tags:
- video-to-audio generation
- selective sound generation
- multimodal deep learning
SelVA: Hear What Matters! Text-conditioned Selective Video-to-Audio Generation
.
├── weights/
│ ├── video_enc_sup_5.pth # text-conditioned video encoder
│ ├── generator_small_16k_sup_5.pth # v2a generator
│ ├── generator_small_44k_sup_5.pth
│ ├── generator_medium_44k_sup_5.pth
│ └── generator_large_44k_sup_5.pth
└── ext_weights/
├── synchformer_state_dict.pth # pretrained Synchformer (24-01-04T16-39-21)
├── best_netG.pt # BigVGAN vocoder
├── v1-16.pth # vae 16kHz
└── v1-44.pth # vae 44kHz