Hugging Face's logo

Spaces:
SetuG
/
sadtalker
Build error

App Files Community
sadtalker / README.md
SetuG's picture
SetuG
Update README.md
22700ae verified
preview code
|
raw
history blame contribute delete
2.5 kB
metadata 
title: SadTalker
emoji: 😭
colorFrom: blue
colorTo: red
sdk: gradio
sdk_version: 6.5.1
app_file: app.py
pinned: false
license: mit

SadTalker: Learning Realistic 3D Motion Coefficients for Stylized Audio-Driven Single Image Talking Face Animation (CVPR 2023)

This is a Gradio app for SadTalker, which can generate talking face videos from a single image and audio.

Features

  • Generate talking face videos from single image + audio
  • Multiple preprocessing options
  • Face enhancement with GFPGAN
  • Multiple pose styles
  • Still mode for fewer head movements

Setup

After cloning this repository to your Hugging Face Space, you'll need to:

  1. Upload model files: Download the following model files and upload them to your repository:

Required Model Files:

SadTalker Models (upload to checkpoints/ folder):

  • SadTalker_V0.0.2_256.safetensors
  • SadTalker_V0.0.2_512.safetensors
  • mapping_00109-model.pth.tar
  • mapping_00229-model.pth.tar

GFPGAN Models (upload to gfpgan/weights/ folder):

  • alignment_WFLW_4HG.pth
  • detection_Resnet50_Final.pth
  • GFPGANv1.4.pth
  • parsing_parsenet.pth

Where to get the models:

  1. Download from the original SadTalker repository: https://github.com/OpenTalker/SadTalker
  2. Or from the model links provided in their documentation

Upload Instructions:

  1. Go to your Hugging Face Space repository
  2. Click "Upload files"
  3. Create the folder structure and upload the model files
  4. Make sure the files are in the correct paths as listed above

Usage

  1. Upload a source image (preferably a portrait with clear face)
  2. Upload an audio file
  3. Adjust settings as needed
  4. Click Generate to create your talking face video

Citation 

@InProceedings{zhang2023sadtalker,
  author={Zhang, Wenxuan and Cun, Xiaodong and Wang, Xuan and Zhang, Yong and Shen, Xi and Guo, Yu and Shan, Ying and Wang, Fei},
  title={SadTalker: Learning Realistic 3D Motion Coefficients for Stylized Audio-Driven Single Image Talking Face Animation},
  booktitle={The IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  month={June},
  year={2023}
}

Links