README.md

metadata

title: Dub Module Gradio App
sdk: gradio
app_file: app.py
python_version: '3.10'
pinned: false

Dub Module Gradio App (HF Space Ready)

This folder provides a Hugging Face Spaces-ready Gradio app for the pipeline in instructions.txt.

Implemented workflow:

• Step 1: Extract face and lip coordinates from the original video.
• Step 2: Extract cropped face video using face coordinates and allow downloads of:
  • face coordinates (.pkl)
  • lip coordinates (.pkl)
  • cropped face video (.mp4)
• Step 3: Manual only (not part of the app).
• Step 4: Merge manual Step 3 output back into the original video and generate final downloadable video.
• Step 5: Not part of the app.

Files

• app.py: Gradio interface (Step 1, 2, and 4).
• pipeline.py: Core logic shared by UI and CLI.
• extract_coordinates.py: CLI wrapper for Step 1.
• extract_face.py: CLI wrapper for Step 2.
• merge_lips.py: CLI wrapper for Step 4.
• requirements.txt: Python dependencies.

Local Run

pip install -r requirements.txt
python app.py

Hugging Face Space Setup

1. Create a new Gradio Space.
2. Upload all files from this folder to the root of the Space.
3. Ensure README.md and requirements.txt are present.
4. Space will auto-build and run app.py.

CLI Usage (Optional)

Step 1:

python extract_coordinates.py --video input.mp4 --output-dir outputs

Step 2:

python extract_face.py --video input.mp4 --face-coords outputs/face_coords_avg.pkl --output outputs/cropped_face.mp4

Step 4:

python merge_lips.py --original-video input.mp4 --lip-synced-video lipsynced_face.mp4 --face-coords outputs/face_coords_avg.pkl --lip-coords outputs/lip_coords_avg.pkl --output outputs/final.mp4 --audio tts.wav

Notes

• Coordinates are generated per video and should not be reused across unrelated videos.
• If no external audio is uploaded in Step 4, the app attempts to pull audio from the lip-synced video, then from the original video.
• Generated files are stored in work/ during runtime.