README.md

metadata

title: Lip Forcing
emoji: 🗣️
colorFrom: indigo
colorTo: green
sdk: gradio
app_file: app.py
pinned: false
python_version: '3.10'
short_description: Few-step autoregressive diffusion for real-time lip sync
startup_duration_timeout: 60m

Lip Forcing

Gradio demo for Lip Forcing: Few-Step Autoregressive Diffusion for Real-time Lip Synchronization (KAIST AI · AIPARK), running the released self-contained 14B student checkpoint.

Given a talking-head reference video and a driving audio clip, the model detects and aligns the face to 512×512, then regenerates the mouth region to match the audio using a causal 2-step autoregressive diffusion student, and composites the result back into the original frames.

• Paper: https://arxiv.org/abs/2606.11180
• Project page: https://cvlab-kaist.github.io/LipForcing/
• Code: https://github.com/cvlab-kaist/LipForcing
• Weights: https://huggingface.co/JinhyukJang/lipforcing

This demo reproduces the official streaming inference pipeline (scripts/inference/inference_streaming.py) 1:1 on ZeroGPU. Driving audio is capped to the first few seconds per request to keep a single call within the GPU time budget.