| --- |
| tags: |
| - lip-sync |
| - emotion |
| - talking-face |
| - facial-animation |
| license: mit |
| --- |
| |
| # EMOLIPS: Emotion-Driven Facial Animation for Realistic Lip-Sync Synthesis |
|
|
| ## Overview |
| Emotion-controllable lip-sync framework using FiLM-based modulation of 3DMM expression coefficients. |
|
|
| ## Architecture |
| - **Backbone:** SadTalker (3DMM-based talking face) |
| - **Novel Module:** Emotion-Conditioned Fusion Module (ECFM) |
| - **Conditioning:** FiLM layers + Lip Consistency Gate |
| - **Emotions:** Neutral, Happy, Sad, Angry, Fear, Surprise, Disgust |
|
|
| ## Results |
| See `EMOLIPS_Submission.ipynb` for full methodology, architecture diagrams, outputs, and metrics. |
|
|
| ### Output Samples |
| | Emotion | Video | |
| |---------|-------| |
| | Neutral | [emolips_neutral.mp4](outputs/emolips_neutral.mp4) | |
| | Happy | [emolips_happy.mp4](outputs/emolips_happy.mp4) | |
| | Sad | [emolips_sad.mp4](outputs/emolips_sad.mp4) | |
| | Angry | [emolips_angry.mp4](outputs/emolips_angry.mp4) | |
| | Fear | [emolips_fear.mp4](outputs/emolips_fear.mp4) | |
| | Surprise | [emolips_surprise.mp4](outputs/emolips_surprise.mp4) | |
| | Disgust | [emolips_disgust.mp4](outputs/emolips_disgust.mp4) | |
|
|
| ## Quick Start |
| ```bash |
| git clone https://huggingface.co/primal-sage/emolips |
| cd emolips/code |
| bash setup.sh |
| python inference.py --audio ../samples/input_audio.wav --image ../samples/input_face.jpg --all-emotions |
| ``` |
|
|
| ## Citation |
| ```bibtex |
| @article{emolips2026, |
| title={EMOLIPS: Emotion-Driven Facial Animation for Realistic Lip-Sync Synthesis}, |
| year={2026} |
| } |
| ``` |
|
|
