| title: SHeaP - Self-Supervised Head Geometry Predictor | |
| emoji: 🐑 | |
| colorFrom: blue | |
| colorTo: green | |
| sdk: gradio | |
| sdk_version: "5.50.0" | |
| app_file: gradio_demo.py | |
| pinned: false | |
| license: mit | |
| # SHeaP: Self-Supervised Head Geometry Predictor Learned via 2D Gaussians | |
| Upload an image or video to predict head geometry and render a 3D FLAME mesh overlay! | |
| **Liam Schoneveld, Zhe Chen, Davide Davoli, Jiapeng Tang, Saimon Terazawa, Ko Nishino, Matthias Nießner** | |
| - [Project Page](https://nlml.github.io/sheap) | |
| - [Paper](https://arxiv.org/abs/2504.12292) | |
| - [GitHub Repository](https://github.com/nlml/sheap) | |
| ## About | |
| SHeaP learns to predict head geometry (FLAME parameters) from a single image by predicting and rendering 2D Gaussians. | |
| The output shows three views: | |
| - **Left**: Original cropped face | |
| - **Center**: Rendered FLAME mesh | |
| - **Right**: Mesh overlaid on original | |
| ## Setup Instructions | |
| Before deploying to Hugging Face Spaces, you need to: | |
| 1. Download the FLAME model from [FLAME 2020](https://flame.is.tue.mpg.de/) | |
| 2. Convert it using `python convert_flame.py` | |
| 3. Include the `FLAME2020/` directory with the required files: | |
| - `generic_model.pt` | |
| - `eyelids.pt` | |
| - `flame_landmark_idxs_barys.pt` | |
| 4. Include the `models/` directory with: | |
| - `model_expressive.pt` | |
| - `model_paper.pt` | |
| - `model_lightweight.pt` (if available) | |