| | --- |
| | license: apache-2.0 |
| | library_name: diffusers |
| | pipeline_tag: image-to-video |
| | tags: |
| | - normal-estimation |
| | - video |
| | - diffusion |
| | - svd |
| | --- |
| | |
| | # NormalCrafter — Video Normal Map Estimation |
| |
|
| | Mirror of [Yanrui95/NormalCrafter](https://huggingface.co/Yanrui95/NormalCrafter) hosted by [AEmotionStudio](https://huggingface.co/AEmotionStudio) for use with [ComfyUI-FFMPEGA](https://github.com/AEmotionStudio/ComfyUI-FFMPEGA). |
| |
|
| | ## Model Description |
| |
|
| | NormalCrafter generates **temporally consistent surface normal maps** from video using a Stable Video Diffusion (SVD) backbone fine-tuned for normal estimation. Unlike image-based methods (e.g., Marigold), NormalCrafter operates natively on video sequences, producing smooth frame-to-frame normals without flickering. |
| |
|
| | ## Key Features |
| |
|
| | - **Video-native**: Processes temporal sequences for coherent normals across frames |
| | - **SVD backbone**: Built on `stabilityai/stable-video-diffusion-img2vid-xt` |
| | - **High resolution**: Supports up to 1024px inference |
| | - **Apache-2.0 Licensed**: Free for commercial and personal use |
| |
|
| | ## Model Files |
| |
|
| | | File | Size | Description | |
| | |------|------|-------------| |
| | | `unet/diffusion_pytorch_model.safetensors` | 3.05 GB | Fine-tuned UNet for normal estimation | |
| | | `image_encoder/model.fp16.safetensors` | 1.26 GB | CLIP image encoder (fp16) | |
| | | `vae/diffusion_pytorch_model.safetensors` | 196 MB | VAE decoder | |
| |
|
| | ## Usage in ComfyUI-FFMPEGA |
| |
|
| | NormalCrafter is available as: |
| | - **Standalone skill**: `normalcrafter` in the FFMPEGA agent |
| | - **No-LLM mode**: Select `normalcrafter` in the agent node dropdown |
| | - **AI Relighting**: Enable "Use NormalCrafter" in the Video Editor's Relight panel for physically-based relighting |
| |
|
| | ## Citation |
| |
|
| | ```bibtex |
| | @article{normalcrafter2024, |
| | title={NormalCrafter: Learning Temporally Consistent Normals from Video Diffusion Priors}, |
| | author={Yanrui Bin and Wenbo Hu and Haoyuan Wang and Xinya Chen and Bing Wang}, |
| | year={2024} |
| | } |
| | ``` |
| |
|
| | ## License |
| |
|
| | - **Model weights** (this repo): **Apache-2.0** — matching the upstream [Yanrui95/NormalCrafter](https://huggingface.co/Yanrui95/NormalCrafter) HuggingFace repo. See [LICENSE](LICENSE). |
| | - **Source code**: **MIT** — as published at [Binyr/NormalCrafter](https://github.com/Binyr/NormalCrafter) on GitHub. |
| |
|
| | Both licenses are permissive and allow commercial use. |
| |
|
| | ## Links |
| |
|
| | - **Paper**: [NormalCrafter](https://github.com/Binyr/NormalCrafter) |
| | - **Upstream weights**: [Yanrui95/NormalCrafter](https://huggingface.co/Yanrui95/NormalCrafter) |
| | - **ComfyUI-FFMPEGA**: [AEmotionStudio/ComfyUI-FFMPEGA](https://github.com/AEmotionStudio/ComfyUI-FFMPEGA) |
| |
|
