YuanzeLin/Illumicraft-checkpoints

IllumiCraft Checkpoints

This repository contains the pretrained checkpoints for:

IllumiCraft: Unified Geometry and Illumination Diffusion for Controllable Video Generation

Yuanze Lin, Yi-Wen Chen, Yi-Hsuan Tsai, Ronald Clark, Ming-Hsuan Yang

🔗 Links

• 📄 Paper: https://arxiv.org/abs/2506.03150
• 🌐 Project Page: https://yuanze-lin.me/IllumiCraft_page/
• 💻 GitHub: https://github.com/yuanze-lin/IllumiCraft
• 🎥 YouTube: https://youtu.be/qAV58sADEzo
• 🤗 Dataset: https://huggingface.co/datasets/YuanzeLin/IllumiCraft

✨ Overview

IllumiCraft is a controllable video generation framework that jointly models scene geometry and illumination dynamics. By leveraging foreground appearance, HDR lighting maps, and 3D point tracks, IllumiCraft enables temporally consistent video generation with controllable motion and relighting effects.

🚀 Usage

Please refer to the official GitHub repository for installation, dataset preparation, training, and inference instructions:

https://github.com/yuanze-lin/IllumiCraft

📖 Citation

If you find IllumiCraft useful for your research, please consider citing:

@article{lin2026illumicraft,
  title={IllumiCraft: Unified Geometry and Illumination Diffusion for Controllable Video Generation},
  author={Lin, Yuanze and Chen, Yi-Wen and Tsai, Yi-Hsuan and Clark, Ronald and Yang, Ming-Hsuan},
  journal={Advances in Neural Information Processing Systems},
  volume={38},
  pages={27798--27829},
  year={2026}
}