pip install -U diffusers transformers accelerate
from diffusers import ControlNetModel, StableDiffusionControlNetPipeline
controlnet = ControlNetModel.from_pretrained("sbharadwaj/genlit")
pipe = StableDiffusionControlNetPipeline.from_pretrained(
"fill-in-base-model", controlnet=controlnet
)You need to agree to share your contact information to access this model
This repository is publicly accessible, but you have to accept the conditions to access its files and content.
By requesting access, you agree to the GenLit Non-Commercial Research License
(https://genlit.is.tue.mpg.de/license.html). This license restricts use to
non-commercial scientific research, education, and artistic projects.
Log in or Sign Up to review the conditions and access this model content.
GenLit: ControlNet weights for object relighting
Three checkpoints are released, details below. Paper · Code on GitHub
| Subfolder | Base model | Resolution | Frames | Use case |
|---|---|---|---|---|
single_object/ |
stabilityai/stable-video-diffusion-img2vid |
512×512 | 14 | single object |
mit/ |
stabilityai/stable-video-diffusion-img2vid |
768×512 | 14 | for MIT Multi-Illumination dataset |
multi_object/ |
stabilityai/stable-video-diffusion-img2vid-xt |
640×448 | 25 | multiple objects scattered in the scene |
Usage
Install the genlit package, then run:
pip install -e . # from the genlit GitHub repo
huggingface-cli login # gated repo — accept license first
python -m genlit.inference --mode single --img_json examples/single.json --output_dir out/
Weights download automatically from this repo on first run.
Citation
@inproceedings{genlit:sigasia:2025,
title = {{GenLit}: Reformulating Single Image Relighting as Video Generation},
author = {Bharadwaj, Shrisha and Feng, Haiwen and Becherini, Giorgio and
Abrevaya, Victoria Fernandez and Black, Michael J.},
year = {2025},
booktitle = {SIGGRAPH Asia Conference Papers '25},
doi = {10.1145/3757377.3763970},
}
License
Non-commercial research only. See https://genlit.is.tue.mpg.de/license.html.
- Downloads last month
- -
# Gated model: Login with a HF token with gated access permission hf auth login