metadata
license: apache-2.0
pipeline_tag: image-to-video
tags:
- video-generation
LIFT: Layout-In-Future Video Generation under Large Viewpoint Change via On-Policy Self-Distillation
Given a first frame, users can navigate from the first-frame view along a desired camera path and specify layouts using bounding boxes with local text prompts in the final frame. Then, LIFT generates the intended shot that transitions from the input image to the user-defined last-frame layout following the prescribed camera trajectory.
We introduce LIFT, a unified image-to-video generation framework that complements camera control with Layout-In-FuTure control, enabling users to specify what should appear in a future view and where it should appear.
Model Checkpoints
Our models are built on Wan2.1-Fun-V1.1-1.3B-Control-Camera.
| Model | Description |
|---|---|
LIFT/transformer/ |
Last-frame layout student trained by dual-mode on-policy self-distillation from the dense-layout teacher. |
LIFT_dense_layout_teacher/transformer/ |
Dense-layout teacher fine-tuned with dense per-frame layout |
Download
pip install -U "huggingface_hub[cli]"
# Last-frame layout student (dual-mode OPSD)
hf download Overdog/LIFT --include "LIFT/*" --local-dir models
# Dense-layout teacher
hf download Overdog/LIFT --include "LIFT_dense_layout_teacher/*" --local-dir models
Citation
@article{ji2026lift,
title={LIFT: Layout-In-Future Video Generation under Large Viewpoint Change via On-Policy Self-Distillation},
author={Ji, Shengxiang and Wang, Boyang and Xu, Haiyang and Li, Bingnan and Mao, Yucheng and Chen, Zeyuan and Shan, Xiaojun and Zhang, Xiang and Hua, Gang and Xie, Jianwen and Cheng, Zezhou and Tu, Zhuowen},
journal={arXiv preprint arXiv:2609.38146},
year={2026}
}