README.md

Sci-Fi: Symmetric Constraint for Frame Inbetweening

Liuhan Chen¹, Xiaodong Cun^2,*, Xiaoyu Li³, Xianyi He^1,4, Shenghai Yuan^1,4, Jie Chen¹, Ying Shan³, Li Yuan^1,*

¹Shenzhen Graduate School, Peking University     ²GVC Lab, Great Bay University     ³ARC Lab, Tencent PCG     ⁴Rabbitpre Intelligence

We have updated our paper with a new version and chane the name of our framework from Sci-Fi to EF-VI. Arxiv | PDF

Video demos

or click here to download the compressed version

Method comparison

(a) In current I2V-DM-based methods, the end-frame constraint is weaker than the start-frame constraint due to the same injection mechanism but a smaller training scale, causing a distorted predicted path with collapsed content.
(b) Our Sci-Fi maintains start frame processing while enhancing end-frame constraint injection. This achieves symmetric start-end-frame constraints with small training, yielding a fine predicted path close to the real one with smoother inbetweening.

Some challenging examples of our Sci-Fi for frame inbetweening.

Start frame	End frame	Generated video

Deployment for personal use

1. Setup the repository and environment

git clone https://github.com/GVCLab/Sci-Fi.git
cd Sci-Fi
conda create -n Sci-Fi python==3.12
conda activate Sci-Fi
pip install -r requirements.txt

2. Download checkpoint

Download the CogVideoX-5B-I2V model (due to fine-tuning, the weights of the transformer denoiser are different from the original) and EF-Net. The weights are available at 🤗HuggingFace and 🤖ModelScope.

3. Launch the inference script!

The example input keyframe pairs are in examples/ folder, and the corresponding generated videos (720x480, 49 frames) are placed in outputs/ folder.
To interpolate, run:

bash Sci_Fi_frame_inbetweening.sh

Citation

🌟 If you find our work helpful, please leave us a star and cite our paper.

@article{chen2025sci,
  title={Sci-Fi: Symmetric Constraint for Frame Inbetweening},
  author={Chen, Liuhan and Cun, Xiaodong and Li, Xiaoyu and He, Xianyi and Yuan, Shenghai and Chen, Jie and Shan, Ying and Yuan, Li},
  journal={arXiv preprint arXiv:2505.21205},
  year={2025}
}