README.md

---
license: apache-2.0
library_name: MoG
---

# MoG: Motion-Aware Generative Frame Interpolation

<!-- <p style="display: flex; flex-direction: column; justify-content: center; align-items: center;">
  <div style="width: 100%; text-align: center; margin-bottom: 4px;">
    <img src="examples/1.gif" style="zoom:32%;">
    <img src="examples/2.gif" style="zoom:32%;">
    <img src="examples/3.gif" style="zoom:32%;">
  </div>
  <div style="width: 100%; text-align: center;">
    <img src="examples/4.gif" style="zoom:32%;">
    <img src="examples/5.gif" style="zoom:32%;">
    <img src="examples/6.gif" style="zoom:32%;">
  </div>
</p>
 -->
<div style="text-align: center;">
  <img src="examples/1.gif" style="width: 32%; display: inline-block;">
  <img src="examples/2.gif" style="width: 32%; display: inline-block;">
  <img src="examples/3.gif" style="width: 32%; display: inline-block;">
</div>
<div style="text-align: center;">
  <img src="examples/4.gif" style="width: 32%; display: inline-block;">
  <img src="examples/5.gif" style="width: 32%; display: inline-block;">
  <img src="examples/6.gif" style="width: 32%; display: inline-block;">
</div>
 
MoG is a generative video frame interpolation (VFI) model, designed to synthesize intermediate frames between two input frames.

MoG is the first VFI framework to bridge the gap between flow-based stability and generative flexibility. We introduce a dual-level guidance injection design to constrain generated motion using motion trajectories derived from optical flow. To enhance the generative model's ability to dynamically correct flow errors, we implement encoder-only guidance injection and selective parameter fine-tuning. As a result, MoG achieves significant improvements over existing open-source generative VFI methods, delivering superior performance in both real-world and animated scenarios.

Source code is available at [https://github.com/MCG-NJU/MoG-VFI](https://github.com/MCG-NJU/MoG-VFI).

## Network Arichitecture

![pipeline_figure](figs/pipeline_figure.png)

## Model Description

- **Developed by:** Nanjing University, Tencent PCG
- **Model type:** Generative video frame interploation model, takes two still video frames as input.
- **Arxiv paper**: [https://arxiv.org/pdf/2501.03699](https://arxiv.org/pdf/2501.03699)
- **Project page:** [https://mcg-nju.github.io/MoG_Web/](https://mcg-nju.github.io/MoG_Web/)
- **Repository**: [https://github.com/MCG-NJU/MoG-VFI](https://github.com/MCG-NJU/MoG-VFI)
- **License:** Apache 2.0 license.

# Usage

We provide two model checkpoints: `real.ckpt` for real-world scenes and `ani.ckpt` for animation scenes. For detailed instructions on loading the checkpoints and performing inference, please refer to our [official repository](https://github.com/MCG-NJU/MoG-VFI).

## Citation

If you find our code useful or our work relevant, please consider citing:

```
@article{zhang2025motion,
  title={Motion-Aware Generative Frame Interpolation},
  author={Zhang, Guozhen and Zhu, Yuhan and Cui, Yutao and Zhao, Xiaotong and Ma, Kai and Wang, Limin},
  journal={arXiv preprint arXiv:2501.03699},
  year={2025}
}
```