| --- |
| license: other |
| license_name: mixed-upstream-licenses |
| license_link: https://huggingface.co/lefreud/GimbalDiffusion/blob/main/LICENSE.md |
| pipeline_tag: text-to-video |
| base_model: |
| - THUDM/CogVideoX-2b |
| - Wan-AI/Wan2.1-T2V-1.3B |
| - Wan-AI/Wan2.2-TI2V-5B |
| datasets: |
| - YOUSIKI/PanoWan |
| tags: |
| - video-generation |
| - camera-control |
| - camera-trajectory |
| - gravity |
| - gimbaldiffusion |
| --- |
| |
| # GimbalDiffusion |
|
|
| Paper-release checkpoints for **GimbalDiffusion: Gravity-Aware Camera Control for Video Generation** (ECCV 2026). GimbalDiffusion is implemented across CogVideoX-2B, WAN 2.1, and WAN 2.2 using both Plücker-ray and UCPE camera encodings. |
|
|
| - [Project page](https://lvsn.github.io/GimbalDiffusion/) |
| - [Paper](https://arxiv.org/abs/2512.09112) |
| - [Code](https://github.com/lvsn/GimbalDiffusion) |
|
|
| ## Checkpoints |
|
|
| | Path | Backbone | Variant | Size (GB) | |
| | --- | --- | --- | ---: | |
| | `gimbal_ac3d/controlnet.pt` | CogVideoX-2b / AC3D | Main | 0.022 | |
| | `gimbal_ucpe/dit.safetensors` | Wan 2.1 / UCPE | Main | 2.909 | |
| | `gimbal_ucpe_no_np/dit.safetensors` | Wan 2.1 / UCPE | Without null-pitch conditioning | 2.909 | |
| | `gimbal_ucpe_no_absolute/dit.safetensors` | Wan 2.1 / UCPE | Relative poses | 2.909 | |
| | `gimbal_wan_2_1/dit.safetensors` | Wan 2.1 | Main | 2.942 | |
| | `gimbal_wan_2_2/dit.safetensors` | Wan 2.2 | Main | 10.007 | |
| | `gimbal_wan_2_2_no_np/dit.safetensors` | Wan 2.2 | Without null-pitch conditioning | 10.007 | |
|
|
| ## Download |
|
|
| ```bash |
| hf download lefreud/GimbalDiffusion --local-dir checkpoint/gimbaldiffusion |
| hf download THUDM/CogVideoX-2b --local-dir checkpoint/backbones/THUDM/CogVideoX-2b |
| hf download Wan-AI/Wan2.1-T2V-1.3B --local-dir checkpoint/backbones/Wan-AI/Wan2.1-T2V-1.3B |
| hf download Wan-AI/Wan2.2-TI2V-5B --local-dir checkpoint/backbones/Wan-AI/Wan2.2-TI2V-5B |
| ``` |
|
|
| For setup, inference, and training instructions, see the [GimbalDiffusion README](https://github.com/lvsn/GimbalDiffusion#readme). |
|
|
| ## License |
|
|
| This repository contains artifacts governed by different upstream terms. In particular, the AC3D checkpoint is restricted to non-commercial research use. Read [LICENSE.md](LICENSE.md) and [LICENSE-AC3D.txt](LICENSE-AC3D.txt) before use. Base-model weights are not redistributed here and retain their own licenses. |
|
|
| ## Citation |
|
|
| ```bibtex |
| @inproceedings{fortierchouinard2026gimbaldiffusion, |
| title = {GimbalDiffusion: Gravity-Aware Camera Control for Video Generation}, |
| author = {Fortier-Chouinard, Fr{\'e}d{\'e}ric and Hold-Geoffroy, Yannick and Deschaintre, Valentin and Gadelha, Matheus and Lalonde, Jean-Fran{\c{c}}ois}, |
| booktitle = {European Conference on Computer Vision (ECCV)}, |
| year = {2026} |
| } |
| ``` |
|
|
