Add model card and metadata for SymphoMotion
#1
by nielsr HF Staff - opened
README.md
ADDED
|
@@ -0,0 +1,54 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
pipeline_tag: image-to-video
|
| 4 |
+
---
|
| 5 |
+
|
| 6 |
+
# SymphoMotion: Joint Control of Camera Motion and Object Dynamics for Coherent Video Generation
|
| 7 |
+
|
| 8 |
+
SymphoMotion is a unified motion-control framework that jointly governs camera trajectories and object dynamics within a single model. It features a Camera Trajectory Control mechanism that integrates explicit camera paths with geometry-aware cues, and an Object Dynamics Control mechanism that combines 2D visual guidance with 3D trajectory embeddings.
|
| 9 |
+
|
| 10 |
+
[](https://grenoble-zhang.github.io/SymphoMotion/)
|
| 11 |
+
[](https://arxiv.org/abs/2604.03723)
|
| 12 |
+
[](https://github.com/grenoble-zhang/SymphoMotion)
|
| 13 |
+
|
| 14 |
+
<img src="https://raw.githubusercontent.com/Grenoble-Zhang/SymphoMotion/main/img/method.png" width="100%"/>
|
| 15 |
+
|
| 16 |
+
## Quick Start & Installation
|
| 17 |
+
|
| 18 |
+
To set up the environment and use these checkpoints, please refer to the instructions below or check out the [Official Codebase](https://github.com/grenoble-zhang/SymphoMotion).
|
| 19 |
+
|
| 20 |
+
```bash
|
| 21 |
+
git clone https://github.com/Grenoble-Zhang/SymphoMotion.git
|
| 22 |
+
cd SymphoMotion
|
| 23 |
+
|
| 24 |
+
# Create conda environment
|
| 25 |
+
conda create -n symphomotion python=3.10
|
| 26 |
+
conda activate symphomotion
|
| 27 |
+
|
| 28 |
+
# Install dependencies
|
| 29 |
+
pip install -r requirements.txt
|
| 30 |
+
```
|
| 31 |
+
|
| 32 |
+
### Download Models
|
| 33 |
+
|
| 34 |
+
```bash
|
| 35 |
+
# Base model
|
| 36 |
+
huggingface-cli download Wan-AI/Wan2.1-I2V-14B-720P-Diffusers \
|
| 37 |
+
--local-dir pretrained_models/Wan2.1-I2V-14B-720P-Diffusers
|
| 38 |
+
|
| 39 |
+
# SymphoMotion checkpoints
|
| 40 |
+
huggingface-cli download fateforward/Symphomotion \
|
| 41 |
+
--include "pretrained_checkpoints/*" \
|
| 42 |
+
--local-dir .
|
| 43 |
+
```
|
| 44 |
+
|
| 45 |
+
## Citation
|
| 46 |
+
|
| 47 |
+
```bibtex
|
| 48 |
+
@article{zhang2026symphomotion,
|
| 49 |
+
title={SymphoMotion: Joint Control of Camera Motion and Object Dynamics for Coherent Video Generation},
|
| 50 |
+
author={Zhang, Guiyu and Chen, Yabo and Xiang, Xunzhi and Huang, Junchao and Wang, Zhongyu and Jiang, Li},
|
| 51 |
+
journal={arXiv preprint arXiv:2604.03723},
|
| 52 |
+
year={2026}
|
| 53 |
+
}
|
| 54 |
+
```
|