|
|
--- |
|
|
license: cc-by-nc-sa-4.0 |
|
|
pipeline_tag: image-to-3d |
|
|
--- |
|
|
|
|
|
# Motion 3-to-4: 3D Motion Reconstruction for 4D Synthesis |
|
|
|
|
|
**Motion 3-to-4** reconstructs 3D motion from video inputs for 4D synthesis, enabling the generation of animated 3D models with realistic motion in a feed-forward manner. |
|
|
|
|
|
[Paper](https://arxiv.org/abs/2601.14253) | [Project Page](https://motion3-to-4.github.io/) | [Code](https://github.com/Inception3D/Motion324) |
|
|
|
|
|
## Abstract |
|
|
|
|
|
Motion 3-to-4 is a feed-forward framework for synthesising high-quality 4D dynamic objects from a single monocular video and an optional 3D reference mesh. It addresses challenges in 4D synthesis by decomposing the task into static 3D shape generation and motion reconstruction. Using a canonical reference mesh, the model learns a compact motion latent representation and predicts per-frame vertex trajectories to recover complete, temporally coherent geometry. A scalable frame-wise transformer further enables robustness to varying sequence lengths. |
|
|
|
|
|
## Quick Start |
|
|
|
|
|
### Installation |
|
|
|
|
|
```bash |
|
|
git clone https://github.com/Inception3D/Motion324.git |
|
|
cd Motion324 |
|
|
|
|
|
conda create -n Motion324 python=3.11 |
|
|
conda activate Motion324 |
|
|
pip install -r requirements.txt |
|
|
|
|
|
# (Optional) Install Hunyuan3D-2.0 modules |
|
|
cd scripts/hy3dgen/texgen/custom_rasterizer && python3 setup.py install && cd ../../../.. |
|
|
cd scripts/hy3dgen/texgen/differentiable_renderer && python3 setup.py install && cd ../../../.. |
|
|
``` |
|
|
|
|
|
### Inference |
|
|
|
|
|
Download the pre-trained checkpoints and place them in `experiments/checkpoints/`. |
|
|
|
|
|
**Reconstruct 4D from an existing mesh and video:** |
|
|
|
|
|
```bash |
|
|
chmod +x ./scripts/4D_from_existing.sh |
|
|
./scripts/4D_from_existing.sh ./examples/chili.glb ./examples/chili.mp4 ./examples/output |
|
|
``` |
|
|
|
|
|
**Generate 4D animation from a single video input (requires Hunyuan):** |
|
|
|
|
|
```bash |
|
|
chmod +x ./scripts/4D_from_video.sh |
|
|
./scripts/4D_from_video.sh ./examples/tiger.mp4 |
|
|
``` |
|
|
|
|
|
## Citation |
|
|
|
|
|
If you find this work useful, please cite: |
|
|
|
|
|
```bibtex |
|
|
@article{chen2026motion3to4, |
|
|
title={Motion 3-to-4: 3D Motion Reconstruction for 4D Synthesis}, |
|
|
author={Hongyuan, Chen and Xingyu, Chen and Youjia Zhang, and Zexiang, Xu and Anpei, Chen}, |
|
|
journal={arXiv preprint arXiv:2601.14253}, |
|
|
year={2026} |
|
|
} |
|
|
``` |
