metadata
license: mit
pipeline_tag: other
VideoMDM: Towards 3D Human Motion Generation From 2D Supervision
VideoMDM is a diffusion-based framework that trains 3D human motion priors directly from 2D poses extracted from monocular videos, without requiring any 3D ground truth. This approach learns a coherent 3D motion manifold during training and produces high-quality motion generation across various datasets.
- Paper: VideoMDM: Towards 3D Human Motion Generation From 2D Supervision
- Project Page: https://videomdm.github.io/
- Repository: https://github.com/Amir-Mann/VideoMDM_release
Pretrained Checkpoints
The following checkpoints are available in this repository:
| Dataset | Lifter / teacher | Folder |
|---|---|---|
| HumanML3D | MVLift | HUMANML3D_VIDEOMDM_ON_MVLIFT/ |
| Fit3D | WHAM | FIT3D_VIDEOMDM_ON_WHAM/ |
| NBA | ElePose | NBA_VIDEOMDM_ON_ELEPOSE/ |
Usage
This repository follows the structure and conventions of the MDM (Human Motion Diffusion Model) repository.
Download Checkpoints
You can download the checkpoints using the huggingface_hub CLI:
pip install -U huggingface_hub
hf download AmirMann/VideoMDM --local-dir ./save
Generation
To generate motion using a downloaded checkpoint, run the following command (pointing to the specific .pt file). The matching args.json in the same folder will be loaded automatically:
python -m sample.generate --model_path ./save/HUMANML3D_VIDEOMDM_ON_MVLIFT/model000600091.pt
Citation
@article{mann2024videomdm,
title={VideoMDM: Towards 3D Human Motion Generation From 2D Supervision},
author={Mann, Amir and Harari, Gal Michael and Keidar, Merav and Litany, Or},
journal={arXiv preprint arXiv:2606.13364},
year={2024}
}