README.md

---
license: mit
pipeline_tag: other
---

# VideoMDM: Towards 3D Human Motion Generation From 2D Supervision

VideoMDM is a diffusion-based framework that trains 3D human motion priors directly from 2D poses extracted from monocular videos, without requiring any 3D ground truth. This approach learns a coherent 3D motion manifold during training and produces high-quality motion generation across various datasets.

- **Paper:** [VideoMDM: Towards 3D Human Motion Generation From 2D Supervision](https://huggingface.co/papers/2606.13364)
- **Project Page:** [https://videomdm.github.io/](https://videomdm.github.io/)
- **Repository:** [https://github.com/Amir-Mann/VideoMDM_release](https://github.com/Amir-Mann/VideoMDM_release)

## Pretrained Checkpoints

The following checkpoints are available in this repository:

| Dataset | Lifter / teacher | Folder |
|---|---|---|
| HumanML3D | MVLift   | `HUMANML3D_VIDEOMDM_ON_MVLIFT/` |
| Fit3D     | WHAM     | `FIT3D_VIDEOMDM_ON_WHAM/`       |
| NBA       | ElePose  | `NBA_VIDEOMDM_ON_ELEPOSE/`      |

## Usage

This repository follows the structure and conventions of the [MDM (Human Motion Diffusion Model)](https://github.com/GuyTevet/motion-diffusion-model) repository.

### Download Checkpoints
You can download the checkpoints using the `huggingface_hub` CLI:

```bash
pip install -U huggingface_hub
hf download AmirMann/VideoMDM --local-dir ./save
```

### Generation
To generate motion using a downloaded checkpoint, run the following command (pointing to the specific `.pt` file). The matching `args.json` in the same folder will be loaded automatically:

```bash
python -m sample.generate --model_path ./save/HUMANML3D_VIDEOMDM_ON_MVLIFT/model000600091.pt
```

## Citation

```bibtex
@article{mann2024videomdm,
  title={VideoMDM: Towards 3D Human Motion Generation From 2D Supervision},
  author={Mann, Amir and Harari, Gal Michael and Keidar, Merav and Litany, Or},
  journal={arXiv preprint arXiv:2606.13364},
  year={2024}
}
```