nielsr HF Staff commited on
Commit
6440f49
·
verified ·
1 Parent(s): 6d8dc8d

Add model card for VideoMDM

Browse files

Hi! I'm Niels from the Hugging Face community team. I noticed this repository was missing a model card, which is important for discoverability and usability.

This PR adds a comprehensive model card that:
- Includes metadata for the license and pipeline tag.
- Links the repository to the [original paper](https://huggingface.co/papers/2606.13364) and project page.
- Provides a guide for downloading the checkpoints and running the generation code as documented in the official repository.

Files changed (1) hide show
  1. README.md +52 -0
README.md ADDED
@@ -0,0 +1,52 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: mit
3
+ pipeline_tag: other
4
+ ---
5
+
6
+ # VideoMDM: Towards 3D Human Motion Generation From 2D Supervision
7
+
8
+ VideoMDM is a diffusion-based framework that trains 3D human motion priors directly from 2D poses extracted from monocular videos, without requiring any 3D ground truth. This approach learns a coherent 3D motion manifold during training and produces high-quality motion generation across various datasets.
9
+
10
+ - **Paper:** [VideoMDM: Towards 3D Human Motion Generation From 2D Supervision](https://huggingface.co/papers/2606.13364)
11
+ - **Project Page:** [https://videomdm.github.io/](https://videomdm.github.io/)
12
+ - **Repository:** [https://github.com/Amir-Mann/VideoMDM_release](https://github.com/Amir-Mann/VideoMDM_release)
13
+
14
+ ## Pretrained Checkpoints
15
+
16
+ The following checkpoints are available in this repository:
17
+
18
+ | Dataset | Lifter / teacher | Folder |
19
+ |---|---|---|
20
+ | HumanML3D | MVLift | `HUMANML3D_VIDEOMDM_ON_MVLIFT/` |
21
+ | Fit3D | WHAM | `FIT3D_VIDEOMDM_ON_WHAM/` |
22
+ | NBA | ElePose | `NBA_VIDEOMDM_ON_ELEPOSE/` |
23
+
24
+ ## Usage
25
+
26
+ This repository follows the structure and conventions of the [MDM (Human Motion Diffusion Model)](https://github.com/GuyTevet/motion-diffusion-model) repository.
27
+
28
+ ### Download Checkpoints
29
+ You can download the checkpoints using the `huggingface_hub` CLI:
30
+
31
+ ```bash
32
+ pip install -U huggingface_hub
33
+ hf download AmirMann/VideoMDM --local-dir ./save
34
+ ```
35
+
36
+ ### Generation
37
+ To generate motion using a downloaded checkpoint, run the following command (pointing to the specific `.pt` file). The matching `args.json` in the same folder will be loaded automatically:
38
+
39
+ ```bash
40
+ python -m sample.generate --model_path ./save/HUMANML3D_VIDEOMDM_ON_MVLIFT/model000600091.pt
41
+ ```
42
+
43
+ ## Citation
44
+
45
+ ```bibtex
46
+ @article{mann2024videomdm,
47
+ title={VideoMDM: Towards 3D Human Motion Generation From 2D Supervision},
48
+ author={Mann, Amir and Harari, Gal Michael and Keidar, Merav and Litany, Or},
49
+ journal={arXiv preprint arXiv:2606.13364},
50
+ year={2024}
51
+ }
52
+ ```