Add model card for DrawMotion
Browse filesHi, I'm Niels from the community science team at Hugging Face. This PR adds a model card for DrawMotion, which includes:
- Metadata for the pipeline tag and relevant search tags.
- Links to the paper, GitHub repository, and demo video.
- A brief overview of the project.
- Instructions on how to download the model weights using the `huggingface_hub` CLI.
- The BibTeX citation for the paper.
README.md
ADDED
|
@@ -0,0 +1,66 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
pipeline_tag: other
|
| 3 |
+
tags:
|
| 4 |
+
- 3d
|
| 5 |
+
- human-motion-generation
|
| 6 |
+
- diffusion
|
| 7 |
+
---
|
| 8 |
+
|
| 9 |
+
# DrawMotion: Generating 3D Human Motions by Freehand Drawing
|
| 10 |
+
|
| 11 |
+
DrawMotion is an efficient diffusion-based framework designed for multi-condition scenarios, generating 3D human motions based on both conventional text conditions and a novel hand-drawing condition. This approach provides both semantic and spatial control over the generated motions.
|
| 12 |
+
|
| 13 |
+
- **Paper:** [DrawMotion: Generating 3D Human Motions by Freehand Drawing](https://huggingface.co/papers/2605.20955)
|
| 14 |
+
- **Repository:** [InvertedForest/DrawMotion](https://github.com/InvertedForest/DrawMotion)
|
| 15 |
+
- **Demo Video:** [YouTube](https://youtu.be/sy2QTdDD09A)
|
| 16 |
+
|
| 17 |
+
## Overview
|
| 18 |
+
|
| 19 |
+
Text-to-motion generation often faces challenges in precisely conveying intended motions through text alone. DrawMotion addresses this by introducing a multi-condition fusion module (MCM) that integrates text prompts with freehand stickman sketches and 2D trajectories. It also utilizes a training-free guidance strategy to align generated motions with user intentions while preserving fidelity.
|
| 20 |
+
|
| 21 |
+
## Sample Usage
|
| 22 |
+
|
| 23 |
+
### Prepare Weights
|
| 24 |
+
|
| 25 |
+
You can download the required model assets from this repository using the `huggingface_hub` CLI:
|
| 26 |
+
|
| 27 |
+
```bash
|
| 28 |
+
pip install -U huggingface_hub
|
| 29 |
+
hf download I0u0I/DrawMotion \
|
| 30 |
+
--local-dir . \
|
| 31 |
+
--include "logs/human_ml3d/last.ckpt" \
|
| 32 |
+
"mid_feat/t2m/mid_feat.pt" \
|
| 33 |
+
"stickman/weight/real_init/t2m/stickman_encoder.ckpt"
|
| 34 |
+
```
|
| 35 |
+
|
| 36 |
+
The model also requires the OpenAI CLIP text encoder:
|
| 37 |
+
```python
|
| 38 |
+
import clip
|
| 39 |
+
# Download the OpenAI CLIP text encoder used by DrawMotion.
|
| 40 |
+
python -c 'import clip; clip.load("ViT-B/32", device="cpu")'
|
| 41 |
+
```
|
| 42 |
+
|
| 43 |
+
### Running the Web Demo
|
| 44 |
+
|
| 45 |
+
To run the interactive DrawMotion experience locally, use the following command from the repository root:
|
| 46 |
+
|
| 47 |
+
```bash
|
| 48 |
+
DRAWMOTION_CKPT=logs/human_ml3d/last.ckpt \
|
| 49 |
+
DRAWMOTION_GPU=0 \
|
| 50 |
+
uvicorn demo.drawmotion_studio.app:app --host 0.0.0.0 --port 12008
|
| 51 |
+
```
|
| 52 |
+
|
| 53 |
+
## Citation
|
| 54 |
+
|
| 55 |
+
If you find DrawMotion useful, please cite:
|
| 56 |
+
|
| 57 |
+
```bibtex
|
| 58 |
+
@article{wang2026drawmotion,
|
| 59 |
+
title={DrawMotion: Generating 3D Human Motions by Freehand Drawing},
|
| 60 |
+
author={Wang, Tao and Jin, Lei and Wu, Zhihua and He, Qiaozhi and Chu, Jiaming and Cheng, Yu and Xing, Junliang and Zhao, Jian and Yan, Shuicheng and Wang, Li},
|
| 61 |
+
journal={IEEE Transactions on Pattern Analysis and Machine Intelligence},
|
| 62 |
+
year={2026},
|
| 63 |
+
pages={1--17},
|
| 64 |
+
doi={10.1109/TPAMI.2026.3679530}
|
| 65 |
+
}
|
| 66 |
+
```
|