| --- |
| pipeline_tag: other |
| tags: |
| - 3d |
| - human-motion-generation |
| - diffusion |
| --- |
| |
| # DrawMotion: Generating 3D Human Motions by Freehand Drawing |
|
|
| DrawMotion is an efficient diffusion-based framework designed for multi-condition scenarios, generating 3D human motions based on both conventional text conditions and a novel hand-drawing condition. This approach provides both semantic and spatial control over the generated motions. |
|
|
| - **Paper:** [DrawMotion: Generating 3D Human Motions by Freehand Drawing](https://huggingface.co/papers/2605.20955) |
| - **Repository:** [InvertedForest/DrawMotion](https://github.com/InvertedForest/DrawMotion) |
| - **Demo Video:** [YouTube](https://youtu.be/sy2QTdDD09A) |
|
|
| ## Overview |
|
|
| Text-to-motion generation often faces challenges in precisely conveying intended motions through text alone. DrawMotion addresses this by introducing a multi-condition fusion module (MCM) that integrates text prompts with freehand stickman sketches and 2D trajectories. It also utilizes a training-free guidance strategy to align generated motions with user intentions while preserving fidelity. |
|
|
| ## Sample Usage |
|
|
| ### Prepare Weights |
|
|
| You can download the required model assets from this repository using the `huggingface_hub` CLI: |
|
|
| ```bash |
| pip install -U huggingface_hub |
| hf download I0u0I/DrawMotion \ |
| --local-dir . \ |
| --include "logs/human_ml3d/last.ckpt" \ |
| "mid_feat/t2m/mid_feat.pt" \ |
| "stickman/weight/real_init/t2m/stickman_encoder.ckpt" |
| ``` |
|
|
| The model also requires the OpenAI CLIP text encoder: |
| ```python |
| import clip |
| # Download the OpenAI CLIP text encoder used by DrawMotion. |
| python -c 'import clip; clip.load("ViT-B/32", device="cpu")' |
| ``` |
|
|
| ### Running the Web Demo |
|
|
| To run the interactive DrawMotion experience locally, use the following command from the repository root: |
|
|
| ```bash |
| DRAWMOTION_CKPT=logs/human_ml3d/last.ckpt \ |
| DRAWMOTION_GPU=0 \ |
| uvicorn demo.drawmotion_studio.app:app --host 0.0.0.0 --port 12008 |
| ``` |
|
|
| ## Citation |
|
|
| If you find DrawMotion useful, please cite: |
|
|
| ```bibtex |
| @article{wang2026drawmotion, |
| title={DrawMotion: Generating 3D Human Motions by Freehand Drawing}, |
| author={Wang, Tao and Jin, Lei and Wu, Zhihua and He, Qiaozhi and Chu, Jiaming and Cheng, Yu and Xing, Junliang and Zhao, Jian and Yan, Shuicheng and Wang, Li}, |
| journal={IEEE Transactions on Pattern Analysis and Machine Intelligence}, |
| year={2026}, |
| pages={1--17}, |
| doi={10.1109/TPAMI.2026.3679530} |
| } |
| ``` |
