pipeline_tag: other
tags:
- 3d
- human-motion-generation
- diffusion
DrawMotion: Generating 3D Human Motions by Freehand Drawing
DrawMotion is an efficient diffusion-based framework designed for multi-condition scenarios, generating 3D human motions based on both conventional text conditions and a novel hand-drawing condition. This approach provides both semantic and spatial control over the generated motions.
- Paper: DrawMotion: Generating 3D Human Motions by Freehand Drawing
- Repository: InvertedForest/DrawMotion
- Demo Video: YouTube
Overview
Text-to-motion generation often faces challenges in precisely conveying intended motions through text alone. DrawMotion addresses this by introducing a multi-condition fusion module (MCM) that integrates text prompts with freehand stickman sketches and 2D trajectories. It also utilizes a training-free guidance strategy to align generated motions with user intentions while preserving fidelity.
Sample Usage
Prepare Weights
You can download the required model assets from this repository using the huggingface_hub CLI:
pip install -U huggingface_hub
hf download I0u0I/DrawMotion \
--local-dir . \
--include "logs/human_ml3d/last.ckpt" \
"mid_feat/t2m/mid_feat.pt" \
"stickman/weight/real_init/t2m/stickman_encoder.ckpt"
The model also requires the OpenAI CLIP text encoder:
import clip
# Download the OpenAI CLIP text encoder used by DrawMotion.
python -c 'import clip; clip.load("ViT-B/32", device="cpu")'
Running the Web Demo
To run the interactive DrawMotion experience locally, use the following command from the repository root:
DRAWMOTION_CKPT=logs/human_ml3d/last.ckpt \
DRAWMOTION_GPU=0 \
uvicorn demo.drawmotion_studio.app:app --host 0.0.0.0 --port 12008
Citation
If you find DrawMotion useful, please cite:
@article{wang2026drawmotion,
title={DrawMotion: Generating 3D Human Motions by Freehand Drawing},
author={Wang, Tao and Jin, Lei and Wu, Zhihua and He, Qiaozhi and Chu, Jiaming and Cheng, Yu and Xing, Junliang and Zhao, Jian and Yan, Shuicheng and Wang, Li},
journal={IEEE Transactions on Pattern Analysis and Machine Intelligence},
year={2026},
pages={1--17},
doi={10.1109/TPAMI.2026.3679530}
}