metadata
license: mit
pipeline_tag: keypoint-detection
tags:
- arxiv:2604.28130
- motion-capture
- keypoint-detection
- animation
- pose-estimation
- bvh
- retargeting
- animals
library_name: pytorch
MoCapAnything V2 — Pretrained Weights
End-to-End Motion Capture for Arbitrary Skeletons from Monocular Videos.
This repo hosts the released end-to-end video2pose2rot checkpoint: a single network that maps a monocular video (plus a one-frame reference pose of the target species) directly to BVH-ready joint rotations — no analytical IK in the loop.
- 📄 Paper: MoCapAnything V2 on HF Papers · arXiv:2604.28130 · Project page: animotionlab.github.io/MoCapAnythingV2
- 💻 Code: github.com/phongdaot/MocapAnything
- 🗂 Demo data: kehong/MoCapAnythingV2-data-sample
Files
| File | Description |
|---|---|
video2pos2rot_epoch60.pt |
End-to-end video → pose → rotation model (≈464 MB) |
The background remover (briaai/RMBG-1.4) and the video encoder (facebook/dinov2-large) are pulled automatically from their own HuggingFace repos at runtime — they are not re-hosted here.
Usage
git clone https://github.com/phongdaot/MocapAnything.git && cd MocapAnything
pip install torch torchvision numpy opencv-python pillow matplotlib scipy scikit-image \
trimesh roma pyyaml tqdm huggingface_hub transformers gradio imageio-ffmpeg
hf download kehong/MoCapAnythingV2-weights --local-dir ./checkpoints
hf download kehong/MoCapAnythingV2-data-sample --repo-type dataset --local-dir ./demo/data
# command-line inference on the bundled examples
python inference/video2pose2rot.py --config demo/configs/demo_wild.yaml
# or the interactive Gradio demo
python demo/app.py
See the repo's RUN.md for the Blender render setup and full options.
Citation
@article{gong2026mocapanythingv2,
title = {MoCapAnything V2: End-to-End Motion Capture for Arbitrary Skeletons},
author = {Gong, Kehong and Wen, Zhengyu and Phong, Dao Thien and
Xu, Mingxi and He, Weixia and Wang, Qi and Zhang, Ning and
Li, Zhengyu and Hou, Guanli and Lian, Dongze and He, Xiaoyu and
Zhang, Mingyuan and Zhang, Hanwang},
journal = {arXiv preprint arXiv:2604.28130},
year = {2026}
}