| --- |
| license: mit |
| pipeline_tag: keypoint-detection |
| tags: |
| - arxiv:2604.28130 |
| - motion-capture |
| - keypoint-detection |
| - animation |
| - pose-estimation |
| - bvh |
| - retargeting |
| - animals |
| library_name: pytorch |
| --- |
| |
| # MoCapAnything V2 — Pretrained Weights |
|
|
| **End-to-End Motion Capture for Arbitrary Skeletons from Monocular Videos.** |
|
|
| This repo hosts the released end-to-end `video2pose2rot` checkpoint: a single network that maps a monocular video (plus a one-frame reference pose of the target species) directly to BVH-ready joint rotations — no analytical IK in the loop. |
|
|
| - 📄 Paper: [MoCapAnything V2 on HF Papers](https://huggingface.co/papers/2604.28130) · [arXiv:2604.28130](https://arxiv.org/abs/2604.28130) · Project page: [animotionlab.github.io/MoCapAnythingV2](https://animotionlab.github.io/MoCapAnythingV2/) |
| - 💻 Code: [github.com/phongdaot/MocapAnything](https://github.com/phongdaot/MocapAnything) |
| - 🗂 Demo data: [kehong/MoCapAnythingV2-data-sample](https://huggingface.co/datasets/kehong/MoCapAnythingV2-data-sample) |
|
|
| ## Files |
|
|
| | File | Description | |
| | --- | --- | |
| | `video2pos2rot_epoch60.pt` | End-to-end video → pose → rotation model (≈464 MB) | |
|
|
| The background remover (`briaai/RMBG-1.4`) and the video encoder (`facebook/dinov2-large`) are pulled automatically from their own HuggingFace repos at runtime — they are not re-hosted here. |
|
|
| ## Usage |
|
|
| ```bash |
| git clone https://github.com/phongdaot/MocapAnything.git && cd MocapAnything |
| pip install torch torchvision numpy opencv-python pillow matplotlib scipy scikit-image \ |
| trimesh roma pyyaml tqdm huggingface_hub transformers gradio imageio-ffmpeg |
| |
| hf download kehong/MoCapAnythingV2-weights --local-dir ./checkpoints |
| hf download kehong/MoCapAnythingV2-data-sample --repo-type dataset --local-dir ./demo/data |
| |
| # command-line inference on the bundled examples |
| python inference/video2pose2rot.py --config demo/configs/demo_wild.yaml |
| # or the interactive Gradio demo |
| python demo/app.py |
| ``` |
|
|
| See the repo's [RUN.md](https://github.com/phongdaot/MocapAnything/blob/main/RUN.md) for the Blender render setup and full options. |
|
|
| ## Citation |
|
|
| ```bibtex |
| @article{gong2026mocapanythingv2, |
| title = {MoCapAnything V2: End-to-End Motion Capture for Arbitrary Skeletons}, |
| author = {Gong, Kehong and Wen, Zhengyu and Phong, Dao Thien and |
| Xu, Mingxi and He, Weixia and Wang, Qi and Zhang, Ning and |
| Li, Zhengyu and Hou, Guanli and Lian, Dongze and He, Xiaoyu and |
| Zhang, Mingyuan and Zhang, Hanwang}, |
| journal = {arXiv preprint arXiv:2604.28130}, |
| year = {2026} |
| } |
| ``` |
|
|
