---
license: mit
tags:
  - motion-capture
  - pose-estimation
  - inverse-kinematics
  - animation
  - arbitrary-skeleton
library_name: pytorch
---

# pose2rot — Joint Positions → 6D Rotations for Arbitrary Skeletons

Pretrained checkpoints for the **pose2rot** model (`Pose2RotMemoryRestModel`): given a sequence of
3D joint **positions**, predict per-joint **6D rotations** (forward kinematics then recovers the full
skeletal animation). One model handles **arbitrary skeletons** across 72 animal species — quadrupeds,
bipeds, birds, reptiles, dinosaurs, arthropods, limbless snakes — via T5 joint-name embeddings,
skeleton graph attention, and rest-pose FiLM conditioning.

**Code, training recipe, eval & QA scripts:** https://github.com/CHDTevior/pose2rot

This is a derivative work of [MocapAnything](https://github.com/phongdaot/MocapAnything)
(MIT, © 2026 Dao Thien Phong; arXiv:2604.28130 MoCapAnything V2). ~29.7M params.

## Checkpoints

| file | training data | use case |
|---|---|---|
| `pose2rot_v9_alldata_epoch60.pt` | all 72 species | **best for demos / inference** (the species is seen) |
| `pose2rot_v10_heldout_epoch60.pt` | seen/rare/unseen held-out split (test motions excluded) | the **decisive paper model** for honest cross-topology eval |
| `pose2rot_v8b_best_epoch40.pt` | all species (earlier converged best) | reference |

Each `.pt` holds `{model_state, optimizer_state, epoch}`. Configs: `config_v9_alldata.yaml`,
`config_v10_split_heldout.yaml` (model section instantiates `Pose2RotMemoryRestModel`).

## Results (geodesic angle error, degrees; MoCapAnything V2 reports 6.54° unseen / V1 ~17°)

| model | seen | rare | unseen | overall |
|---|---|---|---|---|
| v9 all-data (oracle) | 7.2° | 5.9° | 6.4° | **6.53°** ≈ MoCapAnything 6.54° |
| v10 true held-out | 9.8° | 12.7° | 40.9° | 28.0° |

When the species is **seen**, the model matches SOTA (6.5°). On a **true held-out** test, cross-topology
generalization is a ceiling: unseen species with close training relatives generalize partially (Goat 17°,
Coyote 19°), topologically distinctive ones do not (Pigeon ~67°, Spider ~73°).

## Usage

```python
import torch
from utils.config_utils import load_yaml_config, instantiate_from_config  # from the GitHub repo

cfg = load_yaml_config("config_v9_alldata.yaml")
model = instantiate_from_config(cfg["model"]).eval().cuda()
model.load_state_dict(torch.load("pose2rot_v9_alldata_epoch60.pt", map_location="cpu")["model_state"])
# batch dict: position[B,T,J,3] + rest pose + T5 joint embeddings + skeleton graph + reference (see GitHub data/loader_v2.py)
pred_rot6d = model(batch)["pred_rot6d"]  # [B,T,J,6]
```

See https://github.com/CHDTevior/pose2rot for the full data pipeline, training, and evaluation.

## License & Citation

MIT. Built on [MocapAnything](https://github.com/phongdaot/MocapAnything) (Dao Thien Phong, MIT).
Please also cite MoCapAnything (arXiv:2604.28130).