RSN + RTMPose-X

Code Project page Interactive demo Paper

RSN + RTMPose-X is a top-down, multi-person 2D pose model. It combines RTMPose-X with residual-step refiners, spatial attention and proposal-conditioned confidence. It predicts the 17 COCO joints from each detected person crop. The original RSN was introduced at ECCV 2020 and won the 2019 COCO Keypoint Challenge.

Checkpoint

File Input per person COCO val2017 AP
rsn-rtmpose-x.pth 384 Γ— 288 79.124422

The released file is the epoch-six confidence-calibration checkpoint. Its SHA-256 is 942d0be0bf9779c2bbef88d2a91cccb6ba6e122ec5bd7c12b81ff26ce958de04. It contains the complete model and is loadable with torch.load(..., weights_only=True).

Use

Download the code and install the demo requirements. The default pipeline downloads this checkpoint and uses YOLOX-S to find people.

git clone https://github.com/caiyuanhao1998/RSN.git
cd RSN
python -m pip install -r projects/rsn_rtmpose/requirements-demo.txt
python projects/rsn_rtmpose/pipeline.py input.mp4 output.mp4

You can also pass a JPG or PNG and write an annotated image. The interactive demo accepts both media types. Run the demo

Evaluation

Model Native COCO val2017 AP Pose latency per person
Official RTMPose-X 78.818237 3.338 ms
RSN + RTMPose-X 79.124422 5.401 ms

AP uses all 5,000 COCO val2017 images, H56 person proposals, flip inference and OKS NMS. Latency uses the same RTX A5000 and paired FP16 CUDA Graph runs at batch 1, with fused Conv/BN and one 384 Γ— 288 forward per person. The interactive pipeline uses YOLOX-S for person detection, so its end-to-end time and predictions depend on the frame and number of people.

Training

The release recipe in projects/rsn_rtmpose/train.py runs spatial refinement, attention and confidence stages sequentially with original COCO train2017 and MPII annotations. The training guide provides the inputs, recorded stage boundaries and the scope of reproducibility. Training guide

Citation

@inproceedings{cai2020learning,
  title={Learning Delicate Local Representations for Multi-Person Pose Estimation},
  author={Cai, Yuanhao and Wang, Zhicheng and Luo, Zhengxiong and Yin, Binyi and Du, Angang and Wang, Haoqian and Zhang, Xiangyu and Zhou, Xinyu and Zhou, Erjin and Sun, Jian},
  booktitle={European Conference on Computer Vision},
  year={2020}
}
Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Space using CaiYuanhao/RSN-RTMPose-X 1

Paper for CaiYuanhao/RSN-RTMPose-X