RSN + RTMPose-X
RSN + RTMPose-X is a top-down, multi-person 2D pose model. It combines RTMPose-X with residual-step refiners, spatial attention and proposal-conditioned confidence. It predicts the 17 COCO joints from each detected person crop. The original RSN was introduced at ECCV 2020 and won the 2019 COCO Keypoint Challenge.
Checkpoint
| File | Input per person | COCO val2017 AP |
|---|---|---|
| rsn-rtmpose-x.pth | 384 Γ 288 | 79.124422 |
The released file is the epoch-six confidence-calibration checkpoint. Its SHA-256 is 942d0be0bf9779c2bbef88d2a91cccb6ba6e122ec5bd7c12b81ff26ce958de04. It contains the complete model and is loadable with torch.load(..., weights_only=True).
Use
Download the code and install the demo requirements. The default pipeline downloads this checkpoint and uses YOLOX-S to find people.
git clone https://github.com/caiyuanhao1998/RSN.git
cd RSN
python -m pip install -r projects/rsn_rtmpose/requirements-demo.txt
python projects/rsn_rtmpose/pipeline.py input.mp4 output.mp4
You can also pass a JPG or PNG and write an annotated image. The interactive demo accepts both media types. Run the demo
Evaluation
| Model | Native COCO val2017 AP | Pose latency per person |
|---|---|---|
| Official RTMPose-X | 78.818237 | 3.338 ms |
| RSN + RTMPose-X | 79.124422 | 5.401 ms |
AP uses all 5,000 COCO val2017 images, H56 person proposals, flip inference and OKS NMS. Latency uses the same RTX A5000 and paired FP16 CUDA Graph runs at batch 1, with fused Conv/BN and one 384 Γ 288 forward per person. The interactive pipeline uses YOLOX-S for person detection, so its end-to-end time and predictions depend on the frame and number of people.
Training
The release recipe in projects/rsn_rtmpose/train.py runs spatial refinement, attention and confidence stages sequentially with original COCO train2017 and MPII annotations. The training guide provides the inputs, recorded stage boundaries and the scope of reproducibility. Training guide
Citation
@inproceedings{cai2020learning,
title={Learning Delicate Local Representations for Multi-Person Pose Estimation},
author={Cai, Yuanhao and Wang, Zhicheng and Luo, Zhengxiong and Yin, Binyi and Du, Angang and Wang, Haoqian and Zhang, Xiangyu and Zhou, Xinyu and Zhou, Erjin and Sun, Jian},
booktitle={European Conference on Computer Vision},
year={2020}
}