Sapiens: 2D Human Pose Estimation
Model Zoo
We provide 4 models of varying size. Sapiens-0.3B, Sapiens-0.6B, Sapiens-1B, Sapiens-2B. In general, performance improves with increasing the model size.
Body: 17 Keypoints
Best for general in-the-wild scenarios with body keypoints only, adhering to the COCO keypoint format.
Please download the models from hugging-face-pose.
| Model | Checkpoint Path |
|---|---|
| Sapiens-0.3B | $SAPIENS_CHECKPOINT_ROOT/pose/checkpoints/sapiens_0.3b/sapiens_0.3b_coco_best_coco_AP_796.pth |
| Sapiens-0.6B | $SAPIENS_CHECKPOINT_ROOT/pose/checkpoints/sapiens_0.6b/sapiens_0.6b_coco_best_coco_AP_812.pth |
| Sapiens-1B | $SAPIENS_CHECKPOINT_ROOT/pose/checkpoints/sapiens_1b/sapiens_1b_coco_best_coco_AP_821.pth |
| Sapiens-2B | $SAPIENS_CHECKPOINT_ROOT/pose/checkpoints/sapiens_2b/sapiens_2b_coco_best_coco_AP_822.pth |
Body + Face + Hands + Feet: 133 Keypoints
Offers second-best generalization with body, face, hands, and feet keypoints, following the COCO-WholeBody keypoint format.
Please download the models from hugging-face-pose.
| Model | Checkpoint Path |
|---|---|
| Sapiens-0.3B | $SAPIENS_CHECKPOINT_ROOT/pose/checkpoints/sapiens_0.3b/sapiens_0.3b_coco_wholebody_best_coco_wholebody_AP_620.pth |
| Sapiens-0.6B | $SAPIENS_CHECKPOINT_ROOT/pose/checkpoints/sapiens_0.6b/sapiens_0.6b_coco_wholebody_best_coco_wholebody_AP_695.pth |
| Sapiens-1B | $SAPIENS_CHECKPOINT_ROOT/pose/checkpoints/sapiens_1b/sapiens_1b_coco_wholebody_best_coco_wholebody_AP_727.pth |
| Sapiens-2B | $SAPIENS_CHECKPOINT_ROOT/pose/checkpoints/sapiens_2b/sapiens_2b_coco_wholebody_best_coco_wholebody_AP_745.pth |
Body + Dense Face + Hands + Feet: 308 Keypoints
The highest number of keypoints predictor. Detailed 274 face keypoints. Following the Sociopticon keypoint format. Please download the models from hugging-face.
| Model | Checkpoint Path |
|---|---|
| Sapiens-0.3B | $SAPIENS_CHECKPOINT_ROOT/pose/checkpoints/sapiens_0.3b/sapiens_0.3b_goliath_best_goliath_AP_573.pth |
| Sapiens-0.6B | $SAPIENS_CHECKPOINT_ROOT/pose/checkpoints/sapiens_0.6b/sapiens_0.6b_goliath_best_goliath_AP_609.pth |
| Sapiens-1B | $SAPIENS_CHECKPOINT_ROOT/pose/checkpoints/sapiens_1b/sapiens_1b_goliath_best_goliath_AP_639.pth |
Inference Guide
Navigate to the script directory and launch the inference scripts for your desired keypoint configuration.
Standard Estimation
- Navigate to your script directory:
cd $SAPIENS_ROOT/pose/scripts/demo/local - For 17 keypoints estimation (uncomment your model config line for inference):
./keypoints17.sh - For 133 keypoints estimation (uncomment your model config line for inference):
./keypoints133.sh - For 308 keypoints estimation (uncomment your model config line for inference):
./keypoints308.sh
Define INPUT for your image directory and OUTPUT for results. Visualization and keypoints in JSON format are saved to OUTPUT. Customize LINE_THICKNESS, RADIUS, and KPT_THRES as needed. Adjust JOBS_PER_GPU, TOTAL_GPUS and VALID_GPU_IDS for multi-GPU configurations.
ControlNet Compatible Output
Generate OpenPose skeletons on a black background, compatible with ControlNet.
- Navigate to your script directory:
cd $SAPIENS_ROOT/pose/scripts/demo/local - For 17 keypoints estimation (uncomment your model config line for inference):
./keypoints17_openpose.sh - For 133 keypoints estimation (uncomment your model config line for inference):
./keypoints133_openpose.sh
Customization options as mentioned above are applicable here as well.
