gpue/foundationpose-weights

FoundationPose Model Weights

Pre-trained weights for FoundationPose 6D object pose estimation model.

Model Details

• Refiner weights: 2023-10-28-18-33-37/model_best.pth
• Scorer weights: 2024-01-11-20-02-45/model_best.pth
• Source: Official FoundationPose release
• Paper: FoundationPose: Unified 6D Pose Estimation and Tracking of Novel Objects (CVPR 2024)

Model Architecture

FoundationPose is a unified foundation model for 6D object pose estimation and tracking, supporting both:

• Model-based setup: Using CAD models
• Model-free setup: Using reference images (16-20 views)

Files

.
├── 2023-10-28-18-33-37/
│   ├── config.yml
│   └── model_best.pth (refiner model)
└── 2024-01-11-20-02-45/
    ├── config.yml
    └── model_best.pth (scorer model)

Usage

Download Weights

from huggingface_hub import snapshot_download

# Download all weights
weights_path = snapshot_download(
    repo_id="gpue/foundationpose-weights",
    local_dir="./weights"
)

Use with FoundationPose Space

This model repository is designed to work with the gpue/foundationpose Space.

Set environment variables:

FOUNDATIONPOSE_MODEL_REPO=gpue/foundationpose-weights
USE_HF_WEIGHTS=true
USE_REAL_MODEL=true

Local Usage

import torch
from pathlib import Path

# Load refiner
refiner_weights = torch.load("weights/2023-10-28-18-33-37/model_best.pth")

# Load scorer
scorer_weights = torch.load("weights/2024-01-11-20-02-45/model_best.pth")

Performance

• Accuracy: State-of-the-art on BOP benchmark (as of 2024/03)
• Speed: Real-time capable with GPU acceleration
• Generalization: Works on novel objects without fine-tuning

Citation

If you use these weights, please cite:

@inproceedings{wen2023foundationpose,
  title={FoundationPose: Unified 6D Pose Estimation and Tracking of Novel Objects},
  author={Wen, Bowen and Yang, Wei and Kautz, Jan and Birchfield, Stan},
  booktitle={CVPR},
  year={2024}
}

License

These weights are from the official FoundationPose release and are subject to NVIDIA's Source Code License.

Key restrictions:

• Non-commercial use only
• No redistribution of derivative works
• Academic and research purposes

Related Resources

• Paper: https://arxiv.org/abs/2312.08344
• Code: https://github.com/NVlabs/FoundationPose
• Project Page: https://nvlabs.github.io/FoundationPose/
• Inference Space: https://huggingface.co/spaces/gpue/foundationpose

Model Card

Developed by: NVIDIA Research (Bowen Wen, Wei Yang, Jan Kautz, Stan Birchfield)

Model type: Transformer-based 6D pose estimator

Training data: Large-scale synthetic dataset

Intended use: 6D object pose estimation and tracking for robotics and AR/VR applications

Out-of-scope: Commercial deployment (due to license restrictions)