--- library_name: pytorch license: other tags: - foundation - amd - rocm - image-segmentation pipeline_tag: mask-generation --- ![](https://huggingface.co/AMD-PAVS-AI/sam3/resolve/main/sam3.png) # SAM3: Optimized for AMD ROCm SAM3 (Segment Anything Model 3) is a promptable instance segmentation model that generates high-quality masks from text, box, or point prompts. This repository packages inference for instance segmentation using **PyTorch**, exported and validated for **AMD ROCm** so it runs efficiently on AMD GPUs and CPUs. This is based on the implementation of SAM3 found [here](https://huggingface.co/facebook/sam3). This repository contains configurations and scripts optimized for **AMD® ROCm™** platforms. You can use the [sam3 AMD scripts](https://github.com/AMD-PAVS/physical_ai_sdk/blob/main/models/sam3) to reproduce results or export with custom configurations. More details on model performance can be found [here](#accuracy-pipeline). --- ## Task Overview **Task:** Instance segmentation **Dataset:** COCO val2017 (5,000 images, 80 categories) **Output metrics:** Mask mAP@0.5:0.95, Mask mAP@0.5, Mask mAP@0.75, Mean Mask IoU, Mask Precision, Mask Recall, Mask F1 > **PyTorch note:** CPU runs FP32 only; GPU supports FP32/FP16/BF16 via `torch.autocast` (no INT8 — SAM3 is PyTorch-only, not ONNX/MIGraphX). NPU is not available since ONNX export is unsupported upstream. Hugging Face access is required — SAM3 weights are gated; you must accept the license at [facebook/sam3](https://huggingface.co/facebook/sam3) and export `HF_TOKEN`. --- ## AMD ROCm Optimization This model export has been adapted and validated for **AMD Instinct™ / Radeon™ GPUs** running **ROCm**, as well as AMD CPUs. Key points: - Validated backend: **PyTorch** (native ROCm HIP kernels via `torch.autocast`), FP32/FP16/BF16 on GPU, FP32 on CPU. - No code changes required versus the upstream SAM3 implementation — only environment/runtime configuration differs. - No NPU path — SAM3 ONNX export is unsupported upstream, and VitisAI requires an ONNX graph. | Runtime | Precision | Backend | Hardware | Notes | |---|---|---|---|---| | PyTorch | FP32 | HIP (ROCm) | AMD CPU | — | | PyTorch | FP32 / FP16 / BF16 | HIP (ROCm) | AMD Instinct™ / Radeon™ GPU | Via `torch.autocast`; no INT8 path | --- ## Getting Started For setup instructions, evaluation scripts, and custom configuration options, see the [sam3 on GitHub](https://github.com/AMD-PAVS/physical_ai_sdk/blob/main/models/sam3). --- ## Model Details **Model Type:** Promptable instance segmentation (text, box, or point prompts) **Base Model:** `facebook/sam3` (gated) **Model Stats:** - Precision tested: FP32 (CPU); FP32, FP16, BF16 (GPU) --- ## Accuracy Pipeline Higher Mask mAP means the model's predicted masks agree more closely with the ground-truth segmentation annotations — 1.0 would be perfect mask overlap, 0.0 means no correct masks. Because SAM3 is a promptable segmentation model (not a detector), these metrics measure pure mask quality independent of detection accuracy. Since full-dataset inference is too slow for a complete 5,000-image COCO val2017 pass, evaluation uses three nested random subsets (N=100, 300, 500) drawn once with a fixed seed for reproducibility. ### Metrics Explained | Metric | Description | |--------|-------------| | Mask mAP@0.5:0.95 | Primary COCO mask metric — mean AP averaged across IoU thresholds 0.5–0.95. The strictest, most holistic mask accuracy number; higher means predicted masks overlap ground truth tightly across a range of thresholds. | | Mask mAP@0.5 | AP at a single, looser IoU threshold of 0.5 — a predicted mask only needs 50% overlap to count as correct, so this is typically higher than mAP@0.5:0.95 and reflects "did it segment the object" more than "how precisely." | | Mask mAP@0.75 | AP at a stricter IoU threshold of 0.75 — the predicted mask must overlap ground truth by 75%, rewarding precise segmentation boundaries over merely finding the object. | | Mean Mask IoU | Average intersection-over-union across all predicted masks — a direct geometric measure of how well each individual mask covers its ground-truth region, complementing the AP-based aggregate metrics. | | Mask Precision | Of all masks the model produced, what fraction were correct — high precision means few false-positive masks (segments that don't correspond to real objects). | | Mask Recall | Of all ground-truth objects, what fraction did the model successfully segment — high recall means few missed objects (false negatives). | | Mask F1 | Harmonic mean of Mask Precision and Mask Recall — balances the two failure modes (false positives vs. false negatives) into a single number. | ### Accuracy Results **Sampled Evaluation (COCO val2017)** — nested subsets, seed=42: | Device | Precision | N | Mask mAP@0.5:0.95 | Mask mAP@0.5 | Mean IoU | Mask Precision | Mask Recall | Mask F1 | |--------|-----------|---|-------------------|-------------|----------|----------------|-------------|---------| | CPU | FP32 | 100 | 0.6269 | 0.8982 | 0.7933 | 0.5895 | 0.6166 | 0.6027 | | CPU | FP32 | 300 | 0.5991 | 0.8937 | 0.7878 | 0.5525 | 0.6026 | 0.5765 | | CPU | FP32 | 500 | 0.5813 | 0.9045 | 0.7886 | 0.5390 | 0.5938 | 0.5651 | | GPU | FP32 | 100 | 0.6269 | 0.8982 | 0.7933 | 0.5895 | 0.6166 | 0.6027 | | GPU | FP32 | 300 | 0.5991 | 0.8937 | 0.7878 | 0.5525 | 0.6026 | 0.5765 | | GPU | FP32 | 500 | 0.5813 | 0.9045 | 0.7886 | 0.5390 | 0.5938 | 0.5651 | --- ## Dig Deeper Want to explore the full evaluation scripts, config options, and other AMD-optimized model examples? 📂 **[View the full project on GitHub](https://github.com/AMD-PAVS/physical_ai_sdk/blob/main/models/sam3)** The GitHub repository includes: - Setup and prerequisites for ROCm environments - Text/box/point prompt-based inference scripts - COCO val2017 sampled evaluation pipeline with fixed-seed reproducibility - Benchmarking and reproduction instructions for CPU and GPU