Instructions to use 404-Gen/sam3 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use 404-Gen/sam3 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("mask-generation", model="404-Gen/sam3")# Load model directly from transformers import AutoProcessor, AutoModel processor = AutoProcessor.from_pretrained("404-Gen/sam3") model = AutoModel.from_pretrained("404-Gen/sam3") - Notebooks
- Google Colab
- Kaggle
Simple helper to visualize masks (#3)
Browse files- Simple helper to visualize masks (33ef473408dc097b28115dae73221fd417c8c0ac)
- Remove comment (aea8749833b53bd864a29d3908f31ec9115f1dde)
Co-authored-by: Pedro Cuenca <pcuenq@users.noreply.huggingface.co>
README.md
CHANGED
|
@@ -124,6 +124,38 @@ SAM3 performs Promptable Concept Segmentation (PCS) on images, taking text and/o
|
|
| 124 |
>>> # - scores: Confidence scores
|
| 125 |
```
|
| 126 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 127 |
#### Single Bounding Box Prompt
|
| 128 |
|
| 129 |
Segment objects using a bounding box:
|
|
@@ -187,6 +219,7 @@ Use multiple boxes with positive and negative labels to refine the concept:
|
|
| 187 |
... mask_threshold=0.5,
|
| 188 |
... target_sizes=inputs.get("original_sizes").tolist()
|
| 189 |
... )[0]
|
|
|
|
| 190 |
```
|
| 191 |
|
| 192 |
#### Combined Prompts (Text + Negative Box)
|
|
|
|
| 124 |
>>> # - scores: Confidence scores
|
| 125 |
```
|
| 126 |
|
| 127 |
+
You can display masks using a simple helper like the following:
|
| 128 |
+
|
| 129 |
+
```python
|
| 130 |
+
import numpy as np
|
| 131 |
+
import matplotlib
|
| 132 |
+
|
| 133 |
+
def overlay_masks(image, masks):
|
| 134 |
+
image = image.convert("RGBA")
|
| 135 |
+
masks = 255 * masks.cpu().numpy().astype(np.uint8)
|
| 136 |
+
|
| 137 |
+
n_masks = masks.shape[0]
|
| 138 |
+
cmap = matplotlib.colormaps.get_cmap("rainbow").resampled(n_masks)
|
| 139 |
+
colors = [
|
| 140 |
+
tuple(int(c * 255) for c in cmap(i)[:3])
|
| 141 |
+
for i in range(n_masks)
|
| 142 |
+
]
|
| 143 |
+
|
| 144 |
+
for mask, color in zip(masks, colors):
|
| 145 |
+
mask = Image.fromarray(mask)
|
| 146 |
+
overlay = Image.new("RGBA", image.size, color + (0,))
|
| 147 |
+
alpha = mask.point(lambda v: int(v * 0.5))
|
| 148 |
+
overlay.putalpha(alpha)
|
| 149 |
+
image = Image.alpha_composite(image, overlay)
|
| 150 |
+
return image
|
| 151 |
+
```
|
| 152 |
+
|
| 153 |
+
Then you can save the resulting composite image or display it in a notebook:
|
| 154 |
+
|
| 155 |
+
```python
|
| 156 |
+
>>> overlay_masks(image, results["masks"])
|
| 157 |
+
```
|
| 158 |
+
|
| 159 |
#### Single Bounding Box Prompt
|
| 160 |
|
| 161 |
Segment objects using a bounding box:
|
|
|
|
| 219 |
... mask_threshold=0.5,
|
| 220 |
... target_sizes=inputs.get("original_sizes").tolist()
|
| 221 |
... )[0]
|
| 222 |
+
>>> overlay_masks(kitchen_image, results["masks"])
|
| 223 |
```
|
| 224 |
|
| 225 |
#### Combined Prompts (Text + Negative Box)
|