unsloth/sam3.1

SAM 3.1 by Meta in the transformers Sam3Model format: give it an image and a text prompt such as "sofa" and it returns a mask, a box and a score for every match. It powers "Mask from text" in Unsloth Studio.

Prompts: sofa, red lamp, pillow, potted plant, rug

Prompts: person, bicycle, dog, yellow car

File	Size	Mask IoU vs BF16 (mean / min)
model.safetensors (BF16)	1.68 GB	1.0000 / 1.0000
sam3.1-INT8-ConvRot.safetensors	1.05 GB	0.9984 / 0.9913
sam3.1-INT8.safetensors	0.99 GB	0.9976 / 0.9829
sam3.1-FP8.safetensors	1.05 GB	0.9971 / 0.9805

53 text prompts on 6 images; 1.0 means identical masks. Unsloth Studio uses INT8-ConvRot on NVIDIA GPUs and BF16 elsewhere.

Usage

import torch
from PIL import Image
from transformers import Sam3Model, Sam3Processor

model = Sam3Model.from_pretrained("unsloth/sam3.1", dtype=torch.bfloat16).to("cuda").eval()
processor = Sam3Processor.from_pretrained("unsloth/sam3.1")

image = Image.open("living_room.png").convert("RGB")
inputs = processor(images=image, text="sofa", return_tensors="pt").to("cuda", torch.bfloat16)
with torch.inference_mode():
    outputs = model(**inputs)
results = processor.post_process_instance_segmentation(
    outputs, threshold=0.5, mask_threshold=0.5, target_sizes=inputs.get("original_sizes").tolist()
)[0]
print(results["scores"], results["boxes"], results["masks"].shape)

---

SAM 3.1

SAM 3 (Segment Anything with Concepts) is a unified foundation model from Meta for promptable segmentation in images and videos. It detects, segments, and tracks objects using text or visual prompts such as points, boxes, and masks. SAM 3 introduces the ability to exhaustively segment all instances of an open-vocabulary concept specified by a short text phrase, handling over 50x more unique concepts than existing benchmarks. SAM 3.1 builds on this with Object Multiplex, a shared-memory approach for joint multi-object tracking that delivers ~7x faster inference at 128 objects on a single H100 GPU without sacrificing accuracy, along with improved VOS performance on 6 out of 7 benchmarks.

For the video tracker, installation, code and full documentation, see the SAM 3 GitHub repository and facebook/sam3.1.

License

SAM 3.1 is released by Meta under the SAM License. This repository is a derivative of the SAM Materials and is distributed under the same agreement; the full text is in LICENSE. By using or distributing these weights you agree to the SAM License. If you publish research results that use this model, the license asks you to acknowledge the use of the SAM Materials. Not an official Meta release, and not endorsed by Meta.