See our collection for all versions of SAM.

Run SAM3 with Keras 3: JAX, PyTorch, or TensorFlow

kerasformers/sam3

Paper: SAM 3: Segment Anything with Concepts (arXiv:2511.16719) · HF Papers

SAM3 segments by concept, not location: give it a noun phrase and it finds every matching instance. A ViT-L backbone and FPN feed a DETR-style encoder/decoder with object queries; a CLIP text encoder supplies the open-vocabulary side. Boxes can still be mixed with text.

For more details on the model, please go to Meta's original model card.

Pure-Keras 3 conversion of facebook/sam3 for kerasformers. One implementation runs unmodified on TensorFlow / Torch / JAX.

This is a concept-prompted checkpoint (SAM3InstanceSegment / SAM3Detect / SAM3SemanticSegment): pass a text noun phrase (backbone ViT-L/14).

✨ Quick start

import os
os.environ["KERAS_BACKEND"] = "torch"  # or "jax" / "tensorflow"

from kerasformers.models.sam3 import SAM3InstanceSegment

segmenter = SAM3InstanceSegment(variant="kerasformers/sam3")
result = segmenter.predict(
    images="your_image.jpg", text="person", threshold=0.3
)[0]
print(len(result["scores"]), result["masks"].shape)

Load any SAM / SAM2 / SAM3 variant the same way with from_weights("kerasformers/<variant>") (use SAM3InstanceSegment for this repo):

Variant	Hub	Family
sam_vit_base	kerasformers/sam_vit_base	SAM
sam_vit_large	kerasformers/sam_vit_large	SAM
sam_vit_huge	kerasformers/sam_vit_huge	SAM
sam2_hiera_small	kerasformers/sam2_hiera_small	SAM2
sam2_hiera_base_plus	kerasformers/sam2_hiera_base_plus	SAM2
sam2_hiera_large	kerasformers/sam2_hiera_large	SAM2
sam3	kerasformers/sam3	SAM3

Tips

• Set KERAS_BACKEND before importing Keras / kerasformers.
• SAM / SAM2: point coordinates are in original pixel space; box prompts need enable_boxes=True / include_box_input=True when building the graph.
• SAM2 in this port is image-only (no video memory bank).
• SAM3: prefer SAM3InstanceSegment.predict(...) for text prompts; upstream facebook/sam3 is gated.
• See SAM3 docs and Loading Weights.
• Community / upstream safetensors still work via the hf: prefix, e.g. SAM3Model.from_weights("hf:facebook/sam3").

Special Thanks

A huge thank you to the Meta SAM 3 authors for creating and releasing these models.

License: see the SAM 3 LICENSE (Hub tag: other / sam-license). Upstream facebook/sam3 is gated.