How to use from the
Use from the
sam2 library
# Use SAM2 with images
import torch
from sam2.sam2_image_predictor import SAM2ImagePredictor

predictor = SAM2ImagePredictor.from_pretrained("rostrumstudios/sam2-tiny")

with torch.inference_mode(), torch.autocast("cuda", dtype=torch.bfloat16):
    predictor.set_image(<your_image>)
    masks, _, _ = predictor.predict(<input_prompts>)
# Use SAM2 with videos
import torch
from sam2.sam2_video_predictor import SAM2VideoPredictor

predictor = SAM2VideoPredictor.from_pretrained("rostrumstudios/sam2-tiny")

with torch.inference_mode(), torch.autocast("cuda", dtype=torch.bfloat16):
    state = predictor.init_state(<your_video>)

    # add new prompts and instantly get the output on the same frame
    frame_idx, object_ids, masks = predictor.add_new_points(state, <your_prompts>)

    # propagate the prompts to get masklets throughout the video
    for frame_idx, object_ids, masks in predictor.propagate_in_video(state):
        ...

SAM 2.1 Hiera Tiny (ONNX encoder + decoder), mirrored for Rostrum

Meta's Segment Anything 2.1 (Hiera Tiny) as an ONNX image encoder and prompt decoder: unmodified inference exports (PyTorch 2.9.0, opset 18) made by the Voxide project.

This repository is a pinned, public copy of the exact files the Rostrum apps download. Rostrum fetches each file at a fixed commit of this repository and checks its SHA-256 before use.

What Rostrum uses it for

Compose's AI roto: it finds what you click on in a shot, and the mask becomes an editable shape that follows the subject. It never draws or changes pixels.

Rostrum uses it for analysis only, never generation: it reads what is in a recording or a picture and makes no new sound, voice or image.

Files

File Bytes SHA-256
sam2.1_hiera_tiny_encoder.onnx 109,471,930 ac335045c272ef63a74488da140fe4cf2e8ddd5db2e65c4ad29949964ac81720
sam2.1_hiera_tiny_decoder.onnx 16,520,098 1e1e45efa3f9e6072e3d99a4f387c57d227703d5e54a4aee1c66f91872a892f0

In Rostrum's models folder they go under sam2.1-hiera-tiny/.

Source

What Where Commit / version
Exact files (unchanged copy) https://huggingface.co/voxide/voxide-models 5f1fd124e832c7bbea49a518a9b38ab3bf8bb337
Original weights, Meta https://huggingface.co/facebook/sam2.1-hiera-tiny de431c4043854a71d8101e17995dfe596bf101a5 (main when mirrored)
Original code, Meta https://github.com/facebookresearch/sam2 2b90b9f5ceec907a1c18123530e92e794ad901a4 (main when mirrored)

Authors

Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman Rädle, Chloe Rolland, Laura Gustafson, Eric Mintun, Junting Pan, Kalyan Vasudev Alwala, Nicolas Carion, Chao-Yuan Wu, Ross Girshick, Piotr Dollár, Christoph Feichtenhofer (Meta FAIR, SAM 2); Voxide (ONNX export).

Licence

Apache-2.0 for the code, the weights and the exports (LICENSE, from the original repository). Files redistributed unchanged.

Citation

The original authors ask that you cite:

@article{ravi2024sam2,
  title={SAM 2: Segment Anything in Images and Videos},
  author={Ravi, Nikhila and Gabeur, Valentin and Hu, Yuan-Ting and Hu, Ronghang and Ryali, Chaitanya and Ma, Tengyu and Khedr, Haitham and R{\"a}dle, Roman and Rolland, Chloe and Gustafson, Laura and Mintun, Eric and Pan, Junting and Alwala, Kalyan Vasudev and Carion, Nicolas and Wu, Chao-Yuan and Girshick, Ross and Doll{\'a}r, Piotr and Feichtenhofer, Christoph},
  journal={arXiv preprint arXiv:2408.00714},
  url={https://arxiv.org/abs/2408.00714},
  year={2024}
}
Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for rostrumstudios/sam2-tiny

Quantized
(20)
this model

Collection including rostrumstudios/sam2-tiny

Paper for rostrumstudios/sam2-tiny