Instructions to use rostrumstudios/sam2-tiny with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- sam2
How to use rostrumstudios/sam2-tiny with sam2:
# Use SAM2 with images import torch from sam2.sam2_image_predictor import SAM2ImagePredictor predictor = SAM2ImagePredictor.from_pretrained("rostrumstudios/sam2-tiny") with torch.inference_mode(), torch.autocast("cuda", dtype=torch.bfloat16): predictor.set_image(<your_image>) masks, _, _ = predictor.predict(<input_prompts>)# Use SAM2 with videos import torch from sam2.sam2_video_predictor import SAM2VideoPredictor predictor = SAM2VideoPredictor.from_pretrained("rostrumstudios/sam2-tiny") with torch.inference_mode(), torch.autocast("cuda", dtype=torch.bfloat16): state = predictor.init_state(<your_video>) # add new prompts and instantly get the output on the same frame frame_idx, object_ids, masks = predictor.add_new_points(state, <your_prompts>) # propagate the prompts to get masklets throughout the video for frame_idx, object_ids, masks in predictor.propagate_in_video(state): ... - Notebooks
- Google Colab
- Kaggle
|
Download README.md from rostrumstudios/sam2-tiny: direct link, hf CLI and curl.
- Browser
- Download file 2.9 kB
-
https://huggingface.co/rostrumstudios/sam2-tiny/resolve/main/README.md
- Command line
-
hf download hf://rostrumstudios/sam2-tiny/README.md
-
curl -L -o README.md https://huggingface.co/rostrumstudios/sam2-tiny/resolve/main/README.md
2.9 kB
| license: apache-2.0 | |
| base_model: | |
| - facebook/sam2.1-hiera-tiny | |
| pipeline_tag: mask-generation | |
| library_name: onnx | |
| tags: | |
| - onnx | |
| - rostrum | |
| - mask-generation | |
| - segment-anything | |
| - sam2 | |
| # SAM 2.1 Hiera Tiny (ONNX encoder + decoder), mirrored for Rostrum | |
| Meta's Segment Anything 2.1 (Hiera Tiny) as an ONNX image encoder and prompt decoder: unmodified inference exports (PyTorch 2.9.0, opset 18) made by the Voxide project. | |
| This repository is a pinned, public copy of the exact files the [Rostrum](https://huggingface.co/rostrumstudios) apps download. Rostrum fetches each file at a fixed commit of this repository and checks its SHA-256 before use. | |
| ## What Rostrum uses it for | |
| Compose's AI roto: it finds what you click on in a shot, and the mask becomes an editable shape that follows the subject. It never draws or changes pixels. | |
| Rostrum uses it for **analysis only, never generation**: it reads what is in a recording or a picture and makes no new sound, voice or image. | |
| ## Files | |
| | File | Bytes | SHA-256 | | |
| |---|---:|---| | |
| | `sam2.1_hiera_tiny_encoder.onnx` | 109,471,930 | `ac335045c272ef63a74488da140fe4cf2e8ddd5db2e65c4ad29949964ac81720` | | |
| | `sam2.1_hiera_tiny_decoder.onnx` | 16,520,098 | `1e1e45efa3f9e6072e3d99a4f387c57d227703d5e54a4aee1c66f91872a892f0` | | |
| In Rostrum's models folder they go under `sam2.1-hiera-tiny/`. | |
| ## Source | |
| | What | Where | Commit / version | | |
| |---|---|---| | |
| | Exact files (unchanged copy) | https://huggingface.co/voxide/voxide-models | 5f1fd124e832c7bbea49a518a9b38ab3bf8bb337 | | |
| | Original weights, Meta | https://huggingface.co/facebook/sam2.1-hiera-tiny | de431c4043854a71d8101e17995dfe596bf101a5 (main when mirrored) | | |
| | Original code, Meta | https://github.com/facebookresearch/sam2 | 2b90b9f5ceec907a1c18123530e92e794ad901a4 (main when mirrored) | | |
| ## Authors | |
| Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman Rädle, Chloe Rolland, Laura Gustafson, Eric Mintun, Junting Pan, Kalyan Vasudev Alwala, Nicolas Carion, Chao-Yuan Wu, Ross Girshick, Piotr Dollár, Christoph Feichtenhofer (Meta FAIR, SAM 2); Voxide (ONNX export). | |
| ## Licence | |
| **Apache-2.0** for the code, the weights and the exports (`LICENSE`, from the original repository). Files redistributed unchanged. | |
| ## Citation | |
| The original authors ask that you cite: | |
| ```bibtex | |
| @article{ravi2024sam2, | |
| title={SAM 2: Segment Anything in Images and Videos}, | |
| author={Ravi, Nikhila and Gabeur, Valentin and Hu, Yuan-Ting and Hu, Ronghang and Ryali, Chaitanya and Ma, Tengyu and Khedr, Haitham and R{\"a}dle, Roman and Rolland, Chloe and Gustafson, Laura and Mintun, Eric and Pan, Junting and Alwala, Kalyan Vasudev and Carion, Nicolas and Wu, Chao-Yuan and Girshick, Ross and Doll{\'a}r, Piotr and Feichtenhofer, Christoph}, | |
| journal={arXiv preprint arXiv:2408.00714}, | |
| url={https://arxiv.org/abs/2408.00714}, | |
| year={2024} | |
| } | |
| ``` | |