| --- |
| pipeline_tag: image-segmentation |
| --- |
| |
| # InstructSAM: Segment Any Instance with Any Instructions |
|
|
| InstructSAM is a unified and streamlined framework designed for multi-instance segmentation under arbitrary instructions. It formulates instruction-driven instance segmentation as a set-structured query prediction problem, bridging a vision-language model (VLM) and SAM3. This design equips SAM3 with high-level instruction understanding and compositional reasoning without modifying its core architecture. |
|
|
| - **Paper:** [InstructSAM: Segment Any Instance with Any Instructions](https://huggingface.co/papers/2605.26102) |
| - **Repository:** [https://github.com/DCDmllm/InstructSAM](https://github.com/DCDmllm/InstructSAM) |
|
|
| ## Usage |
|
|
| To use this model, please refer to the [official repository](https://github.com/DCDmllm/InstructSAM) for environment setup and installation. |
|
|
| You can run single-image inference using the provided inference script: |
|
|
| ```bash |
| python3 -m instructsam.infer \ |
| --model_path CircleRadon/InstructSAM-2B \ |
| --image-path path/to/image.jpg \ |
| --query "Please segment the object in the image." \ |
| --output-dir vis |
| ``` |
|
|
| The script prints the generated text and mask scores, then writes mask overlays to `vis/`. |
|
|
| ## Citation |
|
|
| If you find this project useful, please cite using this BibTeX: |
|
|
| ```bibtex |
| @article{yuan2026instructsam, |
| title = {InstructSAM: Segment Any Instance with Any Instructions}, |
| author = {Yuqian Yuan, Wentong Li, Zhaocheng Li Yutong Lin, Juncheng Li, Siliang Tang, Jun Xiao, Yueting Zhuang, Wenqiao Zhang}, |
| year = {2026}, |
| journal = {arXiv}, |
| } |
| ``` |
