File size: 1,666 Bytes
625a17f |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 |
# PSALM: Pixelwise SegmentAtion with Large Multi-Modal Model
> #### Zheng Zhang\*, Yeyao Ma\*, Enming Zhang\*, Xiang Bai
>
> <sup>* Equal Contribution
[Arxiv Paper](https://arxiv.org/abs/2403.14598)
### Features
* A powerful extension of the Large Multi-modal Model for generic (panoptic, instance, semantic) segmentation, referring segmentation and interactivate segmentation.
* Support joint training across multiple segmentation tasks and visual-language tasks.
* Demonstrates zero-shot capabilities on unseen task, such as open-vocabulary segmentation, generalizaed referring segmentation, and video object segmentation.
## Updates
- [x] Release evaluation code
- [x] Release training code
## Installation
See [Installation instructions](docs/INSTALL.md).
## Getting Started
See [Preparing Datasets for PSALM.](docs/DATASET.md)
See [Getting Started with PSALM.](docs/GETTING_STARTED.md)
## Model Zoo
- Download PSALM [here](https://huggingface.co/EnmingZhang/PSALM).
## Citation
If you think this work is useful for your research, please use the following BibTeX entry.
```
@misc{zhang2024psalm,
title={PSALM: Pixelwise SegmentAtion with Large Multi-Modal Model},
author={Zheng Zhang and Yeyao Ma and Enming Zhang and Xiang Bai},
year={2024},
eprint={2403.14598},
archivePrefix={arXiv},
primaryClass={cs.CV}
}
```
## Acknowledgement
Thanks for awesome works: [Mask2former](https://github.com/facebookresearch/Mask2Former/tree/main), [Mask2former-Simplify](https://github.com/zzubqh/Mask2Former-Simplify)
and [LLaVA](https://github.com/haotian-liu/LLaVA). Code is based on these works.
|