nielsr HF Staff commited on
Commit
88f0a99
·
verified ·
1 Parent(s): 799ace9

Improve model card: Add pipeline tag, library name, abstract, and updated links

Browse files

This PR significantly enhances the model card for the SimCIS model on the Hugging Face Hub by:

* **Adding `pipeline_tag: image-segmentation`** to the metadata, ensuring the model appears under relevant filters on the Hugging Face Models page.
* **Adding `library_name: detectron2`** to the metadata, correctly associating the model with its primary framework for better categorization and searchability.
* **Updating the paper link** to `https://huggingface.co/papers/2507.07831` (the official Hugging Face Papers page), replacing the local PDF link and improving discoverability. The badge and other mentions of the paper link in the content have also been updated.
* **Including the paper's abstract** under a new "

Files changed (1) hide show
  1. README.md +126 -10
README.md CHANGED
@@ -1,19 +1,137 @@
1
  ---
2
- language:
3
- - en
4
- license: mit
 
5
  ---
6
 
7
  # SimCIS
8
 
9
  **Rethinking Query-based Transformer for Continual Image Segmentation.** *(CVPR2025)*
10
 
11
- [![License](https://img.shields.io/badge/License-MIT-blue.svg?style=flat-square)](LICENSE)[![Paper](https://badgen.net/badge/icon/arXiv?icon=awesome&label&color=red&style=flat-square)](./assets/CVPR_2025_SimCIS.pdf)
12
- ![SimCLS](assets/framework.svg)
13
 
14
- By [Yuchen Zhu*](https://github.com/ZhuYuChenNO1), [Cheng Shi*](https://chengshiest.github.io/), Dingyou Wang, Jiajin Tang, Zhengxuan Wei, Yu Wu, Guanbin Li and [Sibei Yang†](https://faculty.sist.shanghaitech.edu.cn/yangsibei/)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
15
 
16
- ## SEE [Github](https://github.com/SooLab/SimCIS) to use the weights!!!
17
 
18
  ## 📖 Cite Us
19
  If you find this repository useful in your research, please consider giving a star ⭐ and a citation
@@ -30,6 +148,4 @@ If you find this repository useful in your research, please consider giving a st
30
 
31
  ## 👍 Acknowledgement and Related Work
32
  * This code is mainly based on [Mask2Former](https://github.com/facebookresearch/Mask2Former). We thank them for their excellent work.
33
- * Related work for continual image segmentation: [Balconpas](https://github.com/jinpeng0528/BalConpas/tree/master), [ECLIPSE](https://github.com/clovaai/ECLIPSE). We appreciate the contributions of these researchers.
34
-
35
-
 
1
  ---
2
+ language: en
3
+ license: mit
4
+ pipeline_tag: image-segmentation
5
+ library_name: detectron2
6
  ---
7
 
8
  # SimCIS
9
 
10
  **Rethinking Query-based Transformer for Continual Image Segmentation.** *(CVPR2025)*
11
 
12
+ [![License](https://img.shields.io/badge/License-MIT-blue.svg?style=flat-square)](LICENSE)[![Paper](https://img.shields.io/badge/Paper-2507.07831-red?style=flat-square)](https://huggingface.co/papers/2507.07831)[![GitHub](https://img.shields.io/badge/GitHub-Code-black.svg?logo=github&style=flat-square)](https://github.com/SooLab/SimCIS)
 
13
 
14
+ ![SimCLS](https://huggingface.co/LightningNO1/SimCIS/resolve/main/assets/framework.svg)
15
+
16
+ By [Yuchen Zhu*](https://github.com/ZhuYuChenNO1), [Cheng Shi*](https://chengshiest.github.io/), Dingyou Wang, Jiajin Tang, Zhengxuan Wei, Yu Wu, Guanbin Li and [Sibei Yang†](https://faculty.sist.shanghaitech.edu.cn/yangsibei/)
17
+
18
+ *Equal contribution; †Corresponding Author
19
+
20
+ ## Abstract
21
+ Class-incremental/Continual image segmentation (CIS) aims to train an image segmenter in stages, where the set of available categories differs at each stage. To leverage the built-in objectness of query-based transformers, which mitigates catastrophic forgetting of mask proposals, current methods often decouple mask generation from the continual learning process. This study, however, identifies two key issues with decoupled frameworks: loss of plasticity and heavy reliance on input data order. To address these, we conduct an in-depth investigation of the built-in objectness and find that highly aggregated image features provide a shortcut for queries to generate masks through simple feature alignment. Based on this, we propose SimCIS, a simple yet powerful baseline for CIS. Its core idea is to directly select image features for query assignment, ensuring "perfect alignment" to preserve objectness, while simultaneously allowing queries to select new classes to promote plasticity. To further combat catastrophic forgetting of categories, we introduce cross-stage consistency in selection and an innovative "visual query"-based replay mechanism. Experiments demonstrate that SimCIS consistently outperforms state-of-the-art methods across various segmentation tasks, settings, splits, and input data orders. All models and codes will be made publicly available at this https URL.
22
+
23
+ ## 📣 News
24
+
25
+ * [2025.06.17] 🤗 🤗 🤗 We Release the weights on [huggingface](https://huggingface.co/LightningNO1/SimCIS).
26
+ * [2025.06.09] 🤗 We fully release SimCIS, including both code and [paper](https://huggingface.co/papers/2507.07831)!
27
+ * [2025.03.03] We are preparing the code and camera ready version of our paper!
28
+ * [2025.02.27] Our paper is accepted by CVPR2025!
29
+
30
+
31
+ ## 📝 To-do list
32
+
33
+ - [x] Release the code and paper.
34
+ - [x] Release the weights in the next few days.
35
+ - [x] More detailed instructions.
36
+
37
+
38
+ ## 💡 Quick Start
39
+ ### 1. Set up environments
40
+
41
+ ```bash
42
+ conda create --name simcis python=3.8 -y
43
+ conda activate simcis
44
+ conda install pytorch==1.9.0 torchvision==0.10.0 cudatoolkit=11.1 -c pytorch -c nvidia
45
+ pip install -U opencv-python
46
+
47
+ git clone git@github.com:SooLab/SimCIS.git
48
+ cd SimCIS
49
+
50
+ git clone git@github.com:facebookresearch/detectron2.git
51
+ cd detectron2
52
+ pip install -e .
53
+ pip install git+https://github.com/cocodataset/panopticapi.git
54
+ pip install git+https://github.com/mcordts/cityscapesScripts.git
55
+ cd ..
56
+ pip install -r requirements.txt
57
+
58
+ ```
59
+
60
+ #### CUDA kernel for MSDeformAttn
61
+ After preparing the required environment, run the following command to compile CUDA kernel for MSDeformAttn:
62
+
63
+ `CUDA_HOME` must be defined and points to the directory of the installed CUDA toolkit.
64
+
65
+ ```bash
66
+ cd mask2former/modeling/pixel_decoder/ops
67
+ sh make.sh
68
+ ```
69
+
70
+ ## 2. Data Preparation
71
+
72
+ We follow the previous work [Balconpas](https://github.com/jinpeng0528/BalConpas/tree/master) to prepare the training data.
73
+
74
+ Please download the ADE20K dataset and its instance annotation from [here](http://sceneparsing.csail.mit.edu/), then place the dataset in or create a symbolic link to the `./datasets` directory. The structure of data path should be organized as follows:
75
+ ```
76
+ ADEChallengeData2016/
77
+ images/
78
+ annotations/
79
+ objectInfo150.txt
80
+ sceneCategories.txt
81
+ annotations_instance/
82
+ annotations_detectron2/
83
+ ade20k_panoptic_{train,val}.json
84
+ ade20k_panoptic_{train,val}/
85
+ ade20k_instance_{train,val}.json
86
+ ```
87
+ The directory `annotations_detectron2` is generated by running `python datasets/prepare_ade20k_sem_seg.py`.
88
+ Then, run `python datasets/prepare_ade20k_pan_seg.py` to combine semantic and instance annotations for panoptic annotations and run `python datasets/prepare_ade20k_ins_seg.py` to extract instance annotations in COCO format.
89
+
90
+ To fit the requirements of continual segmentation tasks, run `python continual/prepare_datasets.py` to reorganize the annotations (reorganized annotations will be placed in `./json`).
91
+
92
+ #### Example data preparation
93
+ ```bash
94
+ # for Mask2Former
95
+ cd datasets
96
+ wget http://data.csail.mit.edu/places/ADEchallenge/ADEChallengeData2016.zip
97
+ unzip ADEChallengeData2016.zip
98
+ cd ADEChallengeData2016
99
+ wget http://sceneparsing.csail.mit.edu/data/ChallengeData2017/annotations_instance.tar
100
+ tar -xvf annotations_instance.tar
101
+ cd ../..
102
+ python datasets/prepare_ade20k_sem_seg.py
103
+ python datasets/prepare_ade20k_pan_seg.py
104
+ python datasets/prepare_ade20k_ins_seg.py
105
+
106
+ # for continual segmentation
107
+ python continual/prepare_datasets.py
108
+ ```
109
+
110
+ ## 🔥 Training
111
+
112
+ Download the weights of the base step(step1) from [huggingface](https://huggingface.co/LightningNO1/SimCIS).
113
+
114
+ Please follow the [scripts](./scripts) to train SimCIS!
115
+
116
+ For example:
117
+
118
+ ```bash
119
+ bash scripts/pan_100-5.sh
120
+ ```
121
+
122
+ ## ⚡️ Evaluation
123
+
124
+ Download the weights from [huggingface](https://huggingface.co/LightningNO1/SimCIS).
125
+
126
+ Please follow the [scripts](./scripts) to evaluate SimCIS!
127
+
128
+ For example:
129
+
130
+ ```bash
131
+ # 11 means the 11th step(last step for 100-5 setting)
132
+ bash scripts/panoptic_eval.sh 11
133
+ ```
134
 
 
135
 
136
  ## 📖 Cite Us
137
  If you find this repository useful in your research, please consider giving a star ⭐ and a citation
 
148
 
149
  ## 👍 Acknowledgement and Related Work
150
  * This code is mainly based on [Mask2Former](https://github.com/facebookresearch/Mask2Former). We thank them for their excellent work.
151
+ * Related work for continual image segmentation: [Balconpas](https://github.com/jinpeng0528/BalConpas/tree/master), [ECLIPSE](https://github.com/clovaai/ECLIPSE). We appreciate the contributions of these researchers.