FoSSIL: A Unified Framework for Continual Semantic Segmentation in 2D and 3D Domains
Abstract
Continual semantic segmentation remains an underexplored area in both 2D and 3D domains. The problem becomes particularly challenging when classes and domains evolve over time, with incremental classes having only a few labeled samples. In this setting, the model must simultaneously address catastrophic forgetting of old classes, overfitting due to the limited labeled data of new classes, and domain shifts arising from changes in data distribution. Existing methods fail to simultaneously address these real-world constraints. We introduce the FoSSIL framework, which integrates guided noise injection and prototype-guided pseudo-label refinement (PLR) to enhance continual learning across class-incremental (CIL), domain-incremental (DIL), and few-shot scenarios. Guided noise injection perturbs parameters with overfitted or saturated gradients more strongly, while perturbing parameters with highly changing or large gradients less, preserving critical weights, allowing less critical parameters to explore alternative solutions in the parameter space, mitigating forgetting, reducing overfitting, and improving robustness to domain shifts. For incremental classes with unlabeled data, PLR enables semi-supervised learning by refining pseudo-labels and filtering out incorrect high-confidence predictions, ensuring reliable supervision for incremental classes. Together, these components work synergistically to enhance stability, generalization, and continual learning across all learning regimes.
Code: https://github.com/anony34/FoSSIL
Webpage: https://anony34.github.io/Fossil_webpage/
Theoretical analysis: https://anony34.github.io/Fossil_webpage/theory.html
🌿 Checkpoints for Med FoSSIL-Disjoint
| Session | Download Link |
|---|---|
| Session 0 | Download |
| Session 1 | Download |
| Session 2 | Download |
| Session 3 | Download |
| Session 4 | Download |
| Session 5 | Download |
🌿 Checkpoints for Semi-Supervised Natural-FoSSIL
🌿 Checkpoints for Natural-FoSSIL (SAM)
| Session | Download Link |
|---|---|
| Session 0 | Download |