| # DMNSP: Dynamic Multi-Layer Null Space Projection for Vision-Language Continual Learning | |
| [](https://www.python.org/downloads/release/python-380/) | |
| [](https://pytorch.org/) | |
| [](LICENSE) | |
| Official implementation of the paper "Dynamic Multi-Layer Null Space Projection for Vision-Language Continual Learning" (ICCV 2025) in PyTorch. | |
| ## 🎯 Abstract | |
| Vision-Language Models (VLM) have emerged as a highly promising approach for Continual Learning (CL) due to their powerful generalized features. While adapter-based VLM can exploit both task-specific and task-agnostic features, current CL methods have largely overlooked the distinct and evolving parameter distributions in visual and language modalities, which are found crucial for effectively mitigating catastrophic forgetting.In this study, we find that the **visual modality experiences a broader parameter distribution and greater variance** during class increments than the textual modality, leading to higher vulnerability to forgetting. Consequently, we handle the branches of the two modalities asymmetrically. | |
| ### Key Contributions | |
| - 🔍 **Asymmetric Modality Handling**: We propose handling visual and language modalities differently based on their distinct parameter distribution characteristics | |
| - 🚀 **Multi-layer Null Space Projection**: A novel strategy applied only to the visual modality branch to restrict parameter updates within specific subspaces | |
| - ⚖️ **Dynamic Projection Coefficient**: Precise control of gradient projection magnitude for optimal stability-plasticity balance | |
| ## 🛠️ Installation | |
| ### Setup Environment | |
| ```bash | |
| # Install dependencies | |
| pip install -r requirements.txt | |
| ``` | |
| ## 📊 Datasets | |
| The framework supports the following datasets for class incremental learning: | |
| - **CIFAR100**: 100 classes, various incremental settings (2-2, 5-5, 10-10) | |
| - **TinyImageNet**: 200 classes, incremental settings (200-100-5, 200-100-10, 200-100-20) | |
| ### Data Preparation | |
| 1. The datasets will be automatically downloaded when running experiments | |
| 2. Update the `dataset_root` path in your configuration files or command line | |
| 3. Ensure sufficient disk space for dataset storage | |
| ## 🚀 Quick Start | |
| ### Basic Usage | |
| ```bash | |
| # Run CIFAR100 with 10 initial classes and 10 incremental classes | |
| sh run_cifar100-10-10.sh | |
| # Or run with custom parameters | |
| python main.py \ | |
| --config-path ./configs/class \ | |
| --config-name cifar100_10-10.yaml \ | |
| dataset_root="/path/to/your/data" \ | |
| class_order="./class_orders/cifar100.yaml" | |
| ``` | |
| ### Configuration Options | |
| The project uses Hydra for configuration management. Key parameters include: | |
| ```yaml | |
| # Model settings | |
| model_name: "ViT-B/16" # CLIP model variant | |
| prompt_template: "a bad photo of a {}." # Text prompt template | |
| # Training settings | |
| batch_size: 128 # Training batch size | |
| lr: 1e-3 # Learning rate | |
| weight_decay: 0.0 # Weight decay | |
| ls: 0.0 # Label smoothing | |
| # Incremental learning settings | |
| initial_increment: 10 # Initial number of classes | |
| increment: 10 # Classes per incremental step | |
| method: "DMNSP" # Method name | |
| ``` | |
| ## 🔧 Advanced Usage | |
| ### Custom Datasets | |
| To add support for new datasets: | |
| 1. Add dataset configuration in `continual_clip/datasets.py` | |
| 2. Create corresponding class order file in `class_orders/` | |
| 3. Add configuration YAML in `configs/class/` | |
| ## 📄 License | |
| This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details. | |
| ## 📚 Citation | |
| If you find this work useful in your research, please consider citing: | |
| ```bibtex | |
| @inproceedings{Kang2025DMNSP, | |
| title={Dynamic Multi-Layer Null Space Projection for Vision-Language Continual Learning}, | |
| author={Borui Kang, Lei Wang, Zhiping Wu, Tao Feng, Yawen Li, Yang Gao, Wenbin Li}, | |
| journal={ICCV}, | |
| year={2025} | |
| } | |
| ``` | |
| ## 📞 Contact | |
| For questions or issues, please: | |
| - Open an issue on GitHub | |
| - Contact the authors at [kangborui.cn@gmail.com] | |
| --- | |
| **Note**: This implementation is for research purposes. Please ensure you comply with the respective licenses of the datasets and models used. | |