doem1997/rs_lt

MetaPEFT checkpoints (CVPR 2025 Highlight)

Model weights for the CVPR 2025 Highlight paper "Meta-Learning Hyperparameters for Parameter Efficient Fine-Tuning" by Zichen Tian, Yaoyao Liu and Qianru Sun.

Paper

• Title: Meta-Learning Hyperparameters for Parameter Efficient Fine-Tuning
• Venue: CVPR 2025 (Highlight), pp. 23037-23047
• arXiv: https://arxiv.org/abs/2603.01759
• CVF Open Access: https://openaccess.thecvf.com/content/CVPR2025/html/Tian_Meta-Learning_Hyperparameters_for_Parameter_Efficient_Fine-Tuning_CVPR_2025_paper.html
• Project page: https://www.doem1997.com/metapeft/
• Code: https://github.com/doem97/metalora
• Dataset: https://huggingface.co/datasets/doem1997/rs_lt

Abstract

Training large foundation models from scratch for domain-specific applications is almost impossible due to data limits and long-tailed distributions -- taking remote sensing (RS) as an example. Fine-tuning natural image pre-trained models on RS images is a straightforward solution. To reduce computational costs and improve performance on tail classes, existing methods apply parameter-efficient fine-tuning (PEFT) techniques, such as LoRA and AdaptFormer. However, we observe that fixed hyperparameters -- such as intra-layer positions, layer depth, and scaling factors, can considerably hinder PEFT performance, as fine-tuning on RS images proves highly sensitive to these settings. To address this, we propose MetaPEFT, a method incorporating adaptive scalers that dynamically adjust module influence during fine-tuning. MetaPEFT dynamically adjusts three key factors of PEFT on RS images: module insertion, layer selection, and module-wise learning rates, which collectively control the influence of PEFT modules across the network. We conduct extensive experiments on three transfer-learning scenarios and five datasets in both RS and natural image domains. The results show that MetaPEFT achieves state-of-the-art performance in cross-spectral adaptation, requiring only a small amount of trainable parameters and improving tail-class accuracy significantly.

Citation

@InProceedings{Tian_2025_CVPR,
    author    = {Tian, Zichen and Liu, Yaoyao and Sun, Qianru},
    title     = {Meta-Learning Hyperparameters for Parameter Efficient Fine-Tuning},
    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
    month     = {June},
    year      = {2025},
    pages     = {23037-23047}
}