---
license: apache-2.0
tags:
  - remote-sensing
  - long-tailed
  - parameter-efficient-fine-tuning
  - peft
  - lora
  - meta-learning
  - cvpr2025
  - arxiv:2603.01759
---

# MetaPEFT checkpoints (CVPR 2025 Highlight)

Model weights for the CVPR 2025 **Highlight** paper
**"Meta-Learning Hyperparameters for Parameter Efficient Fine-Tuning"**
by Zichen Tian, Yaoyao Liu and Qianru Sun.

<!-- TODO Zichen: 一句话描述这个 repo 里是什么 checkpoint（backbone、在哪个 split 上训的），我不知道就没写 -->

## Paper

- **Title:** Meta-Learning Hyperparameters for Parameter Efficient Fine-Tuning
- **Venue:** CVPR 2025 (Highlight), pp. 23037-23047
- **arXiv:** https://arxiv.org/abs/2603.01759
- **CVF Open Access:** https://openaccess.thecvf.com/content/CVPR2025/html/Tian_Meta-Learning_Hyperparameters_for_Parameter_Efficient_Fine-Tuning_CVPR_2025_paper.html
- **Project page:** https://www.doem1997.com/metapeft/
- **Code:** https://github.com/doem97/metalora
- **Dataset:** https://huggingface.co/datasets/doem1997/rs_lt

## Abstract

Training large foundation models from scratch for domain-specific applications is almost impossible due to data limits and long-tailed distributions -- taking remote sensing (RS) as an example. Fine-tuning natural image pre-trained models on RS images is a straightforward solution. To reduce computational costs and improve performance on tail classes, existing methods apply parameter-efficient fine-tuning (PEFT) techniques, such as LoRA and AdaptFormer. However, we observe that fixed hyperparameters -- such as intra-layer positions, layer depth, and scaling factors, can considerably hinder PEFT performance, as fine-tuning on RS images proves highly sensitive to these settings. To address this, we propose MetaPEFT, a method incorporating adaptive scalers that dynamically adjust module influence during fine-tuning. MetaPEFT dynamically adjusts three key factors of PEFT on RS images: module insertion, layer selection, and module-wise learning rates, which collectively control the influence of PEFT modules across the network. We conduct extensive experiments on three transfer-learning scenarios and five datasets in both RS and natural image domains. The results show that MetaPEFT achieves state-of-the-art performance in cross-spectral adaptation, requiring only a small amount of trainable parameters and improving tail-class accuracy significantly.

## Citation

```bibtex
@InProceedings{Tian_2025_CVPR,
    author    = {Tian, Zichen and Liu, Yaoyao and Sun, Qianru},
    title     = {Meta-Learning Hyperparameters for Parameter Efficient Fine-Tuning},
    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
    month     = {June},
    year      = {2025},
    pages     = {23037-23047}
}
```