README.md

---
license: mit
language:
- en
---

# PETAL<i>face</i> Model Card

<div align="center">

[**Project Page**](https://kartik-3004.github.io/PETALface/) **|** [**Paper (ArXiv)**](https://arxiv.org/abs/2412.07771) **|** [**Code**](https://github.com/Kartik-3004/PETALface)


</div>

## Introduction

<div  align="center">
<img src='assets/visual_abstract.png' height="50%" width="50%">
</div>

PETAL<i>face</i> is the first work which uses image-quality adaptive LoRA layers for low-resolution face recgonition. 
The main contributions of our work are:
1. We introduce the use of the LoRA-based PETL technique to adapt large pre-trained face-recognition models to low-resolution datasets. 
2. We propose an image-quality-based weighting of LoRA modules to create separate proxy encoders for high-resolution and low-resolution data, 
ensuring effective extraction of embeddings for face recognition. 
3. We demonstrate the superiority of PETAL<i>face</i> in adapting to low-resolution datasets, outperforming other state-of-the-art models on 
low-resolution benchmarks while maintaining performance on high-resolution and mixed-quality datasets.



## Training Framework
<div  align="center">
<img src='assets/petalface.png'>
</div>

Overview of the proposed PETAL<i>face</i> approach: We include an additional trainable module in linear layers present in attention layers and the 
final feature projection MLP. The trainable module is highlighted on the right. Specifically, we add two LoRA layers, where the weightage α is 
decided based on the input-image quality, computed using an off-the-shelf image quality assessment network (IQA).



## Usage

The pre-trained weights can be downloaded directly from this repository or using python:
```python
from huggingface_hub import hf_hub_download

# Finetuned Weights

# The filename "swin_arcface_webface4m_tinyface" indicates that the model has a swin bakcbone and pretraind
# on webface4m dataset with arcface loss function and finetuned on tinyface.
hf_hub_download(repo_id="kartiknarayan/PETALface", filename="swin_arcface_webface4m_tinyface/model.pt", local_dir="./weights")
hf_hub_download(repo_id="kartiknarayan/PETALface", filename="swin_cosface_webface4m_tinyface/model.pt", local_dir="./weights")
hf_hub_download(repo_id="kartiknarayan/PETALface", filename="swin_cosface_webface4m_briar/model.pt", local_dir="./weights")
hf_hub_download(repo_id="kartiknarayan/PETALface", filename="swin_cosface_webface12m_briar/model.pt", local_dir="./weights")

# Pre-trained Weights
hf_hub_download(repo_id="kartiknarayan/PETALface", filename="swin_arcface_webface4m/model.pt", local_dir="./weights")
hf_hub_download(repo_id="kartiknarayan/PETALface", filename="swin_cosface_webface4m/model.pt", local_dir="./weights")
hf_hub_download(repo_id="kartiknarayan/PETALface", filename="swin_arcface_webface12m/model.pt", local_dir="./weights")
hf_hub_download(repo_id="kartiknarayan/PETALface", filename="swin_cosface_webface12m/model.pt", local_dir="./weights")
```

## Citation
```bibtex
@inproceedings{narayan2025petalface,
  title={Petalface: Parameter efficient transfer learning for low-resolution face recognition},
  author={Narayan, Kartik and Nair, Nithin Gopalakrishnan and Xu, Jennifer and Chellappa, Rama and Patel, Vishal M},
  booktitle={2025 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)},
  pages={804--814},
  year={2025},
  organization={IEEE}
}
```

Please check our [GitHub repository](https://github.com/Kartik-3004/PETALface) for complete instructions.