---
tags:
- zero-shot-image-classification
- vision-language
- adversarial-robustness
- clip
- pytorch
- cvpr-2026
- arxiv:2604.11576
pipeline_tag: zero-shot-image-classification
base_model: openai/clip-vit-base-patch32
base_model_relation: finetune
---

# AdvFLYP

Official model checkpoints for **Finetune Like You Pretrain: Boosting Zero-shot Adversarial Robustness in Vision-language Models** (CVPR Findings 2026).

- Paper: [arXiv:2604.11576](https://arxiv.org/abs/2604.11576)
- Code and evaluation instructions: [Sxing2/AdvFLYP](https://github.com/Sxing2/AdvFLYP)

AdvFLYP adversarially finetunes CLIP ViT-B/32 on one million image-text pairs sampled from LAION-400M. The full variant adds logit- and feature-level regularisation.

## Checkpoints

| File | Variant | SHA-256 |
|---|---|---|
| `AdvFLYP_full_checkpoint.pth.tar` | AdvFLYP with logit- and feature-level regularisation (`AdvFLYP_full` in the paper) | `97fe54264a98d09787a3e819405cf32e0d6e2c2eb0dd592746d42ec3974cac37` |
| `AdvFLYP_NonReg_checkpoint.pth.tar` | AdvFLYP without regularisation | `263f07ba2612cdd96de9684b4ce822fddd545298e38ef7cbde21283cc1616377` |

The files are the original PyTorch training checkpoints released by the authors. They contain the finetuned partial state dict and training state expected by the official repository.

## Download

```python
from huggingface_hub import hf_hub_download

checkpoint_path = hf_hub_download(
    repo_id="xingslong/AdvFLYP",
    filename="AdvFLYP_full_checkpoint.pth.tar",
)
print(checkpoint_path)
```

For the non-regularised checkpoint, set `filename="AdvFLYP_NonReg_checkpoint.pth.tar"`.

## Evaluation

Clone the official repository and prepare its environment and datasets as described there. Then run:

```bash
git clone https://github.com/Sxing2/AdvFLYP.git
cd AdvFLYP

TEST_MODEL_PATH=/path/to/AdvFLYP_full_checkpoint.pth.tar
TEST_SET=(cifar10 cifar100 STL10 Caltech101 Caltech256 oxfordpet flowers102 Food101 StanfordCars SUN397 Country211 fgvc_aircraft EuroSAT dtd)

python -m code.main \
  --evaluate \
  --root ./data \
  --resume "$TEST_MODEL_PATH" \
  --test_set "${TEST_SET[@]}" \
  --batch_size 256 \
  --test_attack_type pgd \
  --test_eps 1 \
  --test_numsteps 10 \
  --test_stepsize 1
```

The official implementation initializes OpenAI CLIP ViT-B/32 and loads the checkpoint's `partial_state_dict` with `strict=False`.

## Training data

The models were finetuned on one million reachable image-text pairs randomly sampled from LAION-400M. See the GitHub repository for the released data and preparation details.

## Intended use and limitations

These checkpoints are intended for research on zero-shot image classification and adversarial robustness. Evaluation in the paper covers 14 downstream datasets under the stated attack configuration. Performance outside those datasets, threat models, and preprocessing settings has not been established.

Because the training images originate from web-scale LAION data, the checkpoints may inherit biases and other limitations of CLIP and the underlying data.

## Citation

```bibtex
@InProceedings{Xing_2026_CVPR,
  author    = {Xing, Songlong and Wang, Weijie and Zhao, Zhengyu and Gu, Jindong and Torr, Philip and Sebe, Nicu},
  title     = {Finetune Like You Pretrain: Boosting Zero-shot Adversarial Robustness in Vision-language Models},
  booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Findings},
  month     = {June},
  year      = {2026},
  pages     = {737--747}
}
```