Update README.md
Browse files
README.md
CHANGED
|
@@ -13,13 +13,11 @@ datasets:
|
|
| 13 |
- genecis
|
| 14 |
pipeline_tag: feature-extraction
|
| 15 |
library_name: pytorch
|
| 16 |
-
base_model:
|
| 17 |
-
- openai/clip-vit-large-patch14
|
| 18 |
---
|
| 19 |
|
| 20 |
# Pretrain like Your Inference: Masked Tuning Improves Zero-Shot Composed Image Retrieval
|
| 21 |
|
| 22 |
-
This repository contains the official pre-trained and tuned model weights for **PLI (Pretrain like Your Inference)**, accepted at **ICME 2025**.
|
| 23 |
|
| 24 |
[](https://github.com/Chen-Junyang-cn/PLI)
|
| 25 |
[](https://arxiv.org/abs/2311.07622)
|
|
@@ -55,15 +53,14 @@ import clip
|
|
| 55 |
from huggingface_hub import hf_hub_download
|
| 56 |
|
| 57 |
# 1. Download the weights from Hugging Face
|
| 58 |
-
# Replace 'Chen-Junyang/PLI' with your actual Hugging Face repo ID
|
| 59 |
checkpoint_path = hf_hub_download(
|
| 60 |
-
repo_id="
|
| 61 |
-
filename="
|
| 62 |
)
|
| 63 |
|
| 64 |
-
# 2. Initialize the base CLIP model (
|
| 65 |
device = "cuda" if torch.cuda.is_available() else "cpu"
|
| 66 |
-
model, preprocess = clip.load("
|
| 67 |
|
| 68 |
# 3. Load the tuned weights
|
| 69 |
state_dict = torch.load(checkpoint_path, map_location=device)
|
|
@@ -74,32 +71,6 @@ print("PLI model weights loaded successfully!")
|
|
| 74 |
|
| 75 |
---
|
| 76 |
|
| 77 |
-
## 📊 Evaluation Results
|
| 78 |
-
|
| 79 |
-
PLI achieves state-of-the-art performance on four major ZS-CIR benchmark datasets:
|
| 80 |
-
|
| 81 |
-
### 1. FashionIQ
|
| 82 |
-
Evaluated on the validation set of FashionIQ.
|
| 83 |
-
|
| 84 |
-
| Category | R@10 | R@50 |
|
| 85 |
-
| :--- | :---: | :---: |
|
| 86 |
-
| Dress | - | - |
|
| 87 |
-
| Shirt | - | - |
|
| 88 |
-
| Toptee | - | - |
|
| 89 |
-
| **Average** | **-** | **-** |
|
| 90 |
-
|
| 91 |
-
### 2. CIRR
|
| 92 |
-
Evaluated on the validation set of CIRR.
|
| 93 |
-
|
| 94 |
-
| Method | R@1 | R@5 | R@10 | R@50 |
|
| 95 |
-
| :--- | :---: | :---: | :---: | :---: |
|
| 96 |
-
| CLIP (RN50) | - | - | - | - |
|
| 97 |
-
| **PLI (Ours)** | **-** | **-** | **-** | **-** |
|
| 98 |
-
|
| 99 |
-
*Note: Please update the tables above with the exact values from your paper (Table 1, Table 2, etc.).*
|
| 100 |
-
|
| 101 |
-
---
|
| 102 |
-
|
| 103 |
## 🛠️ Method Overview
|
| 104 |
|
| 105 |
```
|
|
@@ -121,15 +92,17 @@ Evaluated on the validation set of CIRR.
|
|
| 121 |
If you find our work or weights useful in your research, please consider citing our paper:
|
| 122 |
|
| 123 |
```bibtex
|
| 124 |
-
@
|
| 125 |
-
title={Pretrain like
|
| 126 |
author={Chen, Junyang and Lai, Hanjiang},
|
| 127 |
-
|
| 128 |
-
|
|
|
|
|
|
|
| 129 |
}
|
| 130 |
```
|
| 131 |
|
| 132 |
---
|
| 133 |
|
| 134 |
## 📭 Contact / Feedback
|
| 135 |
-
For questions or feedback, please raise an issue on our [GitHub Repository](https://github.com/Chen-Junyang-cn/PLI)
|
|
|
|
| 13 |
- genecis
|
| 14 |
pipeline_tag: feature-extraction
|
| 15 |
library_name: pytorch
|
|
|
|
|
|
|
| 16 |
---
|
| 17 |
|
| 18 |
# Pretrain like Your Inference: Masked Tuning Improves Zero-Shot Composed Image Retrieval
|
| 19 |
|
| 20 |
+
This repository contains the official pre-trained and tuned model weights (CLIP-ViT-L/14 backbone) for **PLI (Pretrain like Your Inference)**, accepted at **ICME 2025**.
|
| 21 |
|
| 22 |
[](https://github.com/Chen-Junyang-cn/PLI)
|
| 23 |
[](https://arxiv.org/abs/2311.07622)
|
|
|
|
| 53 |
from huggingface_hub import hf_hub_download
|
| 54 |
|
| 55 |
# 1. Download the weights from Hugging Face
|
|
|
|
| 56 |
checkpoint_path = hf_hub_download(
|
| 57 |
+
repo_id="jayong/PLI-CLIP-VIT-L-14",
|
| 58 |
+
filename="best.pth"
|
| 59 |
)
|
| 60 |
|
| 61 |
+
# 2. Initialize the base CLIP model (ViT-L/14)
|
| 62 |
device = "cuda" if torch.cuda.is_available() else "cpu"
|
| 63 |
+
model, preprocess = clip.load("ViT-L/14", device=device)
|
| 64 |
|
| 65 |
# 3. Load the tuned weights
|
| 66 |
state_dict = torch.load(checkpoint_path, map_location=device)
|
|
|
|
| 71 |
|
| 72 |
---
|
| 73 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 74 |
## 🛠️ Method Overview
|
| 75 |
|
| 76 |
```
|
|
|
|
| 92 |
If you find our work or weights useful in your research, please consider citing our paper:
|
| 93 |
|
| 94 |
```bibtex
|
| 95 |
+
@inproceedings{chen2025pretrain,
|
| 96 |
+
title={Pretrain like your inference: Masked tuning improves zero-shot composed image retrieval},
|
| 97 |
author={Chen, Junyang and Lai, Hanjiang},
|
| 98 |
+
booktitle={2025 IEEE International Conference on Multimedia and Expo (ICME)},
|
| 99 |
+
pages={1--6},
|
| 100 |
+
year={2025},
|
| 101 |
+
organization={IEEE}
|
| 102 |
}
|
| 103 |
```
|
| 104 |
|
| 105 |
---
|
| 106 |
|
| 107 |
## 📭 Contact / Feedback
|
| 108 |
+
For questions or feedback, please raise an issue on our [GitHub Repository](https://github.com/Chen-Junyang-cn/PLI).
|