Lee-zixu commited on
Commit
e08a850
Β·
verified Β·
1 Parent(s): 4690f03

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +128 -3
README.md CHANGED
@@ -1,3 +1,128 @@
1
- ---
2
- license: apache-2.0
3
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+
2
+ ---
3
+ license: apache-2.0
4
+ task_categories:
5
+ - image-retrieval
6
+ - vision-language-navigation
7
+ tags:
8
+ - composed-image-retrieval
9
+ - robust-learning
10
+ - blip-2
11
+ - pytorch
12
+ - aaai-2026
13
+ ---
14
+
15
+ <a id="top"></a>
16
+ <div align="center">
17
+ <h1>(AAAI 2026) HABIT: Chrono-Synergia Robust Progressive Learning Framework for Composed Image Retrieval (Model Weights)</h1>
18
+ <div>
19
+ <a target="_blank" href="https://lee-zixu.github.io/">Zixu&#160;Li</a><sup>1</sup>,
20
+ <a target="_blank" href="https://faculty.sdu.edu.cn/huyupeng1/zh_CN/index.htm">Yupeng&#160;Hu</a><sup>1&#9993</sup>,
21
+ <a target="_blank" href="https://zivchen-ty.github.io/">Zhiwei&#160;Chen</a><sup>1</sup>,
22
+ Shiqi&#160;Zhang<sup>1</sup>,
23
+ <a target="_blank" href="https://windlikeo.github.io/HQL.github.io/">Qinlei&#160;Huang</a><sup>1</sup>,
24
+ <a target="_blank" href="https://zhihfu.github.io/">Zhiheng&#160;Fu</a><sup>1</sup>,
25
+ <a target="_blank" href="https://faculty.sdu.edu.cn/weiyinwei1/zh_CN/index.htm">Yinwei&#160;Wei</a><sup>1</sup>
26
+ </div>
27
+ <sup>1</sup>School of Software, Shandong University &#160&#160&#160</span>
28
+ <br />
29
+ <sup>&#9993&#160;</sup>Corresponding author&#160;&#160;</span>
30
+ <br/>
31
+ <p>
32
+ <a href="https://aaai.org/Conferences/AAAI-26/"><img src="https://img.shields.io/badge/AAAI-2026-blue.svg?style=flat-square" alt="AAAI 2026"></a>
33
+ <a href="https://ojs.aaai.org/index.php/AAAI/article/view/37608"><img alt='Paper' src="https://img.shields.io/badge/Paper-AAAI.37608-green.svg"></a>
34
+ <a href="https://lee-zixu.github.io/HABIT.github.io/"><img alt='Project Page' src="https://img.shields.io/badge/Website-orange"></a>
35
+ <a href="https://github.com/Lee-zixu/HABIT"><img alt='GitHub' src="https://img.shields.io/badge/GitHub-Repository-black?style=flat-square&logo=github"></a>
36
+ </p>
37
+ </div>
38
+
39
+ This repository hosts the official pre-trained checkpoints for **HABIT**, a highly robust progressive learning framework designed to tackle the Noise Triplet Correspondence (NTC) problem in Composed Image Retrieval (CIR).
40
+
41
+ ---
42
+
43
+ ## πŸ“Œ Model Information
44
+
45
+ ### 1. Model Name
46
+ **HABIT** (cHrono-synergiA roBust progressIve learning framework for composed image reTrieval) Checkpoints.
47
+
48
+ ### 2. Task Type & Applicable Tasks
49
+ - **Task Type:** Composed Image Retrieval (CIR) / Vision-Language Retrieval.
50
+ - **Applicable Tasks:** Retrieving target images based on a reference image and a modification text. These weights are specifically robust against noisy training data (Noise Triplet Correspondence).
51
+
52
+ ### 3. Project Introduction
53
+ Existing Composed Image Retrieval (CIR) methods often suffer from the "Noise Triplet Correspondence (NTC)" problem in real-world scenarios, struggling to precisely estimate composed semantic discrepancies. **HABIT** effectively addresses this through:
54
+ - 🧠 **Mutual Knowledge Estimation (MKE):** Quantifies sample cleanliness by computing the transition rate of mutual knowledge.
55
+ - ⏳ **Dual-consistency Progressive Learning (DPL):** A collaborative mechanism between historical and current models to simulate human habit formation (retaining good habits, calibrating bad ones).
56
+
57
+ Based on the BLIP-2 architecture, HABIT maintains State-of-the-Art (SOTA) retrieval performance under various noise ratios.
58
+
59
+ ### 4. Training Data Source & Hosted Weights
60
+ The models were trained on the **FashionIQ** and **CIRR** datasets under varying simulated noise ratios ($N \in \{0.2, 0.5, 0.8\}$). This Hugging Face repository provides the corresponding `.pt` checkpoint files organized by dataset:
61
+
62
+ * πŸ“‚ `fiq/`
63
+ * `HABIT-FIQ_N0.2.pt` (Trained on FashionIQ with 20% noise)
64
+ * `HABIT-FIQ_N0.5.pt` (Trained on FashionIQ with 50% noise)
65
+ * `HABIT-FIQ_N0.8.pt` (Trained on FashionIQ with 80% noise)
66
+ * πŸ“‚ `cirr/`
67
+ * `HABIT-CIRR_N0.2.pt` (Trained on CIRR with 20% noise)
68
+ * `HABIT-CIRR_N0.5.pt` (Trained on CIRR with 50% noise)
69
+ * `HABIT-CIRR_N0.8.pt` (Trained on CIRR with 80% noise)
70
+
71
+ ---
72
+
73
+ ## πŸš€ Usage & Basic Inference
74
+
75
+ These weights are designed to be evaluated seamlessly using the official [HABIT GitHub repository](https://github.com/Lee-zixu/HABIT).
76
+
77
+ ### Step 1: Prepare the Environment
78
+ Clone the GitHub repository and install dependencies:
79
+ ```bash
80
+ git clone [https://github.com/Lee-zixu/HABIT](https://github.com/Lee-zixu/HABIT)
81
+ cd HABIT
82
+ conda create -n habit python=3.8 -y
83
+ conda activate habit
84
+ pip install torch==2.1.0 torchvision==0.16.0 torchaudio==2.1.0 --index-url [https://download.pytorch.org/whl/cu121](https://download.pytorch.org/whl/cu121)
85
+ pip install open-clip-torch==2.24.0 scikit-learn==1.3.2 transformers==4.25.0 salesforce-lavis==1.0.2 timm==0.9.16
86
+ ```
87
+
88
+ ### Step 2: Download Model Weights
89
+ Download the specific `.pt` files you wish to evaluate from this Hugging Face repository. Place them into a `checkpoints/` directory within your cloned GitHub repo. For example, to evaluate the CIRR model trained with 50% noise:
90
+
91
+ ```text
92
+ HABIT/
93
+ └── checkpoints/
94
+ └── cirr_noise0.5/
95
+ └── HABIT-CIRR_N0.5.pt <-- (Rename to best_model.pt if required by your specific test script)
96
+ ```
97
+
98
+ ### Step 3: Run Testing / Evaluation
99
+ To generate prediction files on the CIRR dataset for the [CIRR Evaluation Server](https://cirr.cecs.anu.edu.au/), point the test script to the directory containing your downloaded checkpoint:
100
+
101
+ ```bash
102
+ # Example for testing the CIRR 50% noise model
103
+ python src/cirr_test_submission.py checkpoints/cirr_noise0.5/
104
+ ```
105
+ *(The script will automatically output `.json` files based on the checkpoint for online evaluation.)*
106
+
107
+ ---
108
+
109
+ ## ⚠️ Limitations & Notes
110
+
111
+ - **Hardware Requirements:** Because HABIT is built upon the powerful BLIP-2 architecture, inference and further fine-tuning require GPUs with sufficient memory (e.g., NVIDIA A40 48G / V100 32G is recommended).
112
+ - **Intended Use:** These weights are provided for academic research and to facilitate reproducibility of the AAAI 2026 paper.
113
+
114
+ ---
115
+
116
+ ## πŸ“β­οΈ Citation
117
+
118
+ If you find our work, code, or these model weights useful in your research, please consider leaving a **Star** ⭐️ on our GitHub repository and citing our paper:
119
+
120
+ ```bibtex
121
+ @inproceedings{HABIT,
122
+ title={HABIT: Chrono-Synergia Robust Progressive Learning Framework for Composed Image Retrieval},
123
+ author={Li, Zixu and Hu, Yupeng and Chen, Zhiwei and Zhang, Shiqi and Huang, Qinlei and Fu, Zhiheng and Wei, Yinwei},
124
+ booktitle={Proceedings of the AAAI Conference on Artificial Intelligence},
125
+ year={2026}
126
+ }
127
+ ```
128
+ ```