Lee-zixu commited on
Commit
cecc32d
Β·
verified Β·
1 Parent(s): 36139d8

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +126 -3
README.md CHANGED
@@ -1,3 +1,126 @@
1
- ---
2
- license: apache-2.0
3
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+
2
+ ---
3
+ license: apache-2.0
4
+ task_categories:
5
+ - image-retrieval
6
+ - vision-language-navigation
7
+ tags:
8
+ - composed-image-retrieval
9
+ - robust-learning
10
+ - optimal-transport
11
+ - blip-2
12
+ - cvpr-2026
13
+ ---
14
+
15
+ <a id="top"></a>
16
+ <div align="center">
17
+ <h1>(CVPR 2026) ConeSep: Cone-based Robust Noise-Unlearning Compositional Network for CIR (Model Weights)</h1>
18
+ <div>
19
+ <a target="_blank" href="https://lee-zixu.github.io/">Zixu&#160;Li</a><sup>1</sup>,
20
+ <a target="_blank" href="https://faculty.sdu.edu.cn/huyupeng1/zh_CN/index.htm">Yupeng&#160;Hu</a><sup>1&#9993</sup>,
21
+ <a target="_blank" href="https://zivchen-ty.github.io/">Zhiwei&#160;Chen</a><sup>1</sup>,
22
+ <a target="_blank" href="https://zh-mingyu.github.io/">Mingyu&#160;Zhang</a><sup>1</sup>,
23
+ <a target="_blank" href="https://zhihfu.github.io/">Zhiheng&#160;Fu</a><sup>1</sup>,
24
+ <a target="_blank" href="https://liqiangnie.github.io">Liqiang&#160;Nie</a><sup>2</sup>
25
+ </div>
26
+ <sup>1</sup>School of Software, Shandong University &#160&#160&#160</span> <br>
27
+ <sup>2</sup>School of Computer Science and Technology, Harbin Institute of Technology (Shenzhen), &#160&#160&#160</span>
28
+ <br />
29
+ <sup>&#9993&#160;</sup>Corresponding author&#160;&#160;</span>
30
+ <br/>
31
+ <p>
32
+ <a href="https://cvpr.thecvf.com/"><img src="https://img.shields.io/badge/CVPR-2026-blue.svg?style=flat-square" alt="CVPR 2026"></a>
33
+ <a href="https://arxiv.org/abs/coming soon"><img alt='arXiv' src="https://img.shields.io/badge/arXiv-Coming.Soon-b31b1b.svg"></a>
34
+ <a href="https://lee-zixu.github.io/ConeSep.github.io/"><img alt='Project Page' src="https://img.shields.io/badge/Website-orange"></a>
35
+ <a href="https://github.com/Lee-zixu/ConeSep"><img alt='GitHub' src="https://img.shields.io/badge/GitHub-Repository-black?style=flat-square&logo=github"></a>
36
+ </p>
37
+ </div>
38
+
39
+ This repository hosts the official pre-trained checkpoints for **ConeSep**, a robust noise-unlearning framework that leverages geometric boundary estimation and optimal transport to solve the Noisy Triplet Correspondence (NTC) problem in Composed Image Retrieval (CIR).
40
+
41
+ ---
42
+
43
+ ## πŸ“Œ Model Information
44
+
45
+ ### 1. Model Name
46
+ **ConeSep** (Cone-based robust noisE-unlearning comPositional network) Checkpoints.
47
+
48
+ ### 2. Task Type & Applicable Tasks
49
+ - **Task Type:** Composed Image Retrieval (CIR).
50
+ - **Applicable Tasks:** Retrieving target images based on a reference image and a modification text. These weights provide unmatched robustness under varying degrees of noisy training data (Noise Triplet Correspondence).
51
+
52
+ ### 3. Project Introduction
53
+ Existing Composed Image Retrieval methods struggle with the "Noisy Triplet Correspondence (NTC)" problem, leading to Modality Suppression, Negative Anchor Deficiency, and Unlearning Backlash. **ConeSep** actively perceives, structurally models, and precisely "unlearns" noise through three core modules:
54
+ - πŸ“ **Geometric Fidelity Quantization (GFQ):** Estimates a noise boundary using cone space geometric separability to quantify sample fidelity.
55
+ - πŸ›‘ **Negative Boundary Learning (NBL):** Learns a "diagonal negative combination" for each query as an explicit semantic opposite-anchor.
56
+ - 🎯 **Boundary-based Targeted Unlearning (BTU):** Models noisy correction as an Optimal Transport (OT) problem to execute precise unlearning without backlash on clean samples.
57
+
58
+ ### 4. Training Data Source & Hosted Weights
59
+ The models were trained on the **FashionIQ** and **CIRR** datasets across different simulated noise ratios ($N \in \{0.2, 0.5, 0.8\}$). This Hugging Face repository provides the corresponding `.pt` checkpoint files organized by dataset and noise ratio:
60
+
61
+ * πŸ“‚ `fashioniq/`
62
+ * `ConeSep-FIQ_N0.2.pt` (Trained with 20% noise)
63
+ * `ConeSep-FIQ_N0.5.pt` (Trained with 50% noise)
64
+ * `ConeSep-FIQ_N0.8.pt` (Trained with 80% noise)
65
+ * πŸ“‚ `cirr/`
66
+ * `ConeSep-CIRR_N0.2.pt` (Trained with 20% noise)
67
+ * `ConeSep-CIRR_N0.5.pt` (Trained with 50% noise)
68
+ * `ConeSep-CIRR_N0.8.pt` (Trained with 80% noise)
69
+
70
+ ---
71
+
72
+ ## πŸš€ Usage & Basic Inference
73
+
74
+ These weights are designed to be evaluated out-of-the-box using the official [ConeSep GitHub repository](https://github.com/iLearn-Lab/CVPR26-ConeSep).
75
+
76
+ ### Step 1: Prepare the Environment
77
+ Clone the GitHub repository and set up the environment:
78
+ ```bash
79
+ git clone https://github.com/iLearn-Lab/CVPR26-ConeSep
80
+ cd ConeSep
81
+ conda create -n conesep python=3.8
82
+ conda activate conesep
83
+ pip install torch==2.1.0 torchvision==0.16.0 torchaudio==2.1.0 --index-url [https://download.pytorch.org/whl/cu121](https://download.pytorch.org/whl/cu121)
84
+ pip install scikit-learn==1.3.2 transformers==4.25.0 salesforce-lavis==1.0.2 timm==0.9.16
85
+ ```
86
+
87
+ ### Step 2: Download Model Weights
88
+ Download the specific `.pt` files you need from this Hugging Face repository and place them into a `checkpoints/` directory within your cloned repo. For example, to evaluate the CIRR model trained with 50% noise:
89
+
90
+ ```text
91
+ ConeSep/
92
+ └── checkpoints/
93
+ └── cirr_noise0.5/
94
+ └���─ best_model.pt <-- (Rename the downloaded ConeSep-CIRR_N0.5.pt to best_model.pt)
95
+ ```
96
+
97
+ ### Step 3: Run Testing / Evaluation
98
+ To generate prediction files on the CIRR dataset for the [CIRR Evaluation Server](https://cirr.cecs.anu.edu.au/), run:
99
+
100
+ ```bash
101
+ # Example for testing the CIRR 50% noise model
102
+ python src/cirr_test_submission.py checkpoints/cirr_noise0.5/
103
+ ```
104
+ *(The script will automatically generate the required `.json` files based on the checkpoint for online evaluation.)*
105
+
106
+ ---
107
+
108
+ ## ⚠️ Limitations & Notes
109
+
110
+ - **Hardware Requirements:** ConeSep is built upon the BLIP-2 architecture. It is highly recommended to run inference and training on GPUs with sufficient memory (e.g., NVIDIA A40 48GB or V100 32GB).
111
+ - **Intended Use:** These weights are intended for academic research, robustness evaluation, and reproducing the results reported in the CVPR 2026 paper.
112
+
113
+ ---
114
+
115
+ ## πŸ“β­οΈ Citation
116
+
117
+ If you find our framework, code, or these weights useful in your research, please consider leaving a **Star** ⭐️ on our GitHub repository and citing our CVPR 2026 paper:
118
+
119
+ ```bibtex
120
+ @InProceedings{ConeSep,
121
+ title={ConeSep: Cone-based Robust Noise-Unlearning Compositional Network for Composed Image Retrieval},
122
+ author={Li, Zixu and Hu, Yupeng and Chen, Zhiwei and Zhang, Mingyu and Fu, Zhiheng and Nie, Liqiang},
123
+ booktitle={Proceedings of the Computer Vision and Pattern Recognition Conference (CVPR)},
124
+ year = {2026}
125
+ }
126
+ ```