|
|
--- |
|
|
license: mit |
|
|
library_name: open_clip |
|
|
tags: |
|
|
- image-text-retrieval |
|
|
- retrieval |
|
|
- openclip |
|
|
- vit |
|
|
- icar |
|
|
--- |
|
|
|
|
|
# ICAR Weights (OpenCLIP ViT-L-14, LAION-2B) |
|
|
|
|
|
This repository provides ICAR fine-tuned OpenCLIP ViT-L-14 (LAION-2B) weights, |
|
|
including early-exit variants, used in the paper "Image Complexity-Aware Adaptive |
|
|
Retrieval for Efficient Vision-Language Models". |
|
|
|
|
|
Paper: https://arxiv.org/abs/2512.15372 |
|
|
Code: https://github.com/MikelWL/ICAR |
|
|
|
|
|
These weights are intended to be used with the ICAR codebase. The standard |
|
|
retrieval setting reported in the paper evaluates COCO/Flickr test queries |
|
|
against a gallery augmented with 100k LAION-COCO distractors. |
|
|
|
|
|
## Contents |
|
|
|
|
|
All weights are stored under the `checkpoints/` directory to match the ICAR |
|
|
codebase layout. |
|
|
|
|
|
### Baselines (full path) |
|
|
|
|
|
- `checkpoints/baseline_coco/latest_checkpoint.pt` |
|
|
- `checkpoints/baseline_flickr/latest_checkpoint.pt` |
|
|
|
|
|
### Early-exit variants (ICAR) |
|
|
|
|
|
- `checkpoints/icar_coco/layer_8/latest_checkpoint.pt` |
|
|
- `checkpoints/icar_coco/layer_12/latest_checkpoint.pt` |
|
|
- `checkpoints/icar_coco/layer_16/latest_checkpoint.pt` |
|
|
- `checkpoints/icar_coco/layer_20/latest_checkpoint.pt` |
|
|
- `checkpoints/icar_flickr/layer_8/latest_checkpoint.pt` |
|
|
- `checkpoints/icar_flickr/layer_12/latest_checkpoint.pt` |
|
|
- `checkpoints/icar_flickr/layer_16/latest_checkpoint.pt` |
|
|
- `checkpoints/icar_flickr/layer_20/latest_checkpoint.pt` |
|
|
|
|
|
## Usage |
|
|
|
|
|
Use the ICAR scripts as the primary entrypoints. See the main repo README for |
|
|
reference scripts and required data paths. |
|
|
|
|
|
Example (mixed evaluation): |
|
|
|
|
|
```bash |
|
|
python scripts/evaluate_mixed_preprocessed.py \ |
|
|
--config icar/configs/coco.yaml \ |
|
|
--checkpoint checkpoints/icar_coco/layer_12/latest_checkpoint.pt \ |
|
|
--base-dataset mscoco \ |
|
|
--base-data-root /path/to/coco-images \ |
|
|
--laion-data-root /path/to/laion_coco_100k \ |
|
|
--complexity-scores /path/to/laion_coco_100k_metadata/complexity_scores.json \ |
|
|
--early-exit-layer 12 \ |
|
|
--use-icc-routing \ |
|
|
--icc-checkpoint /path/to/icc_weights.pt \ |
|
|
--eval-category |
|
|
``` |
|
|
|
|
|
## ICC Weights |
|
|
|
|
|
ICC (Image Complexity Classifier) weights are hosted separately due to |
|
|
non-commercial licensing restrictions on the ImageNet-pretrained ConvNeXt-V2 |
|
|
checkpoint. Please download ICC weights from the dedicated ICC repository. |
|
|
|
|
|
## License |
|
|
|
|
|
The ICAR weights in this repository follow the OpenCLIP license (MIT). |
|
|
See the OpenCLIP repository for details and attribution requirements. |
|
|
|
