nielsr HF Staff commited on
Commit
ea5cf03
·
verified ·
1 Parent(s): ba0bec8

Enhance model card with metadata, abstract, links, and citation

Browse files

This PR significantly enhances the model card for CLIC-ViT-B-32-224-PixPr-RedCaps by:
- Adding essential metadata: `license: apache-2.0`, `library_name: open_clip`, and `pipeline_tag: zero-shot-image-classification`. These additions will improve the model's discoverability on the Hugging Face Hub and enable the automated `open_clip` usage widget.
- Providing the paper's abstract for comprehensive context about the model.
- Including direct links to the paper ([Advancing Compositional Awareness in CLIP with Efficient Fine-Tuning](https://huggingface.co/papers/2505.24424)), the project page (https://clic-compositional-clip.github.io/), and the GitHub repository (https://github.com/AmitPeleg/CLIC).
- Incorporating the BibTeX citation for proper academic attribution.
- Correcting the sample usage snippet by explicitly importing `urlopen` to ensure it runs without errors.

Please review these improvements.

Files changed (1) hide show
  1. README.md +27 -3
README.md CHANGED
@@ -1,6 +1,17 @@
 
 
 
 
 
1
 
2
  # Model Card for CLIC-ViT-B-32-224-PixPr-RedCaps
3
 
 
 
 
 
 
 
4
  ## Model Details
5
 
6
  <!-- Provide the basic links for the model. -->
@@ -10,10 +21,12 @@
10
  ## Model Usage
11
  ### With OpenCLIP
12
 
13
- ```
14
  import torch
15
  from PIL import Image
16
  import open_clip
 
 
17
 
18
  model, _, image_processor = open_clip.create_model_and_transforms('hf-hub:nmndeep/CLIC-ViT-B-32-224-PixPr-RedCaps')
19
 
@@ -38,5 +51,16 @@ with torch.no_grad(), torch.autocast("cuda"):
38
 
39
  text_probs = (100.0 * image_features @ text_features.T).softmax(dim=-1)
40
  idx = torch.argmax(text_probs)
41
- print("Output label:", texts[idx])
42
- ```
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ pipeline_tag: zero-shot-image-classification
4
+ library_name: open_clip
5
+ ---
6
 
7
  # Model Card for CLIC-ViT-B-32-224-PixPr-RedCaps
8
 
9
+ **Advancing Compositional Awareness in CLIP with Efficient Fine-Tuning**
10
+
11
+ [[Paper](https://huggingface.co/papers/2505.24424)] | [[Project Page](https://clic-compositional-clip.github.io/)] | [[Code](https://github.com/AmitPeleg/CLIC)]
12
+
13
+ Vision-language models like CLIP have demonstrated remarkable zero-shot capabilities in classification and retrieval. However, these models often struggle with compositional reasoning - the ability to understand the relationships between concepts. A recent benchmark, SugarCrepe++, reveals that previous works on improving compositionality have mainly improved lexical sensitivity but neglected semantic understanding. In addition, downstream retrieval performance often deteriorates, although one would expect that improving compositionality should enhance retrieval. In this work, we introduce CLIC (Compositionally-aware Learning in CLIP), a fine-tuning method based on a novel training technique combining multiple images and their associated captions. CLIC improves compositionality across architectures as well as differently pre-trained CLIP models, both in terms of lexical and semantic understanding, and achieves consistent gains in retrieval performance. This even applies to the recent CLIPS, which achieves SOTA retrieval performance. Nevertheless, the short fine-tuning with CLIC leads to an improvement in retrieval and to the best compositional CLIP model on SugarCrepe++. All our models and code are available.
14
+
15
  ## Model Details
16
 
17
  <!-- Provide the basic links for the model. -->
 
21
  ## Model Usage
22
  ### With OpenCLIP
23
 
24
+ ```python
25
  import torch
26
  from PIL import Image
27
  import open_clip
28
+ from urllib.request import urlopen
29
+
30
 
31
  model, _, image_processor = open_clip.create_model_and_transforms('hf-hub:nmndeep/CLIC-ViT-B-32-224-PixPr-RedCaps')
32
 
 
51
 
52
  text_probs = (100.0 * image_features @ text_features.T).softmax(dim=-1)
53
  idx = torch.argmax(text_probs)
54
+ print("Output label:", texts[idx])
55
+ ```
56
+
57
+ ## Citation
58
+ If you find this repository useful, please consider citing our paper:
59
+ ```bibtex
60
+ @inproceedings{peleg2025advancing,
61
+ title={Advancing Compositional Awareness in CLIP with Efficient Fine-Tuning},
62
+ author={Peleg, Amit and Singh, Naman Deep and Hein, Matthias},
63
+ booktitle = {NeurIPS},
64
+ year = {2025}
65
+ }
66
+ ```