Instructions to use nmndeep/CLIC-ViT-B-32-224-PixPr-RedCaps with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- OpenCLIP
How to use nmndeep/CLIC-ViT-B-32-224-PixPr-RedCaps with OpenCLIP:
import open_clip model, preprocess_train, preprocess_val = open_clip.create_model_and_transforms('hf-hub:nmndeep/CLIC-ViT-B-32-224-PixPr-RedCaps') tokenizer = open_clip.get_tokenizer('hf-hub:nmndeep/CLIC-ViT-B-32-224-PixPr-RedCaps') - Notebooks
- Google Colab
- Kaggle
Enhance model card with metadata, abstract, links, and citation
Browse filesThis PR significantly enhances the model card for CLIC-ViT-B-32-224-PixPr-RedCaps by:
- Adding essential metadata: `license: apache-2.0`, `library_name: open_clip`, and `pipeline_tag: zero-shot-image-classification`. These additions will improve the model's discoverability on the Hugging Face Hub and enable the automated `open_clip` usage widget.
- Providing the paper's abstract for comprehensive context about the model.
- Including direct links to the paper ([Advancing Compositional Awareness in CLIP with Efficient Fine-Tuning](https://huggingface.co/papers/2505.24424)), the project page (https://clic-compositional-clip.github.io/), and the GitHub repository (https://github.com/AmitPeleg/CLIC).
- Incorporating the BibTeX citation for proper academic attribution.
- Correcting the sample usage snippet by explicitly importing `urlopen` to ensure it runs without errors.
Please review these improvements.
|
@@ -1,6 +1,17 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
|
| 2 |
# Model Card for CLIC-ViT-B-32-224-PixPr-RedCaps
|
| 3 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 4 |
## Model Details
|
| 5 |
|
| 6 |
<!-- Provide the basic links for the model. -->
|
|
@@ -10,10 +21,12 @@
|
|
| 10 |
## Model Usage
|
| 11 |
### With OpenCLIP
|
| 12 |
|
| 13 |
-
```
|
| 14 |
import torch
|
| 15 |
from PIL import Image
|
| 16 |
import open_clip
|
|
|
|
|
|
|
| 17 |
|
| 18 |
model, _, image_processor = open_clip.create_model_and_transforms('hf-hub:nmndeep/CLIC-ViT-B-32-224-PixPr-RedCaps')
|
| 19 |
|
|
@@ -38,5 +51,16 @@ with torch.no_grad(), torch.autocast("cuda"):
|
|
| 38 |
|
| 39 |
text_probs = (100.0 * image_features @ text_features.T).softmax(dim=-1)
|
| 40 |
idx = torch.argmax(text_probs)
|
| 41 |
-
print("Output label:", texts[idx])
|
| 42 |
-
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
pipeline_tag: zero-shot-image-classification
|
| 4 |
+
library_name: open_clip
|
| 5 |
+
---
|
| 6 |
|
| 7 |
# Model Card for CLIC-ViT-B-32-224-PixPr-RedCaps
|
| 8 |
|
| 9 |
+
**Advancing Compositional Awareness in CLIP with Efficient Fine-Tuning**
|
| 10 |
+
|
| 11 |
+
[[Paper](https://huggingface.co/papers/2505.24424)] | [[Project Page](https://clic-compositional-clip.github.io/)] | [[Code](https://github.com/AmitPeleg/CLIC)]
|
| 12 |
+
|
| 13 |
+
Vision-language models like CLIP have demonstrated remarkable zero-shot capabilities in classification and retrieval. However, these models often struggle with compositional reasoning - the ability to understand the relationships between concepts. A recent benchmark, SugarCrepe++, reveals that previous works on improving compositionality have mainly improved lexical sensitivity but neglected semantic understanding. In addition, downstream retrieval performance often deteriorates, although one would expect that improving compositionality should enhance retrieval. In this work, we introduce CLIC (Compositionally-aware Learning in CLIP), a fine-tuning method based on a novel training technique combining multiple images and their associated captions. CLIC improves compositionality across architectures as well as differently pre-trained CLIP models, both in terms of lexical and semantic understanding, and achieves consistent gains in retrieval performance. This even applies to the recent CLIPS, which achieves SOTA retrieval performance. Nevertheless, the short fine-tuning with CLIC leads to an improvement in retrieval and to the best compositional CLIP model on SugarCrepe++. All our models and code are available.
|
| 14 |
+
|
| 15 |
## Model Details
|
| 16 |
|
| 17 |
<!-- Provide the basic links for the model. -->
|
|
|
|
| 21 |
## Model Usage
|
| 22 |
### With OpenCLIP
|
| 23 |
|
| 24 |
+
```python
|
| 25 |
import torch
|
| 26 |
from PIL import Image
|
| 27 |
import open_clip
|
| 28 |
+
from urllib.request import urlopen
|
| 29 |
+
|
| 30 |
|
| 31 |
model, _, image_processor = open_clip.create_model_and_transforms('hf-hub:nmndeep/CLIC-ViT-B-32-224-PixPr-RedCaps')
|
| 32 |
|
|
|
|
| 51 |
|
| 52 |
text_probs = (100.0 * image_features @ text_features.T).softmax(dim=-1)
|
| 53 |
idx = torch.argmax(text_probs)
|
| 54 |
+
print("Output label:", texts[idx])
|
| 55 |
+
```
|
| 56 |
+
|
| 57 |
+
## Citation
|
| 58 |
+
If you find this repository useful, please consider citing our paper:
|
| 59 |
+
```bibtex
|
| 60 |
+
@inproceedings{peleg2025advancing,
|
| 61 |
+
title={Advancing Compositional Awareness in CLIP with Efficient Fine-Tuning},
|
| 62 |
+
author={Peleg, Amit and Singh, Naman Deep and Hein, Matthias},
|
| 63 |
+
booktitle = {NeurIPS},
|
| 64 |
+
year = {2025}
|
| 65 |
+
}
|
| 66 |
+
```
|