| --- |
| license: mit |
| base_model: openai/clip-vit-base-patch32 |
| tags: |
| - clip |
| - zero-shot-image-classification |
| - vision |
| - hierarcaps |
| pipeline_tag: zero-shot-image-classification |
| --- |
| |
| # HierarCaps CLIP-B |
|
|
| Fine-tuned CLIP-B (ViT-B/32) checkpoint from the paper **[Emergent Visual-Semantic Hierarchies in Image-Text Representations](https://arxiv.org/abs/2407.08521)** (ECCV 2024). |
|
|
| - **Paper:** https://arxiv.org/abs/2407.08521 |
| - **Project page:** https://hierarcaps.github.io/ |
| - **Code:** https://github.com/TAU-VAILab/hierarcaps |
| - **Base model:** [openai/clip-vit-base-patch32](https://huggingface.co/openai/clip-vit-base-patch32) |
|
|
| This model was fine-tuned on the HierarCaps dataset to enhance hierarchical organization in CLIP's embedding space, improving hierarchical reasoning about specificity between images and texts. |
|
|
| ## Usage |
|
|
| ```python |
| from transformers import CLIPModel, AutoProcessor |
| |
| model = CLIPModel.from_pretrained("malper/hierarcaps-clip-b") |
| processor = AutoProcessor.from_pretrained("malper/hierarcaps-clip-b") |
| ``` |
|
|
| See the [official code repository](https://github.com/TAU-VAILab/hierarcaps) for training and evaluation scripts used with this checkpoint. |
|
|
| ## Citation |
|
|
| ```bibtex |
| @InProceedings{alper2024hierarcaps, |
| author = {Morris Alper and Hadar Averbuch-Elor}, |
| title = {Emergent Visual-Semantic Hierarchies in Image-Text Representations}, |
| booktitle = {Proceedings of the European Conference on Computer Vision (ECCV)}, |
| year = {2024} |
| } |
| ``` |
|
|
