---
license: mit
base_model: openai/clip-vit-base-patch32
tags:
- clip
- zero-shot-image-classification
- vision
- hierarcaps
pipeline_tag: zero-shot-image-classification
---

# HierarCaps CLIP-B

Fine-tuned CLIP-B (ViT-B/32) checkpoint from the paper **[Emergent Visual-Semantic Hierarchies in Image-Text Representations](https://arxiv.org/abs/2407.08521)** (ECCV 2024).

- **Paper:** https://arxiv.org/abs/2407.08521
- **Project page:** https://hierarcaps.github.io/
- **Code:** https://github.com/TAU-VAILab/hierarcaps
- **Base model:** [openai/clip-vit-base-patch32](https://huggingface.co/openai/clip-vit-base-patch32)

This model was fine-tuned on the HierarCaps dataset to enhance hierarchical organization in CLIP's embedding space, improving hierarchical reasoning about specificity between images and texts.

## Usage

```python
from transformers import CLIPModel, AutoProcessor

model = CLIPModel.from_pretrained("malper/hierarcaps-clip-b")
processor = AutoProcessor.from_pretrained("malper/hierarcaps-clip-b")
```

See the [official code repository](https://github.com/TAU-VAILab/hierarcaps) for training and evaluation scripts used with this checkpoint.

## Citation

```bibtex
@InProceedings{alper2024hierarcaps,
  author    = {Morris Alper and Hadar Averbuch-Elor},
  title     = {Emergent Visual-Semantic Hierarchies in Image-Text Representations},
  booktitle = {Proceedings of the European Conference on Computer Vision (ECCV)},
  year      = {2024}
}
```