metadata
license: mit
base_model: openai/clip-vit-base-patch32
tags:
- clip
- zero-shot-image-classification
- vision
- hierarcaps
pipeline_tag: zero-shot-image-classification
HierarCaps CLIP-B
Fine-tuned CLIP-B (ViT-B/32) checkpoint from the paper Emergent Visual-Semantic Hierarchies in Image-Text Representations (ECCV 2024).
- Paper: https://arxiv.org/abs/2407.08521
- Project page: https://hierarcaps.github.io/
- Code: https://github.com/TAU-VAILab/hierarcaps
- Base model: openai/clip-vit-base-patch32
This model was fine-tuned on the HierarCaps dataset to enhance hierarchical organization in CLIP's embedding space, improving hierarchical reasoning about specificity between images and texts.
Usage
from transformers import CLIPModel, AutoProcessor
model = CLIPModel.from_pretrained("malper/hierarcaps-clip-b")
processor = AutoProcessor.from_pretrained("malper/hierarcaps-clip-b")
See the official code repository for training and evaluation scripts used with this checkpoint.
Citation
@InProceedings{alper2024hierarcaps,
author = {Morris Alper and Hadar Averbuch-Elor},
title = {Emergent Visual-Semantic Hierarchies in Image-Text Representations},
booktitle = {Proceedings of the European Conference on Computer Vision (ECCV)},
year = {2024}
}