Zero-Shot Image Classification
Transformers
Safetensors
English
clip
fashion
multimodal
image-search
text-search
embeddings
contrastive-learning
zero-shot-classification
Instructions to use Leacb4/gap-clip with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Leacb4/gap-clip with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("zero-shot-image-classification", model="Leacb4/gap-clip") pipe( "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/hub/parrots.png", candidate_labels=["animals", "humans", "landscape"], )# Load model directly from transformers import AutoProcessor, AutoModelForZeroShotImageClassification processor = AutoProcessor.from_pretrained("Leacb4/gap-clip") model = AutoModelForZeroShotImageClassification.from_pretrained("Leacb4/gap-clip") - Notebooks
- Google Colab
- Kaggle
| *.7z filter=lfs diff=lfs merge=lfs -text | |
| *.arrow filter=lfs diff=lfs merge=lfs -text | |
| *.bin filter=lfs diff=lfs merge=lfs -text | |
| *.bz2 filter=lfs diff=lfs merge=lfs -text | |
| *.ckpt filter=lfs diff=lfs merge=lfs -text | |
| *.ftz filter=lfs diff=lfs merge=lfs -text | |
| *.gz filter=lfs diff=lfs merge=lfs -text | |
| *.h5 filter=lfs diff=lfs merge=lfs -text | |
| *.joblib filter=lfs diff=lfs merge=lfs -text | |
| *.lfs.* filter=lfs diff=lfs merge=lfs -text | |
| *.mlmodel filter=lfs diff=lfs merge=lfs -text | |
| *.model filter=lfs diff=lfs merge=lfs -text | |
| *.msgpack filter=lfs diff=lfs merge=lfs -text | |
| *.npy filter=lfs diff=lfs merge=lfs -text | |
| *.npz filter=lfs diff=lfs merge=lfs -text | |
| *.onnx filter=lfs diff=lfs merge=lfs -text | |
| *.ot filter=lfs diff=lfs merge=lfs -text | |
| *.parquet filter=lfs diff=lfs merge=lfs -text | |
| *.pb filter=lfs diff=lfs merge=lfs -text | |
| *.pickle filter=lfs diff=lfs merge=lfs -text | |
| *.pkl filter=lfs diff=lfs merge=lfs -text | |
| *.pt filter=lfs diff=lfs merge=lfs -text | |
| *.pth filter=lfs diff=lfs merge=lfs -text | |
| *.rar filter=lfs diff=lfs merge=lfs -text | |
| *.safetensors filter=lfs diff=lfs merge=lfs -text | |
| saved_model/**/* filter=lfs diff=lfs merge=lfs -text | |
| *.tar.* filter=lfs diff=lfs merge=lfs -text | |
| *.tar filter=lfs diff=lfs merge=lfs -text | |
| *.tflite filter=lfs diff=lfs merge=lfs -text | |
| *.tgz filter=lfs diff=lfs merge=lfs -text | |
| *.wasm filter=lfs diff=lfs merge=lfs -text | |
| *.xz filter=lfs diff=lfs merge=lfs -text | |
| *.zip filter=lfs diff=lfs merge=lfs -text | |
| *.zst filter=lfs diff=lfs merge=lfs -text | |
| *tfevents* filter=lfs diff=lfs merge=lfs -text | |
| figures/baseline_blue_pant.png filter=lfs diff=lfs merge=lfs -text | |
| figures/baseline_red_dress.png filter=lfs diff=lfs merge=lfs -text | |
| figures/confusion_matrices/cm_color/kaggle_baseline_image_color_confusion_matrix.png filter=lfs diff=lfs merge=lfs -text | |
| figures/confusion_matrices/cm_color/kaggle_baseline_text_color_confusion_matrix.png filter=lfs diff=lfs merge=lfs -text | |
| figures/confusion_matrices/cm_color/kaggle_image_color_confusion_matrix.png filter=lfs diff=lfs merge=lfs -text | |
| figures/confusion_matrices/cm_color/kaggle_text_color_confusion_matrix.png filter=lfs diff=lfs merge=lfs -text | |
| figures/confusion_matrices/cm_color/local_baseline_image_color_confusion_matrix.png filter=lfs diff=lfs merge=lfs -text | |
| figures/confusion_matrices/cm_color/local_baseline_text_color_confusion_matrix.png filter=lfs diff=lfs merge=lfs -text | |
| figures/confusion_matrices/cm_color/local_image_color_confusion_matrix.png filter=lfs diff=lfs merge=lfs -text | |
| figures/confusion_matrices/cm_color/local_text_color_confusion_matrix.png filter=lfs diff=lfs merge=lfs -text | |
| figures/confusion_matrices/cm_hierarchy/baseline_image_hierarchy_confusion_matrix.png filter=lfs diff=lfs merge=lfs -text | |
| figures/confusion_matrices/cm_hierarchy/baseline_internal_image_hierarchy_confusion_matrix.png filter=lfs diff=lfs merge=lfs -text | |
| figures/confusion_matrices/cm_hierarchy/baseline_internal_text_hierarchy_confusion_matrix.png filter=lfs diff=lfs merge=lfs -text | |
| figures/confusion_matrices/cm_hierarchy/baseline_kagl_marqo_image_hierarchy_confusion_matrix.png filter=lfs diff=lfs merge=lfs -text | |
| figures/confusion_matrices/cm_hierarchy/baseline_kagl_marqo_text_hierarchy_confusion_matrix.png filter=lfs diff=lfs merge=lfs -text | |
| figures/confusion_matrices/cm_hierarchy/baseline_text_hierarchy_confusion_matrix.png filter=lfs diff=lfs merge=lfs -text | |
| figures/confusion_matrices/cm_hierarchy/gap_clip_image_hierarchy_confusion_matrix.png filter=lfs diff=lfs merge=lfs -text | |
| figures/confusion_matrices/cm_hierarchy/gap_clip_internal_image_hierarchy_confusion_matrix.png filter=lfs diff=lfs merge=lfs -text | |
| figures/confusion_matrices/cm_hierarchy/gap_clip_internal_text_hierarchy_confusion_matrix.png filter=lfs diff=lfs merge=lfs -text | |
| figures/confusion_matrices/cm_hierarchy/gap_clip_kagl_marqo_image_hierarchy_confusion_matrix.png filter=lfs diff=lfs merge=lfs -text | |
| figures/confusion_matrices/cm_hierarchy/gap_clip_kagl_marqo_text_hierarchy_confusion_matrix.png filter=lfs diff=lfs merge=lfs -text | |
| figures/confusion_matrices/cm_hierarchy/gap_clip_text_hierarchy_confusion_matrix.png filter=lfs diff=lfs merge=lfs -text | |
| figures/gapclip_blue_pant.png filter=lfs diff=lfs merge=lfs -text | |
| figures/gapclip_red_dress.png filter=lfs diff=lfs merge=lfs -text | |
| figures/heatmap.png filter=lfs diff=lfs merge=lfs -text | |
| figures/heatmap_baseline.jpg filter=lfs diff=lfs merge=lfs -text | |
| figures/red_dress.png filter=lfs diff=lfs merge=lfs -text | |
| figures/scheme.png filter=lfs diff=lfs merge=lfs -text | |
| figures/training_curves.png filter=lfs diff=lfs merge=lfs -text | |
| figures/tsne_baseline.png filter=lfs diff=lfs merge=lfs -text | |
| figures/tsne_hierarchy_baseline.png filter=lfs diff=lfs merge=lfs -text | |
| figures/tsne_hierarchy_our.png filter=lfs diff=lfs merge=lfs -text | |
| figures/tsne_model.png filter=lfs diff=lfs merge=lfs -text | |
| paper/paper.pdf filter=lfs diff=lfs merge=lfs -text | |
| figures/confusion_matrices/cm_main/fashion_baseline_text_hierarchy_confusion_matrix.png filter=lfs diff=lfs merge=lfs -text | |
| figures/confusion_matrices/cm_main/fashion_image_hierarchy_confusion_matrix.png filter=lfs diff=lfs merge=lfs -text | |
| figures/confusion_matrices/cm_main/fashion_text_hierarchy_confusion_matrix.png filter=lfs diff=lfs merge=lfs -text | |
| figures/confusion_matrices/cm_main/fashion_baseline_image_hierarchy_confusion_matrix.png filter=lfs diff=lfs merge=lfs -text | |
| figures/confusion_matrices/cm_hierarchy/baseline_fashion-mnist_(normalized)_image_hierarchy_confusion_matrix.png filter=lfs diff=lfs merge=lfs -text | |
| figures/confusion_matrices/cm_hierarchy/baseline_fashion-mnist_(normalized)_text_hierarchy_confusion_matrix.png filter=lfs diff=lfs merge=lfs -text | |
| figures/confusion_matrices/cm_hierarchy/baseline_local_(normalized)_image_hierarchy_confusion_matrix.png filter=lfs diff=lfs merge=lfs -text | |
| figures/confusion_matrices/cm_hierarchy/baseline_local_(normalized)_text_hierarchy_confusion_matrix.png filter=lfs diff=lfs merge=lfs -text | |
| figures/confusion_matrices/cm_hierarchy/gap_clip_fashion-mnist_(normalized)_image_hierarchy_confusion_matrix.png filter=lfs diff=lfs merge=lfs -text | |
| figures/confusion_matrices/cm_hierarchy/gap_clip_fashion-mnist_(normalized)_text_hierarchy_confusion_matrix.png filter=lfs diff=lfs merge=lfs -text | |
| figures/confusion_matrices/cm_hierarchy/gap_clip_local_(normalized)_image_hierarchy_confusion_matrix.png filter=lfs diff=lfs merge=lfs -text | |
| figures/confusion_matrices/cm_hierarchy/gap_clip_local_(normalized)_text_hierarchy_confusion_matrix.png filter=lfs diff=lfs merge=lfs -text | |
| figures/confusion_matrices/cm_sec533/baseline_image_hierarchy_confusion_matrix.png filter=lfs diff=lfs merge=lfs -text | |
| figures/confusion_matrices/cm_sec533/baseline_internal_image_color_confusion_matrix.png filter=lfs diff=lfs merge=lfs -text | |
| figures/confusion_matrices/cm_sec533/baseline_internal_image_hierarchy_confusion_matrix.png filter=lfs diff=lfs merge=lfs -text | |
| figures/confusion_matrices/cm_sec533/baseline_internal_text_color_confusion_matrix.png filter=lfs diff=lfs merge=lfs -text | |
| figures/confusion_matrices/cm_sec533/baseline_internal_text_hierarchy_confusion_matrix.png filter=lfs diff=lfs merge=lfs -text | |
| figures/confusion_matrices/cm_sec533/baseline_kagl_marqo_image_color_confusion_matrix.png filter=lfs diff=lfs merge=lfs -text | |
| figures/confusion_matrices/cm_sec533/baseline_kagl_marqo_image_hierarchy_confusion_matrix.png filter=lfs diff=lfs merge=lfs -text | |
| figures/confusion_matrices/cm_sec533/baseline_kagl_marqo_text_color_confusion_matrix.png filter=lfs diff=lfs merge=lfs -text | |
| figures/confusion_matrices/cm_sec533/baseline_kagl_marqo_text_hierarchy_confusion_matrix.png filter=lfs diff=lfs merge=lfs -text | |
| figures/confusion_matrices/cm_sec533/baseline_text_hierarchy_confusion_matrix.png filter=lfs diff=lfs merge=lfs -text | |
| figures/confusion_matrices/cm_sec533/gap_clip_image_hierarchy_confusion_matrix.png filter=lfs diff=lfs merge=lfs -text | |
| figures/confusion_matrices/cm_sec533/gap_clip_internal_image_color_confusion_matrix.png filter=lfs diff=lfs merge=lfs -text | |
| figures/confusion_matrices/cm_sec533/gap_clip_internal_image_hierarchy_confusion_matrix.png filter=lfs diff=lfs merge=lfs -text | |
| figures/confusion_matrices/cm_sec533/gap_clip_internal_text_color_confusion_matrix.png filter=lfs diff=lfs merge=lfs -text | |
| figures/confusion_matrices/cm_sec533/gap_clip_internal_text_hierarchy_confusion_matrix.png filter=lfs diff=lfs merge=lfs -text | |
| figures/confusion_matrices/cm_sec533/gap_clip_kagl_marqo_image_color_confusion_matrix.png filter=lfs diff=lfs merge=lfs -text | |
| figures/confusion_matrices/cm_sec533/gap_clip_kagl_marqo_image_hierarchy_confusion_matrix.png filter=lfs diff=lfs merge=lfs -text | |
| figures/confusion_matrices/cm_sec533/gap_clip_kagl_marqo_text_color_confusion_matrix.png filter=lfs diff=lfs merge=lfs -text | |
| figures/confusion_matrices/cm_sec533/gap_clip_kagl_marqo_text_hierarchy_confusion_matrix.png filter=lfs diff=lfs merge=lfs -text | |
| figures/confusion_matrices/cm_sec533/gap_clip_text_hierarchy_confusion_matrix.png filter=lfs diff=lfs merge=lfs -text | |