Instructions to use google/siglip-base-patch16-256 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use google/siglip-base-patch16-256 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("zero-shot-image-classification", model="google/siglip-base-patch16-256") pipe( "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/hub/parrots.png", candidate_labels=["animals", "humans", "landscape"], )# Load model directly from transformers import AutoProcessor, AutoModelForZeroShotImageClassification processor = AutoProcessor.from_pretrained("google/siglip-base-patch16-256") model = AutoModelForZeroShotImageClassification.from_pretrained("google/siglip-base-patch16-256", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Update README.md
Browse files
README.md
CHANGED
|
@@ -4,13 +4,13 @@ license: apache-2.0
|
|
| 4 |
|
| 5 |
# SigLIP (base-sized model)
|
| 6 |
|
| 7 |
-
SigLIP model pre-trained on
|
| 8 |
|
| 9 |
-
Disclaimer: The team releasing
|
| 10 |
|
| 11 |
## Model description
|
| 12 |
|
| 13 |
-
SigLIP is [CLIP](https://huggingface.co/docs/transformers/model_doc/clip) with a better loss function. The sigmoid loss operates solely on image-text pairs and does not require a global view of the pairwise similarities for normalization. This allows further scaling up the batch size, while also performing better at smaller batch sizes.
|
| 14 |
|
| 15 |
## Intended uses & limitations
|
| 16 |
|
|
@@ -27,8 +27,8 @@ import requests
|
|
| 27 |
from transformers import AutoProcessor, AutoModel
|
| 28 |
import torch
|
| 29 |
|
| 30 |
-
model = AutoModel.from_pretrained("
|
| 31 |
-
processor = AutoProcessor.from_pretrained("
|
| 32 |
|
| 33 |
url = "http://images.cocodataset.org/val2017/000000039769.jpg"
|
| 34 |
image = Image.open(requests.get(url, stream=True).raw)
|
|
|
|
| 4 |
|
| 5 |
# SigLIP (base-sized model)
|
| 6 |
|
| 7 |
+
SigLIP model pre-trained on WebLi at resolution 256x256. It was introduced in the paper [Sigmoid Loss for Language Image Pre-Training](https://arxiv.org/abs/2303.15343) by Zhai et al. and first released in [this repository](https://github.com/google-research/big_vision).
|
| 8 |
|
| 9 |
+
Disclaimer: The team releasing SigLIP did not write a model card for this model so this model card has been written by the Hugging Face team.
|
| 10 |
|
| 11 |
## Model description
|
| 12 |
|
| 13 |
+
SigLIP is [CLIP](https://huggingface.co/docs/transformers/model_doc/clip), a multimodal model, with a better loss function. The sigmoid loss operates solely on image-text pairs and does not require a global view of the pairwise similarities for normalization. This allows further scaling up the batch size, while also performing better at smaller batch sizes.
|
| 14 |
|
| 15 |
## Intended uses & limitations
|
| 16 |
|
|
|
|
| 27 |
from transformers import AutoProcessor, AutoModel
|
| 28 |
import torch
|
| 29 |
|
| 30 |
+
model = AutoModel.from_pretrained("google/siglip-base-patch16-256")
|
| 31 |
+
processor = AutoProcessor.from_pretrained("google/siglip-base-patch16-256")
|
| 32 |
|
| 33 |
url = "http://images.cocodataset.org/val2017/000000039769.jpg"
|
| 34 |
image = Image.open(requests.get(url, stream=True).raw)
|