IMvision12 commited on
Commit
0769b9d
·
verified ·
1 Parent(s): 82924bf

fix readme.md

Browse files
Files changed (1) hide show
  1. README.md +77 -7
README.md CHANGED
@@ -1,22 +1,92 @@
1
  ---
2
  pipeline_tag: zero-shot-image-classification
3
  license: apache-2.0
 
4
  library_name: kerasformers
5
  tags:
6
  - keras
7
  - kerasformers
8
  - siglip
9
- - tf
10
- - jax
 
11
  - pytorch
 
 
12
  ---
13
 
14
- # siglip_base_p16_384 (Keras 3)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
15
 
16
- Pure-Keras 3 weights for [kerasformers](https://github.com/IMvision12/KerasFormers), mirrored from the GitHub release. License: `apache-2.0`.
 
 
17
 
18
  ```python
19
- from kerasformers.models.siglip import SigLIPModel, SigLIPTokenizer
20
- model = SigLIPModel.from_weights("siglip_base_p16_384")
21
- tokenizer = SigLIPTokenizer.from_weights("siglip_base_p16_384")
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
22
  ```
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  pipeline_tag: zero-shot-image-classification
3
  license: apache-2.0
4
+ base_model: google/siglip-base-patch16-384
5
  library_name: kerasformers
6
  tags:
7
  - keras
8
  - kerasformers
9
  - siglip
10
+ - zero-shot-image-classification
11
+ - vision
12
+ - arxiv:2303.15343
13
  - pytorch
14
+ - jax
15
+ - tf
16
  ---
17
 
18
+ ## ***See [our collection](https://huggingface.co/collections/kerasformers/siglip-6a6ab1b9bb61206cd508dccd) for all versions of SigLIP.***
19
+
20
+ # Run SigLIP with Keras 3: JAX, PyTorch, or TensorFlow
21
+
22
+ [![GitHub](https://img.shields.io/badge/GitHub-KerasFormers-black?logo=github)](https://github.com/IMvision12/KerasFormers) [![Docs](https://img.shields.io/badge/Docs-SigLIP-blue)](https://imvision12.github.io/KerasFormers/siglip/) [![Collection](https://img.shields.io/badge/HF-SigLIP%20collection-yellow)](https://huggingface.co/collections/kerasformers/siglip-6a6ab1b9bb61206cd508dccd)
23
+
24
+ # kerasformers/siglip_base_p16_384
25
+
26
+ Paper: [Sigmoid Loss for Language Image Pre-Training (arXiv:2303.15343)](https://arxiv.org/abs/2303.15343) · [HF Papers](https://huggingface.co/papers/2303.15343)
27
+
28
+ SigLIP is a vision + text dual encoder trained with a pairwise sigmoid loss instead of CLIP's softmax contrastive loss. Per-pair training scales to large batches and often improves zero-shot accuracy at the same model size.
29
+
30
+ For more details on the model, please go to the upstream [model card](https://huggingface.co/google/siglip-base-patch16-384).
31
+
32
+ Pure-**Keras 3** conversion of [`google/siglip-base-patch16-384`](https://huggingface.co/google/siglip-base-patch16-384) for [kerasformers](https://github.com/IMvision12/KerasFormers). One implementation runs unmodified on **TensorFlow / Torch / JAX**.
33
 
34
+ This is a **zero-shot image-text** checkpoint (`SigLIPZeroShotClassify`): pass image(s) and text prompts at inference time.
35
+
36
+ ## ✨ Quick start
37
 
38
  ```python
39
+ import os
40
+ os.environ["KERAS_BACKEND"] = "torch" # or "jax" / "tensorflow"
41
+
42
+ from kerasformers.models.siglip import (
43
+ SigLIPProcessor,
44
+ SigLIPZeroShotClassify,
45
+ )
46
+
47
+ processor = SigLIPProcessor.from_weights("kerasformers/siglip_base_p16_384")
48
+ model = SigLIPZeroShotClassify.from_weights("kerasformers/siglip_base_p16_384")
49
+
50
+ labels = [
51
+ "a photo of a cat",
52
+ "a photo of a dog",
53
+ "a photo of a car",
54
+ "a photo of a living room",
55
+ ]
56
+ inputs = processor(text=labels, image_paths="your_image.jpg")
57
+ output = model(
58
+ {
59
+ "images": inputs["images"],
60
+ "token_ids": inputs["input_ids"],
61
+ }
62
+ )
63
+ print(output["image_logits"].shape)
64
  ```
65
+
66
+ Load any SigLIP variant the same way with `from_weights("kerasformers/<variant>")`:
67
+
68
+ | Variant | Hub |
69
+ |---|---|
70
+ | `siglip_base_p16_224` | [`kerasformers/siglip_base_p16_224`](https://huggingface.co/kerasformers/siglip_base_p16_224) |
71
+ | `siglip_base_p16_256` | [`kerasformers/siglip_base_p16_256`](https://huggingface.co/kerasformers/siglip_base_p16_256) |
72
+ | `siglip_base_p16_multilingual_256` | [`kerasformers/siglip_base_p16_multilingual_256`](https://huggingface.co/kerasformers/siglip_base_p16_multilingual_256) |
73
+ | `siglip_base_p16_384` | [`kerasformers/siglip_base_p16_384`](https://huggingface.co/kerasformers/siglip_base_p16_384) |
74
+ | `siglip_base_p16_512` | [`kerasformers/siglip_base_p16_512`](https://huggingface.co/kerasformers/siglip_base_p16_512) |
75
+ | `siglip_large_p16_256` | [`kerasformers/siglip_large_p16_256`](https://huggingface.co/kerasformers/siglip_large_p16_256) |
76
+ | `siglip_large_p16_384` | [`kerasformers/siglip_large_p16_384`](https://huggingface.co/kerasformers/siglip_large_p16_384) |
77
+ | `siglip_so400m_p14_224` | [`kerasformers/siglip_so400m_p14_224`](https://huggingface.co/kerasformers/siglip_so400m_p14_224) |
78
+ | `siglip_so400m_p14_384` | [`kerasformers/siglip_so400m_p14_384`](https://huggingface.co/kerasformers/siglip_so400m_p14_384) |
79
+
80
+ ## Tips
81
+
82
+ - Set `KERAS_BACKEND` **before** importing Keras / kerasformers.
83
+ - Prefer `Processor.from_weights(...)` so image size and tokenizer match the variant.
84
+ - Map processor `input_ids` to model `token_ids`. No padding mask is required.
85
+ - See [SigLIP docs](https://imvision12.github.io/KerasFormers/siglip/) and [Loading Weights](https://imvision12.github.io/KerasFormers/loading_weights/).
86
+ - Community / upstream safetensors still work via the `hf:` prefix, e.g. `SigLIPZeroShotClassify.from_weights("hf:google/siglip-base-patch16-384")`.
87
+
88
+ ## Special Thanks
89
+
90
+ A huge thank you to the Google SigLIP authors for creating and releasing these models.
91
+
92
+ License: Apache 2.0.