IMvision12 commited on
Commit
4e85f04
·
verified ·
1 Parent(s): bc9dbac

fix readme.md

Browse files
Files changed (1) hide show
  1. README.md +82 -7
README.md CHANGED
@@ -1,22 +1,97 @@
1
  ---
2
  pipeline_tag: zero-shot-image-classification
3
  license: apache-2.0
 
4
  library_name: kerasformers
5
  tags:
6
  - keras
7
  - kerasformers
8
  - siglip2
9
- - tf
10
- - jax
 
11
  - pytorch
 
 
12
  ---
13
 
14
- # siglip2_large_p16_384 (Keras 3)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
15
 
16
- Pure-Keras 3 weights for [kerasformers](https://github.com/IMvision12/KerasFormers), mirrored from the GitHub release. License: `apache-2.0`.
 
 
17
 
18
  ```python
19
- from kerasformers.models.siglip2 import SigLIP2Model, SigLIP2Tokenizer
20
- model = SigLIP2Model.from_weights("siglip2_large_p16_384")
21
- tokenizer = SigLIP2Tokenizer.from_weights("siglip2_large_p16_384")
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
22
  ```
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  pipeline_tag: zero-shot-image-classification
3
  license: apache-2.0
4
+ base_model: google/siglip2-large-patch16-384
5
  library_name: kerasformers
6
  tags:
7
  - keras
8
  - kerasformers
9
  - siglip2
10
+ - zero-shot-image-classification
11
+ - vision
12
+ - arxiv:2502.14786
13
  - pytorch
14
+ - jax
15
+ - tf
16
  ---
17
 
18
+ ## ***See [our collection](https://huggingface.co/collections/kerasformers/siglip2-6a6ab35c79c0333394386a90) for all versions of SigLIP 2.***
19
+
20
+ # Run SigLIP 2 with Keras 3: JAX, PyTorch, or TensorFlow
21
+
22
+ [![GitHub](https://img.shields.io/badge/GitHub-KerasFormers-black?logo=github)](https://github.com/IMvision12/KerasFormers) [![Docs](https://img.shields.io/badge/Docs-SigLIP--2-blue)](https://imvision12.github.io/KerasFormers/siglip2/) [![Collection](https://img.shields.io/badge/HF-SigLIP--2%20collection-yellow)](https://huggingface.co/collections/kerasformers/siglip2-6a6ab35c79c0333394386a90)
23
+
24
+ # kerasformers/siglip2_large_p16_384
25
+
26
+ Paper: [SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Features (arXiv:2502.14786)](https://arxiv.org/abs/2502.14786) · [HF Papers](https://huggingface.co/papers/2502.14786)
27
+
28
+ SigLIP 2 keeps SigLIP's sigmoid loss and adds captioning-based pretraining, self-distillation, and masked prediction for stronger dense features. It uses a 256k multilingual Gemma vocabulary, so many languages work without a separate multilingual checkpoint.
29
+
30
+ For more details on the model, please go to the upstream [model card](https://huggingface.co/google/siglip2-large-patch16-384).
31
+
32
+ Pure-**Keras 3** conversion of [`google/siglip2-large-patch16-384`](https://huggingface.co/google/siglip2-large-patch16-384) for [kerasformers](https://github.com/IMvision12/KerasFormers). One implementation runs unmodified on **TensorFlow / Torch / JAX**.
33
 
34
+ This is a **zero-shot image-text** checkpoint (`SigLIP2ZeroShotClassify`): pass image(s) and text prompts at inference time.
35
+
36
+ ## ✨ Quick start
37
 
38
  ```python
39
+ import os
40
+ os.environ["KERAS_BACKEND"] = "torch" # or "jax" / "tensorflow"
41
+
42
+ from kerasformers.models.siglip2 import (
43
+ SigLIP2Processor,
44
+ SigLIP2ZeroShotClassify,
45
+ )
46
+
47
+ processor = SigLIP2Processor.from_weights("kerasformers/siglip2_large_p16_384")
48
+ model = SigLIP2ZeroShotClassify.from_weights("kerasformers/siglip2_large_p16_384")
49
+
50
+ labels = [
51
+ "a photo of a cat",
52
+ "a photo of a dog",
53
+ "a photo of a car",
54
+ "a photo of a living room",
55
+ ]
56
+ inputs = processor(text=labels, image_paths="your_image.jpg")
57
+ output = model(
58
+ {
59
+ "images": inputs["images"],
60
+ "token_ids": inputs["input_ids"],
61
+ }
62
+ )
63
+ print(output["image_logits"].shape)
64
  ```
65
+
66
+ Load any SigLIP 2 variant the same way with `from_weights("kerasformers/<variant>")`:
67
+
68
+ | Variant | Hub |
69
+ |---|---|
70
+ | `siglip2_base_p16_224` | [`kerasformers/siglip2_base_p16_224`](https://huggingface.co/kerasformers/siglip2_base_p16_224) |
71
+ | `siglip2_base_p16_256` | [`kerasformers/siglip2_base_p16_256`](https://huggingface.co/kerasformers/siglip2_base_p16_256) |
72
+ | `siglip2_base_p16_384` | [`kerasformers/siglip2_base_p16_384`](https://huggingface.co/kerasformers/siglip2_base_p16_384) |
73
+ | `siglip2_base_p16_512` | [`kerasformers/siglip2_base_p16_512`](https://huggingface.co/kerasformers/siglip2_base_p16_512) |
74
+ | `siglip2_base_p32_256` | [`kerasformers/siglip2_base_p32_256`](https://huggingface.co/kerasformers/siglip2_base_p32_256) |
75
+ | `siglip2_large_p16_256` | [`kerasformers/siglip2_large_p16_256`](https://huggingface.co/kerasformers/siglip2_large_p16_256) |
76
+ | `siglip2_large_p16_384` | [`kerasformers/siglip2_large_p16_384`](https://huggingface.co/kerasformers/siglip2_large_p16_384) |
77
+ | `siglip2_large_p16_512` | [`kerasformers/siglip2_large_p16_512`](https://huggingface.co/kerasformers/siglip2_large_p16_512) |
78
+ | `siglip2_so400m_p14_224` | [`kerasformers/siglip2_so400m_p14_224`](https://huggingface.co/kerasformers/siglip2_so400m_p14_224) |
79
+ | `siglip2_so400m_p14_384` | [`kerasformers/siglip2_so400m_p14_384`](https://huggingface.co/kerasformers/siglip2_so400m_p14_384) |
80
+ | `siglip2_so400m_p16_256` | [`kerasformers/siglip2_so400m_p16_256`](https://huggingface.co/kerasformers/siglip2_so400m_p16_256) |
81
+ | `siglip2_so400m_p16_384` | [`kerasformers/siglip2_so400m_p16_384`](https://huggingface.co/kerasformers/siglip2_so400m_p16_384) |
82
+ | `siglip2_so400m_p16_512` | [`kerasformers/siglip2_so400m_p16_512`](https://huggingface.co/kerasformers/siglip2_so400m_p16_512) |
83
+
84
+ ## Tips
85
+
86
+ - Set `KERAS_BACKEND` **before** importing Keras / kerasformers.
87
+ - Prefer `Processor.from_weights(...)` so image size and tokenizer match the variant.
88
+ - Map processor `input_ids` to model `token_ids`. No padding mask is required.
89
+ - Tokenizer is Gemma-based (multilingual); prefer `Processor.from_weights`.
90
+ - See [SigLIP 2 docs](https://imvision12.github.io/KerasFormers/siglip2/) and [Loading Weights](https://imvision12.github.io/KerasFormers/loading_weights/).
91
+ - Community / upstream safetensors still work via the `hf:` prefix, e.g. `SigLIP2ZeroShotClassify.from_weights("hf:google/siglip2-large-patch16-384")`.
92
+
93
+ ## Special Thanks
94
+
95
+ A huge thank you to the Google SigLIP 2 authors for creating and releasing these models.
96
+
97
+ License: Apache 2.0.