Upload folder using huggingface_hub

README.md

@@ -0,0 +1,83 @@
+---
+pipeline_tag: zero-shot-image-classification
+license: apache-2.0
+base_model: google/tipsv2-so400m14
+library_name: kerasformers
+tags:
+- keras
+- kerasformers
+- tipsv2
+- zero-shot-image-classification
+- vision
+- arxiv:2604.12012
+- pytorch
+- jax
+- tf
+---
+
+## ***See [our collection](https://huggingface.co/collections/kerasformers/tipsv2-6a8a3f36af77204954a49fb4) for all versions of TIPSv2.***
+
+# Run TIPSv2 with Keras 3: JAX, PyTorch, or TensorFlow
+
+[![GitHub](https://img.shields.io/badge/GitHub-KerasFormers-black?logo=github)](https://github.com/IMvision12/KerasFormers) [![Collection](https://img.shields.io/badge/HF-TIPSv2%20collection-yellow)](https://huggingface.co/collections/kerasformers/tipsv2-6a8a3f36af77204954a49fb4)
+
+# kerasformers/tipsv2-so400m14
+
+Paper: [TIPSv2: Advancing Vision-Language Pretraining with Enhanced Patch-Text Alignment (arXiv:2604.12012)](https://huggingface.co/papers/2604.12012)
+
+TIPSv2 (Google DeepMind) is a CLIP/SigLIP-style dual encoder: a DINOv2-style ViT vision tower with register tokens plus a bidirectional text tower, aligned with a temperature-scaled contrastive objective.
+
+For more details on the model, please go to the upstream [model card](https://huggingface.co/google/tipsv2-so400m14).
+
+Pure-**Keras 3** conversion of [`google/tipsv2-so400m14`](https://huggingface.co/google/tipsv2-so400m14) for [kerasformers](https://github.com/IMvision12/KerasFormers). One implementation runs unmodified on **TensorFlow / Torch / JAX**. The full model and both towers load from this single repo.
+
+## ✨ Quick start (zero-shot)
+
+```python
+import os
+os.environ["KERAS_BACKEND"] = "torch"  # or "jax" / "tensorflow"
+
+from PIL import Image
+import numpy as np
+import keras
+from kerasformers.models.tipsv2 import Tipsv2Model, Tipsv2Processor
+
+model = Tipsv2Model.from_weights("kerasformers/tipsv2-so400m14")
+processor = Tipsv2Processor.from_weights("kerasformers/tipsv2-so400m14")
+
+image = Image.open("your_image.jpg").convert("RGB")
+texts = ["a photo of a cat", "a photo of a dog", "a photo of a car"]
+inputs = processor(text=texts, images=np.array(image))
+out = model(inputs)
+probs = keras.ops.softmax(out["logits_per_image"], axis=-1)
+print(keras.ops.convert_to_numpy(probs)[0])
+```
+
+Towers only:
+
+```python
+from kerasformers.models.tipsv2 import Tipsv2VisionModel, Tipsv2TextModel
+vision = Tipsv2VisionModel.from_weights("kerasformers/tipsv2-so400m14")
+text = Tipsv2TextModel.from_weights("kerasformers/tipsv2-so400m14")
+```
+
+All TIPSv2 variants load the same way with `from_weights("kerasformers/<variant>")`:
+
+| Variant | Hub |
+|---|---|
+| `tipsv2-b14` | [`kerasformers/tipsv2-b14`](https://huggingface.co/kerasformers/tipsv2-b14) |
+| `tipsv2-l14` | [`kerasformers/tipsv2-l14`](https://huggingface.co/kerasformers/tipsv2-l14) |
+| `tipsv2-so400m14` | [`kerasformers/tipsv2-so400m14`](https://huggingface.co/kerasformers/tipsv2-so400m14) |
+| `tipsv2-g14` | [`kerasformers/tipsv2-g14`](https://huggingface.co/kerasformers/tipsv2-g14) |
+
+## Tips
+
+- Set `KERAS_BACKEND` **before** importing Keras / kerasformers.
+- The image processor rescales to `[0, 1]` (no mean/std normalization); input resolution is 448.
+- Upstream checkpoints: `Tipsv2Model.from_weights("hf:google/tipsv2-so400m14")`.
+
+## Special Thanks
+
+A huge thank you to the TIPSv2 authors (Google DeepMind) and the HF community.
+
+License: Apache-2.0 (matches the upstream `google/tipsv2-so400m14` checkpoint).

kf_config.json

@@ -0,0 +1,39 @@
+{
+  "library_name": "kerasformers",
+  "kerasformers_version": "1.2.5",
+  "model_module": "kerasformers.models.tipsv2",
+  "model_class": "Tipsv2Model",
+  "variant": "tipsv2-so400m14",
+  "weights": "model.weights.h5",
+  "schema_version": 2,
+  "model_type": "tipsv2",
+  "text_config": {
+    "hidden_dim": 1152,
+    "num_layers": 27,
+    "num_heads": 16,
+    "mlp_dim": 4304,
+    "vocab_size": 32000,
+    "max_seq_len": 64,
+    "hidden_act": "relu",
+    "layer_norm_eps": 1e-05,
+    "scale_sqrt_depth": true,
+    "pooling_epsilon": 1e-08,
+    "pad_token_id": 0
+  },
+  "vision_config": {
+    "hidden_dim": 1152,
+    "num_layers": 27,
+    "num_heads": 16,
+    "mlp_ratio": 3.736111111111111,
+    "hidden_act": "gelu",
+    "image_size": 448,
+    "patch_size": 14,
+    "num_channels": 3,
+    "qkv_bias": true,
+    "layerscale_value": 1.0,
+    "use_swiglu_ffn": false,
+    "num_register_tokens": 1,
+    "layer_norm_eps": 1e-06
+  },
+  "temperature_init_value": 0.004638021811842918
+}

kf_preprocessor.json

@@ -0,0 +1,22 @@
+{
+  "library_name": "kerasformers",
+  "kerasformers_version": "1.2.5",
+  "preprocessor_module": "kerasformers.models.tipsv2",
+  "preprocessor_class": "Tipsv2ImageProcessor",
+  "variant": "tipsv2-so400m14",
+  "image_resolution": 448,
+  "resample": "bilinear",
+  "do_normalize": false,
+  "do_resize": true,
+  "mean": [
+    0.0,
+    0.0,
+    0.0
+  ],
+  "std": [
+    1.0,
+    1.0,
+    1.0
+  ],
+  "data_format": "channels_last"
+}

model.weights.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d60410212fde63658171a9f22efa9e4b97ced41aaf14d32e205bb24416741a35
+size 3449079056