File size: 3,525 Bytes
fd0ec48 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 | ---
pipeline_tag: depth-estimation
license: apache-2.0
base_model: google/tipsv2-b14-dpt
library_name: kerasformers
tags:
- keras
- kerasformers
- tipsv2
- dpt
- depth-estimation
- image-segmentation
- arxiv:2604.12012
- pytorch
- jax
- tf
---
## ***See [our collection](https://huggingface.co/collections/kerasformers/tipsv2-dpt-6a8a3f36cd22fe9f68df6202) for all versions of TIPSv2-DPT.***
# Run TIPSv2-DPT with Keras 3: JAX, PyTorch, or TensorFlow
[](https://github.com/IMvision12/KerasFormers) [](https://huggingface.co/collections/kerasformers/tipsv2-dpt-6a8a3f36cd22fe9f68df6202)
# kerasformers/tipsv2-b14-dpt
Paper: [TIPSv2 (arXiv:2604.12012)](https://huggingface.co/papers/2604.12012)
TIPSv2-DPT stacks DPT (Dense Prediction Transformer) heads on the TIPSv2 vision backbone. This **single** checkpoint serves three task classes: monocular depth estimation and semantic segmentation, or both at once.
For more details on the model, please go to the upstream [model card](https://huggingface.co/google/tipsv2-b14-dpt).
Pure-**Keras 3** conversion of [`google/tipsv2-b14-dpt`](https://huggingface.co/google/tipsv2-b14-dpt) for [kerasformers](https://github.com/IMvision12/KerasFormers). One implementation runs unmodified on **TensorFlow / Torch / JAX**.
## ✨ Quick start
```python
import os
os.environ["KERAS_BACKEND"] = "torch" # or "jax" / "tensorflow"
from PIL import Image
import numpy as np
import keras
from kerasformers.models.tipsv2_dpt import (
Tipsv2DptDensePredict, # depth + segmentation
Tipsv2DptDepthEstimation, # depth only
Tipsv2DptSemanticSegment, # segmentation only
Tipsv2DptImageProcessor,
)
# all three load from the SAME repo
model = Tipsv2DptDensePredict.from_weights("kerasformers/tipsv2-b14-dpt")
proc = Tipsv2DptImageProcessor(image_resolution=448)
image = Image.open("your_image.jpg").convert("RGB")
pixel_values = proc(np.array(image))["pixel_values"]
out = model(pixel_values)
depth = keras.ops.convert_to_numpy(out["predicted_depth"]) # (1, H', W')
seg = keras.ops.convert_to_numpy(out["segmentation_logits"]) # (1, H', W', num_labels)
# single-task variants (same weights, one output each)
depth_model = Tipsv2DptDepthEstimation.from_weights("kerasformers/tipsv2-b14-dpt")
seg_model = Tipsv2DptSemanticSegment.from_weights("kerasformers/tipsv2-b14-dpt")
```
Variants:
| Variant | Hub |
|---|---|
| `tipsv2-b14-dpt` | [`kerasformers/tipsv2-b14-dpt`](https://huggingface.co/kerasformers/tipsv2-b14-dpt) |
| `tipsv2-l14-dpt` | [`kerasformers/tipsv2-l14-dpt`](https://huggingface.co/kerasformers/tipsv2-l14-dpt) |
| `tipsv2-so400m14-dpt` | [`kerasformers/tipsv2-so400m14-dpt`](https://huggingface.co/kerasformers/tipsv2-so400m14-dpt) |
| `tipsv2-g14-dpt` | [`kerasformers/tipsv2-g14-dpt`](https://huggingface.co/kerasformers/tipsv2-g14-dpt) |
## Tips
- Set `KERAS_BACKEND` **before** importing Keras / kerasformers.
- The image processor rescales to `[0, 1]` (no mean/std normalization); input resolution is 448.
- Outputs are at the DPT feature resolution; resize to the input size for visualization.
- Upstream checkpoint: `Tipsv2DptDensePredict.from_weights("hf:google/tipsv2-b14-dpt")`.
## Special Thanks
A huge thank you to the TIPSv2 authors (Google DeepMind) and the HF community.
License: Apache-2.0 (matches the upstream `google/tipsv2-b14-dpt` checkpoint).
|