README.md

---
pipeline_tag: depth-estimation
license: apache-2.0
base_model: google/tipsv2-b14-dpt
library_name: kerasformers
tags:
- keras
- kerasformers
- tipsv2
- dpt
- depth-estimation
- image-segmentation
- arxiv:2604.12012
- pytorch
- jax
- tf
---

## ***See [our collection](https://huggingface.co/collections/kerasformers/tipsv2-dpt-6a8a3f36cd22fe9f68df6202) for all versions of TIPSv2-DPT.***

# Run TIPSv2-DPT with Keras 3: JAX, PyTorch, or TensorFlow

[![GitHub](https://img.shields.io/badge/GitHub-KerasFormers-black?logo=github)](https://github.com/IMvision12/KerasFormers) [![Collection](https://img.shields.io/badge/HF-TIPSv2--DPT%20collection-yellow)](https://huggingface.co/collections/kerasformers/tipsv2-dpt-6a8a3f36cd22fe9f68df6202)

# kerasformers/tipsv2-b14-dpt

Paper: [TIPSv2 (arXiv:2604.12012)](https://huggingface.co/papers/2604.12012)

TIPSv2-DPT stacks DPT (Dense Prediction Transformer) heads on the TIPSv2 vision backbone. This **single** checkpoint serves three task classes: monocular depth estimation and semantic segmentation, or both at once.

For more details on the model, please go to the upstream [model card](https://huggingface.co/google/tipsv2-b14-dpt).

Pure-**Keras 3** conversion of [`google/tipsv2-b14-dpt`](https://huggingface.co/google/tipsv2-b14-dpt) for [kerasformers](https://github.com/IMvision12/KerasFormers). One implementation runs unmodified on **TensorFlow / Torch / JAX**.

## ✨ Quick start

```python
import os
os.environ["KERAS_BACKEND"] = "torch"  # or "jax" / "tensorflow"

from PIL import Image
import numpy as np
import keras
from kerasformers.models.tipsv2_dpt import (
    Tipsv2DptDensePredict,        # depth + segmentation
    Tipsv2DptDepthEstimation,     # depth only
    Tipsv2DptSemanticSegment,     # segmentation only
    Tipsv2DptImageProcessor,
)

# all three load from the SAME repo
model = Tipsv2DptDensePredict.from_weights("kerasformers/tipsv2-b14-dpt")
proc = Tipsv2DptImageProcessor(image_resolution=448)

image = Image.open("your_image.jpg").convert("RGB")
pixel_values = proc(np.array(image))["pixel_values"]
out = model(pixel_values)
depth = keras.ops.convert_to_numpy(out["predicted_depth"])          # (1, H', W')
seg = keras.ops.convert_to_numpy(out["segmentation_logits"])        # (1, H', W', num_labels)

# single-task variants (same weights, one output each)
depth_model = Tipsv2DptDepthEstimation.from_weights("kerasformers/tipsv2-b14-dpt")
seg_model = Tipsv2DptSemanticSegment.from_weights("kerasformers/tipsv2-b14-dpt")
```

Variants:

| Variant | Hub |
|---|---|
| `tipsv2-b14-dpt` | [`kerasformers/tipsv2-b14-dpt`](https://huggingface.co/kerasformers/tipsv2-b14-dpt) |
| `tipsv2-l14-dpt` | [`kerasformers/tipsv2-l14-dpt`](https://huggingface.co/kerasformers/tipsv2-l14-dpt) |
| `tipsv2-so400m14-dpt` | [`kerasformers/tipsv2-so400m14-dpt`](https://huggingface.co/kerasformers/tipsv2-so400m14-dpt) |
| `tipsv2-g14-dpt` | [`kerasformers/tipsv2-g14-dpt`](https://huggingface.co/kerasformers/tipsv2-g14-dpt) |

## Tips

- Set `KERAS_BACKEND` **before** importing Keras / kerasformers.
- The image processor rescales to `[0, 1]` (no mean/std normalization); input resolution is 448.
- Outputs are at the DPT feature resolution; resize to the input size for visualization.
- Upstream checkpoint: `Tipsv2DptDensePredict.from_weights("hf:google/tipsv2-b14-dpt")`.

## Special Thanks

A huge thank you to the TIPSv2 authors (Google DeepMind) and the HF community.

License: Apache-2.0 (matches the upstream `google/tipsv2-b14-dpt` checkpoint).