See our collection for MobileViT / MobileViTV2 DeepLabV3 segmentation.

Run MobileViTV2 DeepLabV3 with Keras 3: JAX, PyTorch, or TensorFlow

kerasformers/mobilevitv2_100_deeplabv3

Papers: MobileViT (arXiv:{PAPER_ARXIV}) · MobileViTV2 (arXiv:2206.02680) · HF Papers

MobileViTV2 backbone + DeepLabV3 ASPP head for Pascal VOC semantic segmentation (21 classes). Resolution is 512, not the 256 used by ImageNet classification checkpoints. Always load the processor with from_weights so resize/crop match.

For more details on the model, please go to the upstream model card.

Pure-Keras 3 conversion of apple/mobilevitv2-1.0-voc-deeplabv3 for kerasformers. One implementation runs unmodified on TensorFlow / Torch / JAX.

This is a semantic segmentation checkpoint (MobileViTV2SemanticSegment).

✨ Quick start

import os
os.environ["KERAS_BACKEND"] = "torch"  # or "jax" / "tensorflow"

from PIL import Image
from kerasformers.models.mobilevitv2 import (
    MobileViTV2SemanticSegment,
    MobileViTV2ImageProcessor,
)

model = MobileViTV2SemanticSegment.from_weights("kerasformers/mobilevitv2_100_deeplabv3")
processor = MobileViTV2ImageProcessor.from_weights("kerasformers/mobilevitv2_100_deeplabv3")

image = Image.open("your_image.jpg").convert("RGB")
output = model(processor(image)["pixel_values"], training=False)
result = processor.post_process_semantic_segmentation(
    output, target_size=(image.height, image.width)
)
print(result["segmentation"].shape)  # (H, W) class ids

Load any DeepLabV3 MobileViT variant the same way with from_weights("kerasformers/<variant>"):

Variant	Hub	Family
`mobilevit_xxs_deeplabv3`	`kerasformers/mobilevit_xxs_deeplabv3`	MobileViT v1
`mobilevit_xs_deeplabv3`	`kerasformers/mobilevit_xs_deeplabv3`	MobileViT v1
`mobilevit_s_deeplabv3`	`kerasformers/mobilevit_s_deeplabv3`	MobileViT v1
`mobilevitv2_100_deeplabv3`	`kerasformers/mobilevitv2_100_deeplabv3`	MobileViT v2
`mobilevitv2_150_deeplabv3`	`kerasformers/mobilevitv2_150_deeplabv3`	MobileViT v2

Tips

Set KERAS_BACKEND before importing Keras / kerasformers.
Do not reuse a classification processor: seg checkpoints need 544/512.
v1 imports from mobilevit; v2 from mobilevitv2.
See docs and Loading Weights.
Upstream: MobileViTV2SemanticSegment.from_weights("hf:apple/mobilevitv2-1.0-voc-deeplabv3").