README.md

metadata

pipeline_tag: image-segmentation
license: cc-by-nc-4.0
base_model: facebook/maskformer-swin-tiny-ade
library_name: zeromodels
tags:
  - keras
  - zeromodels
  - maskformer
  - universal-segmentation
  - image-segmentation
  - arxiv:2107.06278
  - pytorch
  - jax
  - tf

See our collection for all versions of MaskFormer.

Run MaskFormer with Keras 3: JAX, PyTorch, or TensorFlow

zeromodels/maskformer-swin-tiny-ade

Paper: Per-Pixel Classification is Not All You Need for Semantic Segmentation (arXiv:2107.06278) · HF Papers

MaskFormer reframes segmentation as mask classification: a backbone and pixel decoder feed a transformer decoder whose queries each predict a binary mask and a class. One architecture covers semantic, instance, and panoptic outputs via post-processing.

For more details on the model, please go to the upstream model card.

Pure-Keras 3 conversion of facebook/maskformer-swin-tiny-ade for zeromodels. One implementation runs unmodified on TensorFlow / Torch / JAX.

This is a universal segmentation checkpoint (MaskFormerUniversalSegment) trained on ADE20K.

✨ Quick start

import os
os.environ["KERAS_BACKEND"] = "torch"  # or "jax" / "tensorflow"

from PIL import Image
from zeromodels.models.maskformer import MaskFormerUniversalSegment, MaskFormerImageProcessor

model = MaskFormerUniversalSegment.from_weights("zeromodels/maskformer-swin-tiny-ade")
processor = MaskFormerImageProcessor.from_weights("zeromodels/maskformer-swin-tiny-ade")

image = Image.open("your_image.jpg").convert("RGB")
output = model(processor(image)["pixel_values"], training=False)
result = processor.post_process_panoptic_segmentation(
    output, target_size=(image.height, image.width)
)
print(result["segmentation"].shape)

Load any MaskFormer variant the same way with from_weights("zeromodels/<variant>"):

Variant	Hub	Dataset
maskformer-swin-tiny-coco	zeromodels/maskformer-swin-tiny-coco	COCO
maskformer-swin-small-coco	zeromodels/maskformer-swin-small-coco	COCO
maskformer-swin-base-coco	zeromodels/maskformer-swin-base-coco	COCO
maskformer-swin-tiny-ade	zeromodels/maskformer-swin-tiny-ade	ADE20K
maskformer-swin-base-ade	zeromodels/maskformer-swin-base-ade	ADE20K

Tips

• Set KERAS_BACKEND before importing Keras / zeromodels.
• Prefer MaskFormerImageProcessor.from_weights(...) so resolution matches the variant.
• See MaskFormer docs and Loading Weights.
• Community / upstream weights: MaskFormerUniversalSegment.from_weights("hf:facebook/maskformer-swin-tiny-ade").

Special Thanks

A huge thank you to the Facebook AI Research MaskFormer authors for creating and releasing these models.

License: CC-BY-NC-4.0 (non-commercial).