| --- |
| pipeline_tag: image-segmentation |
| license: cc-by-nc-4.0 |
| base_model: facebook/maskformer-swin-tiny-coco |
| library_name: kerasformers |
| tags: |
| - keras |
| - kerasformers |
| - maskformer |
| - universal-segmentation |
| - image-segmentation |
| - arxiv:2107.06278 |
| - pytorch |
| - jax |
| - tf |
| --- |
| |
| ## ***See [our collection](https://huggingface.co/collections/kerasformers/maskformer-6a6a8ece1c77558c676dfb9d) for all versions of MaskFormer.*** |
|
|
| # Run MaskFormer with Keras 3: JAX, PyTorch, or TensorFlow |
|
|
| [](https://github.com/IMvision12/KerasFormers) [](https://imvision12.github.io/KerasFormers/maskformer/) [](https://huggingface.co/collections/kerasformers/maskformer-6a6a8ece1c77558c676dfb9d) |
|
|
| # kerasformers/maskformer-swin-tiny-coco |
|
|
| Paper: [Per-Pixel Classification is Not All You Need for Semantic Segmentation (arXiv:2107.06278)](https://arxiv.org/abs/2107.06278) · [HF Papers](https://huggingface.co/papers/2107.06278) |
|
|
| MaskFormer reframes segmentation as mask classification: a backbone and pixel decoder feed a transformer decoder whose queries each predict a binary mask and a class. One architecture covers semantic, instance, and panoptic outputs via post-processing. |
|
|
| For more details on the model, please go to the upstream [model card](https://huggingface.co/facebook/maskformer-swin-tiny-coco). |
|
|
| Pure-**Keras 3** conversion of [`facebook/maskformer-swin-tiny-coco`](https://huggingface.co/facebook/maskformer-swin-tiny-coco) for [kerasformers](https://github.com/IMvision12/KerasFormers). One implementation runs unmodified on **TensorFlow / Torch / JAX**. |
|
|
| This is a **universal segmentation** checkpoint (`MaskFormerUniversalSegment`) trained on COCO panoptic. |
|
|
| ## ✨ Quick start |
|
|
| ```python |
| import os |
| os.environ["KERAS_BACKEND"] = "torch" # or "jax" / "tensorflow" |
| |
| from PIL import Image |
| from kerasformers.models.maskformer import MaskFormerUniversalSegment, MaskFormerImageProcessor |
| |
| model = MaskFormerUniversalSegment.from_weights("kerasformers/maskformer-swin-tiny-coco") |
| processor = MaskFormerImageProcessor.from_weights("kerasformers/maskformer-swin-tiny-coco") |
| |
| image = Image.open("your_image.jpg").convert("RGB") |
| output = model(processor(image)["pixel_values"], training=False) |
| result = processor.post_process_panoptic_segmentation( |
| output, target_size=(image.height, image.width) |
| ) |
| print(result["segmentation"].shape) |
| ``` |
|
|
| Load any MaskFormer variant the same way with `from_weights("kerasformers/<variant>")`: |
|
|
| | Variant | Hub | Dataset | |
| |---|---|---| |
| | `maskformer-swin-tiny-coco` | [`kerasformers/maskformer-swin-tiny-coco`](https://huggingface.co/kerasformers/maskformer-swin-tiny-coco) | COCO | |
| | `maskformer-swin-small-coco` | [`kerasformers/maskformer-swin-small-coco`](https://huggingface.co/kerasformers/maskformer-swin-small-coco) | COCO | |
| | `maskformer-swin-base-coco` | [`kerasformers/maskformer-swin-base-coco`](https://huggingface.co/kerasformers/maskformer-swin-base-coco) | COCO | |
| | `maskformer-swin-tiny-ade` | [`kerasformers/maskformer-swin-tiny-ade`](https://huggingface.co/kerasformers/maskformer-swin-tiny-ade) | ADE20K | |
| | `maskformer-swin-base-ade` | [`kerasformers/maskformer-swin-base-ade`](https://huggingface.co/kerasformers/maskformer-swin-base-ade) | ADE20K | |
|
|
| ## Tips |
|
|
| - Set `KERAS_BACKEND` **before** importing Keras / kerasformers. |
| - Prefer `MaskFormerImageProcessor.from_weights(...)` so resolution matches the variant. |
| - See [MaskFormer docs]({DOCS_URL}) and [Loading Weights](https://imvision12.github.io/KerasFormers/loading_weights/). |
| - Community / upstream weights: `MaskFormerUniversalSegment.from_weights("hf:facebook/maskformer-swin-tiny-coco")`. |
|
|
| ## Special Thanks |
|
|
| A huge thank you to the Facebook AI Research MaskFormer authors for creating and releasing these models. |
|
|
| License: CC-BY-NC-4.0 (non-commercial). |
|
|
