File size: 4,282 Bytes
1551dab 500ccc3 10f7a76 1551dab 10f7a76 1551dab 500ccc3 1551dab 500ccc3 1551dab 35b27ba 500ccc3 35b27ba 500ccc3 10f7a76 500ccc3 bfde4a8 500ccc3 10f7a76 500ccc3 1551dab 500ccc3 1551dab 500ccc3 10f7a76 500ccc3 10f7a76 500ccc3 1551dab 500ccc3 10f7a76 500ccc3 10f7a76 500ccc3 10f7a76 500ccc3 10f7a76 500ccc3 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 | ---
pipeline_tag: image-segmentation
license: mit
base_model: facebook/mask2former-swin-small-coco-instance
library_name: zeromodels
tags:
- keras
- zeromodels
- mask2former
- instance-segmentation
- image-segmentation
- arxiv:2112.01527
- pytorch
- jax
- tf
---
## ***See [our collection](https://huggingface.co/collections/zeromodels/mask2former-6a8eaf66faaf81a53d54fa03) for all versions of Mask2Former.***
# Run Mask2Former with Keras 3: JAX, PyTorch, or TensorFlow
[](https://github.com/IMvision12/ZeroModels) [](https://imvision12.github.io/ZeroModels/mask2former/) [](https://huggingface.co/collections/zeromodels/mask2former-6a8eaf66faaf81a53d54fa03)
# zeromodels/mask2former-swin-small-coco-instance
Paper: [Masked-attention Mask Transformer for Universal Image Segmentation (arXiv:2112.01527)](https://arxiv.org/abs/2112.01527) · [HF Papers](https://huggingface.co/papers/2112.01527)
Mask2Former improves MaskFormer with masked attention in the transformer decoder, restricting cross-attention to predicted mask regions for sharper boundaries and stronger universal segmentation.
For more details on the model, please go to the upstream [model card](https://huggingface.co/facebook/mask2former-swin-small-coco-instance).
Pure-**Keras 3** conversion of [`facebook/mask2former-swin-small-coco-instance`](https://huggingface.co/facebook/mask2former-swin-small-coco-instance) for [zeromodels](https://github.com/IMvision12/ZeroModels). One implementation runs unmodified on **TensorFlow / Torch / JAX**.
This is a **instance** checkpoint (`Mask2FormerUniversalSegment`) (trained for instance; architecture is universal).
## ✨ Quick start
```python
import os
os.environ["KERAS_BACKEND"] = "torch" # or "jax" / "tensorflow"
from PIL import Image
from zeromodels.models.mask2former import Mask2FormerUniversalSegment, Mask2FormerImageProcessor
model = Mask2FormerUniversalSegment.from_weights("zeromodels/mask2former-swin-small-coco-instance")
processor = Mask2FormerImageProcessor.from_weights("zeromodels/mask2former-swin-small-coco-instance")
image = Image.open("your_image.jpg").convert("RGB")
output = model(processor(image)["pixel_values"], training=False)
result = processor.post_process_panoptic_segmentation(
output, target_size=(image.height, image.width)
)
print(result["segmentation"].shape)
```
Load any Mask2Former variant the same way with `from_weights("zeromodels/<variant>")`:
| Variant | Hub | Task |
|---|---|---|
| `mask2former-swin-tiny-coco-instance` | [`zeromodels/mask2former-swin-tiny-coco-instance`](https://huggingface.co/zeromodels/mask2former-swin-tiny-coco-instance) | instance |
| `mask2former-swin-small-coco-instance` | [`zeromodels/mask2former-swin-small-coco-instance`](https://huggingface.co/zeromodels/mask2former-swin-small-coco-instance) | instance |
| `mask2former-swin-base-coco-instance` | [`zeromodels/mask2former-swin-base-coco-instance`](https://huggingface.co/zeromodels/mask2former-swin-base-coco-instance) | instance |
| `mask2former-swin-large-coco-instance` | [`zeromodels/mask2former-swin-large-coco-instance`](https://huggingface.co/zeromodels/mask2former-swin-large-coco-instance) | instance |
| `mask2former-swin-tiny-coco-panoptic` | [`zeromodels/mask2former-swin-tiny-coco-panoptic`](https://huggingface.co/zeromodels/mask2former-swin-tiny-coco-panoptic) | panoptic |
| `mask2former-swin-tiny-ade-semantic` | [`zeromodels/mask2former-swin-tiny-ade-semantic`](https://huggingface.co/zeromodels/mask2former-swin-tiny-ade-semantic) | semantic |
## Tips
- Set `KERAS_BACKEND` **before** importing Keras / zeromodels.
- The task suffix is what the checkpoint was trained for; post-process accordingly.
- See [Mask2Former docs]({DOCS_URL}) and [Loading Weights](https://imvision12.github.io/ZeroModels/loading_weights/).
- Community / upstream weights: `Mask2FormerUniversalSegment.from_weights("hf:facebook/mask2former-swin-small-coco-instance")`.
## Special Thanks
A huge thank you to the Facebook AI Research Mask2Former authors for creating and releasing these models.
License: MIT.
|