File size: 3,921 Bytes
51f82d6 9f0f37d 1e3840d 51f82d6 1e3840d 51f82d6 9f0f37d 51f82d6 9f0f37d 51f82d6 5690517 9f0f37d 5690517 9f0f37d 1e3840d 9f0f37d ff876b3 9f0f37d 1e3840d 9f0f37d 51f82d6 9f0f37d 51f82d6 9f0f37d 1e3840d 9f0f37d 1e3840d 9f0f37d 51f82d6 9f0f37d 1e3840d 9f0f37d 1e3840d 9f0f37d 1e3840d 9f0f37d 1e3840d 9f0f37d | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 | ---
pipeline_tag: image-segmentation
license: cc-by-nc-4.0
base_model: facebook/maskformer-swin-tiny-coco
library_name: zeromodels
tags:
- keras
- zeromodels
- maskformer
- universal-segmentation
- image-segmentation
- arxiv:2107.06278
- pytorch
- jax
- tf
---
## ***See [our collection](https://huggingface.co/collections/zeromodels/maskformer-6a8eaf6a43e1a5079d6cc8ef) for all versions of MaskFormer.***
# Run MaskFormer with Keras 3: JAX, PyTorch, or TensorFlow
[](https://github.com/IMvision12/ZeroModels) [](https://imvision12.github.io/ZeroModels/maskformer/) [](https://huggingface.co/collections/zeromodels/maskformer-6a8eaf6a43e1a5079d6cc8ef)
# zeromodels/maskformer-swin-tiny-coco
Paper: [Per-Pixel Classification is Not All You Need for Semantic Segmentation (arXiv:2107.06278)](https://arxiv.org/abs/2107.06278) · [HF Papers](https://huggingface.co/papers/2107.06278)
MaskFormer reframes segmentation as mask classification: a backbone and pixel decoder feed a transformer decoder whose queries each predict a binary mask and a class. One architecture covers semantic, instance, and panoptic outputs via post-processing.
For more details on the model, please go to the upstream [model card](https://huggingface.co/facebook/maskformer-swin-tiny-coco).
Pure-**Keras 3** conversion of [`facebook/maskformer-swin-tiny-coco`](https://huggingface.co/facebook/maskformer-swin-tiny-coco) for [zeromodels](https://github.com/IMvision12/ZeroModels). One implementation runs unmodified on **TensorFlow / Torch / JAX**.
This is a **universal segmentation** checkpoint (`MaskFormerUniversalSegment`) trained on COCO panoptic.
## ✨ Quick start
```python
import os
os.environ["KERAS_BACKEND"] = "torch" # or "jax" / "tensorflow"
from PIL import Image
from zeromodels.models.maskformer import MaskFormerUniversalSegment, MaskFormerImageProcessor
model = MaskFormerUniversalSegment.from_weights("zeromodels/maskformer-swin-tiny-coco")
processor = MaskFormerImageProcessor.from_weights("zeromodels/maskformer-swin-tiny-coco")
image = Image.open("your_image.jpg").convert("RGB")
output = model(processor(image)["pixel_values"], training=False)
result = processor.post_process_panoptic_segmentation(
output, target_size=(image.height, image.width)
)
print(result["segmentation"].shape)
```
Load any MaskFormer variant the same way with `from_weights("zeromodels/<variant>")`:
| Variant | Hub | Dataset |
|---|---|---|
| `maskformer-swin-tiny-coco` | [`zeromodels/maskformer-swin-tiny-coco`](https://huggingface.co/zeromodels/maskformer-swin-tiny-coco) | COCO |
| `maskformer-swin-small-coco` | [`zeromodels/maskformer-swin-small-coco`](https://huggingface.co/zeromodels/maskformer-swin-small-coco) | COCO |
| `maskformer-swin-base-coco` | [`zeromodels/maskformer-swin-base-coco`](https://huggingface.co/zeromodels/maskformer-swin-base-coco) | COCO |
| `maskformer-swin-tiny-ade` | [`zeromodels/maskformer-swin-tiny-ade`](https://huggingface.co/zeromodels/maskformer-swin-tiny-ade) | ADE20K |
| `maskformer-swin-base-ade` | [`zeromodels/maskformer-swin-base-ade`](https://huggingface.co/zeromodels/maskformer-swin-base-ade) | ADE20K |
## Tips
- Set `KERAS_BACKEND` **before** importing Keras / zeromodels.
- Prefer `MaskFormerImageProcessor.from_weights(...)` so resolution matches the variant.
- See [MaskFormer docs]({DOCS_URL}) and [Loading Weights](https://imvision12.github.io/ZeroModels/loading_weights/).
- Community / upstream weights: `MaskFormerUniversalSegment.from_weights("hf:facebook/maskformer-swin-tiny-coco")`.
## Special Thanks
A huge thank you to the Facebook AI Research MaskFormer authors for creating and releasing these models.
License: CC-BY-NC-4.0 (non-commercial).
|