|
|
--- |
|
|
license: mit |
|
|
--- |
|
|
|
|
|
# HIPIE: Hierarchical Open-vocabulary Universal Image Segmentation |
|
|
|
|
|
PyTorch implementation of HIPIE from ["Hierarchical Open-vocabulary Universal Image Segmentation"](https://arxiv.org/abs/2307.00764) (Wang et al., NeurIPS 2023). |
|
|
|
|
|
## Pretrained Weights |
|
|
|
|
|
We provide ViT-H and ResNet-50 weights for hierarchical and part-aware image segmentation across multiple datasets: |
|
|
|
|
|
| Format | Filename | Description | |
|
|
|--------|----------|-------------| |
|
|
| ViT-H (O365, COCO, RefCOCO, PACO) | `vit_h_cloud.pth` | Pretrained with O365,COCO,RefCOCO,PACO | |
|
|
| ViT-H (COCO, RefCOCO, Pascal-Parts) | `vit_h_cloud_parts.pth` | Finetuned on COCO,RefCOCO,Pascal-Parts | |
|
|
| ResNet-50 (Pascal-Parts) | `r50_parts.pth` | Pretrained with O365,COCO,RefCOCO,Pascal Panoptic Parts | |
|
|
|
|
|
## Usage |
|
|
|
|
|
For demo notebooks, model configs, and inference scripts, see the [GitHub repository](https://github.com/berkeley-hipie/HIPIE). |
|
|
|
|
|
## Citation |
|
|
``` |
|
|
@inproceedings{wang2023hierarchical, |
|
|
title={Hierarchical Open-vocabulary Universal Image Segmentation}, |
|
|
author={Wang, Xudong and Li, Shufan and Kallidromitis, Konstantinos and Kato, Yusuke and Kozuka, Kazuki and Darrell, Trevor}, |
|
|
booktitle={Thirty-seventh Conference on Neural Information Processing Systems}, |
|
|
year={2023} |
|
|
} |
|
|
``` |
