+[](https://huggingface.co/spaces/sam-hq-team/sam-hq)
+[](https://openxlab.org.cn/apps/detail/keleiwhu/sam-hq)
+[](https://pepy.tech/project/segment-anything-hq)
+
+
+> [**Segment Anything in High Quality**](https://arxiv.org/abs/2306.01567)
+> NeurIPS 2023
+> ETH Zurich & HKUST
+
+We propose HQ-SAM to upgrade SAM for high-quality zero-shot segmentation. Refer to our [paper](https://arxiv.org/abs/2306.01567) for more details.
+
+## Latest updates
+
+**2025/06** -- :fire::fire: HQ-SAM is supported in the [Huggingface Transformers](https://github.com/huggingface/transformers) library. Please see the detailed usage instruction [here](https://huggingface.co/docs/transformers/main/model_doc/sam_hq). The pretrained model checkpoints can also be downloaded at [here](https://huggingface.co/syscv-community).
+
+**2024/11/17 -- HQ-SAM 2 is released**
+
+- A new suite of improved model checkpoints (denoted as **HQ-SAM 2**, beta-version) are released. See [Model Description](sam-hq2/README.md) for details. Change working directory by `cd sam-hq2`
+
+
+
+Updates
+-----------------
+:fire::fire: **SAM for Video Segmentation**: Interested in intersecting SAM and video? HQ-SAM is supported by [DEVA](https://github.com/hkchengrex/Tracking-Anything-with-DEVA) in its text-prompted mode! Also, check the work [MASA](https://github.com/siyuanliii/masa) and [SAM-PT](https://github.com/SysCV/sam-pt) with SAM.
+
+:fire::fire: **SAM in 3D**: Interested in intersecting SAM and 3D Gaussian Splatting? See our new work [Gaussian Grouping](https://github.com/lkeab/gaussian-grouping)! Also, if you are interested in intersecting SAM and NeRF, please see work [SANeRF-HQ](https://github.com/lyclyc52/SANeRF-HQ)!
+
+More: HQ-SAM is adopted in [Osprey](https://arxiv.org/abs/2312.10032), [CaR](https://torrvision.com/clip_as_rnn/), [SpatialRGPT](https://arxiv.org/abs/2406.01584), [GLaMM](https://arxiv.org/abs/2311.03356), [ENIGMA-51](https://iplab.dmi.unict.it/ENIGMA-51/) to provide fine-grained mask annotations.
+
+
+Platform integration: HQ-SAM is supported in the [OpenMMLab PlayGround](https://github.com/open-mmlab/playground/blob/main/label_anything/readme.md) for annotation with Label-Studio, in [segment-geospatial](https://github.com/opengeos/segment-geospatial) for segmenting geospatial data, and mask annotation tool [ISAT](https://github.com/yatengLG/ISAT_with_segment_anything), and [Supervisely](https://supervisely.com/blog/segment-anything-in-high-quality-HQ-SAM/)!
+
+2023/08/11: Support [python package](#quick-installation-via-pip) for easier **pip installation**. Light HQ-SAM is in [EfficientSAM series](https://github.com/IDEA-Research/Grounded-Segment-Anything/tree/main/EfficientSAM) combining with [Grounded SAM](https://github.com/IDEA-Research/Grounded-Segment-Anything/)!
+
+
+
+:rocket::rocket: 2023/07/17: We released **Light HQ-SAM** using TinyViT as backbone, for both fast and high-quality zero-shot segmentation, which reaches **41.2 FPS**. Refer to [Light HQ-SAM vs. MobileSAM](#light-hq-sam-vs-mobilesam-on-coco) for more details.
+
+:trophy::1st_place_medal: 2023/07/14: Grounded **HQ-SAM** obtains the **first place**:1st_place_medal: in the [Segmentation in the Wild](https://eval.ai/web/challenges/challenge-page/1931/leaderboard/4567) competition on zero-shot track (hosted in [CVPR 2023 workshop](https://computer-vision-in-the-wild.github.io/cvpr-2023/)), outperforming Grounded SAM. Refer to our [SGinW evaluation](#grounded-hq-sam-vs-grounded-sam-on-seginw) for more details.
+
+2023/07/05: We released [SAM tuning instuctions](#hq-sam-tuning-and-hq-seg44k-data) and [HQSeg-44K data](#hq-sam-tuning-and-hq-seg44k-data).
+
+2023/07/04: HQ-SAM is adopted in [SAM-PT](https://github.com/SysCV/sam-pt) to improve the SAM-based zero-shot video segmentation performance. Also, HQ-SAM is used in [Grounded-SAM](https://github.com/IDEA-Research/Grounded-Segment-Anything), [Inpaint Anything](https://github.com/Uminosachi/sd-webui-inpaint-anything) and [HQTrack](https://github.com/jiawen-zhu/HQTrack) (2nd in VOTS 2023).
+
+2023/06/28: We released the [ONNX export script](#onnx-export) and [colab notebook](https://colab.research.google.com/drive/11U2La49c2IxahzJkAV-EzPqEH3cz_5hq?usp=sharing) for exporting and using ONNX model.
+
+2023/06/23: Play with HQ-SAM demo at [](https://huggingface.co/spaces/sam-hq-team/sam-hq), which supports point, box and text prompts.
+
+2023/06/14: We released the [colab demo](https://colab.research.google.com/drive/1QwAbn5hsdqKOD5niuBzuqQX4eLCbNKFL?usp=sharing) and [automatic mask generator notebook](https://colab.research.google.com/drive/1dhRq4eR6Fbl-yl1vbQvU9hqyyeOidQaU?usp=sharing).
+
+2023/06/13: We released the [model checkpoints](#model-checkpoints) and [demo visualization codes](#getting-started).
+
+Visual comparison between SAM and HQ-SAM
+-----------------
+**SAM vs. HQ-SAM**
+ |
+  |
+  |
+
 |
+  |
+  |
+
+
+Introduction
+-----------------
+The recent Segment Anything Model (SAM) represents a big leap in scaling up segmentation models, allowing for powerful zero-shot capabilities and flexible prompting. Despite being trained with 1.1 billion masks, SAM's mask prediction quality falls short in many cases, particularly when dealing with objects that have intricate structures. We propose HQ-SAM, equipping SAM with the ability to accurately segment any object, while maintaining SAM's original promptable design, efficiency, and zero-shot generalizability. Our careful design reuses and preserves the pre-trained model weights of SAM, while only introducing minimal additional parameters and computation. We design a learnable High-Quality Output Token, which is injected into SAM's mask decoder and is responsible for predicting the high-quality mask. Instead of only applying it on mask-decoder features, we first fuse them with early and final ViT features for improved mask details. To train our introduced learnable parameters, we compose a dataset of 44K fine-grained masks from several sources. HQ-SAM is only trained on the introduced detaset of 44k masks, which takes only 4 hours on 8 GPUs. We show the efficacy of HQ-SAM in a suite of 9 diverse segmentation datasets across different downstream tasks, where 7 out of them are evaluated in a zero-shot transfer protocol.
+
+
+
+Quantitative comparison between SAM and HQ-SAM
+-----------------
+Note: For box-prompting-based evaluation, we feed SAM, MobileSAM and our HQ-SAM with the same image/video bounding boxes and adopt the single mask output mode of SAM.
+
+We provide comprehensive performance, model size and speed comparison on SAM variants:
+
+
+### Various ViT backbones on COCO:
+
+Note: For the COCO dataset, we use a SOTA detector FocalNet-DINO trained on the COCO dataset as our box prompt generator.
+
+### YTVIS and HQ-YTVIS
+Note:Using ViT-L backbone. We adopt the SOTA detector Mask2Former trained on the YouTubeVIS 2019 dataset as our video boxes prompt generator while reusing its object association prediction.
+
+
+### DAVIS
+Note: Using ViT-L backbone. We adopt the SOTA model XMem as our video boxes prompt generator while reusing its object association prediction.
+
+
+### **Quick Installation via pip**
+```
+pip install segment-anything-hq
+python
+from segment_anything_hq import sam_model_registry
+model_type = "| Model Name | +Encoder | +GroundingDINO | +Mean AP | +Evaluation Script | +Log | +Output Json | + + +
|---|---|---|---|---|---|---|
| Grounded SAM | +vit-h | +swin-b | +48.7 | +script | +log | +result | +
| Grounded HQ-SAM | +vit-h | +swin-b | +49.6 | +script | +log | +result | +
| Model | +Encoder | +AP | +AP@L | +AP@M | +AP@S | +Model Params (MB) | +FPS | +Memory (GB) | + + +
|---|---|---|---|---|---|---|---|---|
| MobileSAM | +TinyViT | +44.3 | +61.8 | +48.1 | +28.8 | +38.6 | +44.8 | +3.7 | +
| Light HQ-SAM | +TinyViT | +45.0 | +62.8 | +48.8 | +29.2 | +40.3 | +41.2 | +3.7 | +
| Model Name | +SAM | +GroundingDINO | +Mean AP | +Airplane-Parts | +Bottles | +Brain-Tumor | +Chicken | +Cows | +Electric-Shaver | +Elephants | +Fruits | +Garbage | +Ginger-Garlic | +Hand-Metal | +Hand | +House-Parts | +HouseHold-Items | +Nutterfly-Squireel | +Phones | +Poles | +Puppies | +Rail | +Salmon-Fillet | +Strawberry | +Tablets | +Toolkits | +Trash | +Watermelon | + +
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Grounded SAM2 | +vit-l | +swin-b | +49.5 | +38.3 | +67.1 | +12.1 | +80.7 | +52.8 | +72.0 | +78.2 | +83.3 | +26.0 | +45.7 | +73.7 | +77.6 | +8.6 | +60.1 | +84.1 | +34.6 | +28.8 | +48.9 | +14.3 | +24.2 | +83.7 | +29.1 | +20.1 | +28.4 | +66.0 | +
| Grounded HQ-SAM2 | +vit-l | +swin-b | +50.0 | +38.6 | +66.8 | +12.0 | +81.0 | +52.8 | +71.9 | +77.2 | +83.3 | +26.1 | +45.5 | +74.8 | +79.0 | +8.6 | +60.1 | +84.7 | +34.3 | +25.5 | +48.9 | +14.1 | +34.1 | +85.7 | +29.2 | +21.5 | +28.9 | +66.6 | + +
+| + | name | +backbone | +Data | +box AP on COCO | +Checkpoint | +Config | +
|---|---|---|---|---|---|---|
| 1 | +GroundingDINO-T | +Swin-T | +O365,GoldG,Cap4M | +48.4 (zero-shot) / 57.2 (fine-tune) | +Github link | HF link | +link | +
+
+
+
+| Model Name | +SAM | +GroundingDINO | +Mean AP | +Airplane-Parts | +Bottles | +Brain-Tumor | +Chicken | +Cows | +Electric-Shaver | +Elephants | +Fruits | +Garbage | +Ginger-Garlic | +Hand-Metal | +Hand | +House-Parts | +HouseHold-Items | +Nutterfly-Squireel | +Phones | +Poles | +Puppies | +Rail | +Salmon-Fillet | +Strawberry | +Tablets | +Toolkits | +Trash | +Watermelon | + + +
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Grounded SAM | +vit-h | +swin-b | +48.7 | +37.2 | +65.4 | +11.9 | +84.5 | +47.5 | +71.7 | +77.9 | +82.3 | +24.0 | +45.8 | +81.2 | +70.0 | +8.4 | +60.1 | +71.3 | +35.4 | +23.3 | +50.1 | +8.7 | +32.9 | +83.5 | +29.8 | +20.8 | +30.0 | +64.2 | +
| Grounded HQ-SAM | +vit-h | +swin-b | +49.6 | +37.6 | +66.3 | +12.0 | +84.5 | +47.8 | +72.1 | +77.5 | +82.3 | +25.0 | +45.6 | +81.2 | +74.8 | +8.5 | +60.1 | +77.1 | +35.3 | +20.1 | +50.1 | +7.7 | +42.2 | +85.6 | +29.7 | +21.8 | +30.0 | +65.6 | + +
| Model Name | +SAM | +GroundingDINO | +Mean AP | +Airplane-Parts | +Bottles | +Brain-Tumor | +Chicken | +Cows | +Electric-Shaver | +Elephants | +Fruits | +Garbage | +Ginger-Garlic | +Hand-Metal | +Hand | +House-Parts | +HouseHold-Items | +Nutterfly-Squireel | +Phones | +Poles | +Puppies | +Rail | +Salmon-Fillet | +Strawberry | +Tablets | +Toolkits | +Trash | +Watermelon | + +
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Grounded SAM2 | +vit-l | +swin-b | +49.5 | +38.3 | +67.1 | +12.1 | +80.7 | +52.8 | +72.0 | +78.2 | +83.3 | +26.0 | +45.7 | +73.7 | +77.6 | +8.6 | +60.1 | +84.1 | +34.6 | +28.8 | +48.9 | +14.3 | +24.2 | +83.7 | +29.1 | +20.1 | +28.4 | +66.0 | +
| Grounded HQ-SAM2 | +vit-l | +swin-b | +50.0 | +38.6 | +66.8 | +12.0 | +81.0 | +52.8 | +71.9 | +77.2 | +83.3 | +26.1 | +45.5 | +74.8 | +79.0 | +8.6 | +60.1 | +84.7 | +34.3 | +25.5 | +48.9 | +14.1 | +34.1 | +85.7 | +29.2 | +21.5 | +28.9 | +66.6 | + +