---
license: afl-3.0
language:
- en
base_model:
- qualcomm/MobileSam
pipeline_tag: image-segmentation
tags:
- SAM
- MobileSAM
- Segmentation
---
# MobileSAM
基于MobileSAM的图像分割Pipeline，支持多种输入提示（框、点、掩码），支持650N/620E系列平台的模型推理。

支持芯片:
- AX650N
- AX620E
- AX630C


支持硬件

  - [M4N-Dock(爱芯派Pro)](https://wiki.sipeed.com/hardware/zh/maixIV/m4ndock/m4ndock.html)
  - [M.2 Accelerator card](https://docs.m5stack.com/zh_CN/ai_hardware/LLM-8850_Card)

原始模型请参考
- [MobileSAM Github](https://github.com/ChaoningZhang/MobileSAM)

## 性能对比

- 输入图片大小 1024x1024

|Chip| Models                | Latency (ms) | CMM Usage (MiB) |
|----| --------------------- | ---------------------- | -------------- |
|650N| mobile_sam_encoder          |49.495               | 48.334   |
|630C| mobile_sam_encoder         |520.044              | 63.231   |
|650N| mobile_sam_decoder          |9.930                | 16.703   | 
|630C| mobile_sam_decoder         |36.382               | 14.970   |

## 模型转换
- 模型转换工具链[Pulsar2](https://huggingface.co/AXERA-TECH/Pulsar2)
- 转换文档[Model Convert](https://github.com/AXERA-TECH/MobileSAM.axera/tree/master/convert)

## 环境准备
- NPU Python API: [pyaxengine](https://github.com/AXERA-TECH/pyaxengine)

安装需要的python库
```pip install -r requirements.txt```

## 运行
```shell
cd python_ax
python3 main.py -i ../images/test.jpg -c 650
```
output:
![point](./images/point_mask_ovlap_point_1.jpg)
![box](./images/box_mask_ovlap_box_1.jpg)