File size: 3,747 Bytes
730faa8 c6e1533 730faa8 c6e1533 730faa8 8e2a8f5 730faa8 822e01d 730faa8 ff5b345 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 |
---
license: afl-3.0
base_model:
- facebook/EdgeTAM
pipeline_tag: image-segmentation
---
# EdgeTAM
基于EdgeTAM的图像分割Pipeline，支持多种输入提示（框、点、掩码），支持650N系列平台的模型推理。
支持芯片:
- AX650N
支持硬件
- [M4N-Dock(爱芯派Pro)](https://wiki.sipeed.com/hardware/zh/maixIV/m4ndock/m4ndock.html)
- [M.2 Accelerator card](https://docs.m5stack.com/zh_CN/ai_hardware/LLM-8850_Card)
原始模型请参考
- [EdgeTAM Github](https://github.com/facebookresearch/EdgeTAM)
- [EdgeTAM Huggingface](https://huggingface.co/facebook/EdgeTAM)
## 性能对比
- 输入图片大小 1024x1024
| Models | Latency (ms) | CMM Usage (MB) |
| --------------------- | ---------------------- | -------------- |
| edgetam_image_encoder | 22.348 | 29.124 |
| edgetam_prompt_encoder | 0.055 | 0.023 |
| edgetam_prompt_mask_encoder | 0.457 | 0.037 |
| edgetam_mask_decoder | 4.729 | 16.730 |
## 模型转换
- 模型转换工具链[Pulsar2](https://huggingface.co/AXERA-TECH/Pulsar2)
- 转换文档[Model Convert](https://github.com/AXERA-TECH/EdgeTAM.Axera/tree/main/model_convert)
## 环境准备
- NPU Python API: [pyaxengine](https://github.com/AXERA-TECH/pyaxengine)
安装需要的python库
```pip install -r requirements.txt```
## 运行
```bash
(myenv) root@ax650:~/EdgeTAM# python3 image_prediction_ax.py --input_box 75,275,1725,850
[INFO] Available providers: ['AxEngineExecutionProvider']
Loading EdgeTAM Onnx models...
[INFO] Using provider: AxEngineExecutionProvider
[INFO] Chip type: ChipType.MC50
[INFO] VNPU type: VNPUType.DISABLED
[INFO] Engine version: 2.12.0s
[INFO] Model type: 2 (triple core)
[INFO] Compiler version: 5.0-patch1-dirty a512c95e-dirty
[INFO] Using provider: AxEngineExecutionProvider
[INFO] Model type: 2 (triple core)
[INFO] Compiler version: 5.0-patch1-dirty a512c95e-dirty
[INFO] Using provider: AxEngineExecutionProvider
[INFO] Model type: 2 (triple core)
[INFO] Compiler version: 5.0-patch1-dirty a512c95e-dirty
[INFO] Using provider: AxEngineExecutionProvider
[INFO] Model type: 2 (triple core)
[INFO] Compiler version: 5.0-patch1-dirty a512c95e-dirty
Get prompts:
input_box: [ 75 275 1725 850]
input_point_coords: None
input_point_labels: None
Only box input provided
Get dense_embeddings_no_mask
[0.9777304]
✅ Saved: ./results/mask_1.png
```
保存结果在 `./results` 目录下：
```
(myenv) root@ax650:~/EdgeTAM# python3 image_prediction_ax.py --image_path ./examples/images/truck.jpg --input_box 425,600,700,875 --input_point_coords 575,750 --input_point_labels 0
[INFO] Available providers: ['AxEngineExecutionProvider']
Loading EdgeTAM Onnx models...
[INFO] Using provider: AxEngineExecutionProvider
[INFO] Chip type: ChipType.MC50
[INFO] VNPU type: VNPUType.DISABLED
[INFO] Engine version: 2.12.0s
[INFO] Model type: 2 (triple core)
[INFO] Compiler version: 5.0-patch1-dirty a512c95e-dirty
[INFO] Using provider: AxEngineExecutionProvider
[INFO] Model type: 2 (triple core)
[INFO] Compiler version: 5.0-patch1-dirty a512c95e-dirty
[INFO] Using provider: AxEngineExecutionProvider
[INFO] Model type: 2 (triple core)
[INFO] Compiler version: 5.0-patch1-dirty a512c95e-dirty
[INFO] Using provider: AxEngineExecutionProvider
[INFO] Model type: 2 (triple core)
[INFO] Compiler version: 5.0-patch1-dirty a512c95e-dirty
['575,750']
575,750
Get prompts:
input_box: [425 600 700 875]
input_point_coords: [[575 750]]
input_point_labels: [0]
Get dense_embeddings_no_mask
[0.90291053]
✅ Saved: ./results/mask_1.png
```
 |