Hugging Face's logo

AXERA-TECH
/
EdgeTAM

Image Segmentation
ONNX
ONNX
Model card Files
xet
Community
wli1995 commited on
Commit
730faa8
·
verified ·
1 Parent(s): 8f56fa3

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +107 -101
README.md CHANGED
@@ -1,102 +1,108 @@
1
- # EdgeTAM
2
- 基于EdgeSAM的图像分割Pipeline，支持多种输入提示（框、点、掩码），支持650N系列平台的模型推理。
3
-
4
- 支持芯片:
5
- - AX650N
6
-
7
-
8
- 支持硬件
9
-
10
- - [M4N-Dock(爱芯派Pro)](https://wiki.sipeed.com/hardware/zh/maixIV/m4ndock/m4ndock.html)
11
- - [M.2 Accelerator card](https://axcl-docs.readthedocs.io/zh-cn/latest/doc_guide_hardware.html)
12
-
13
- 原始模型请参考
14
- - [EdgeTAM Github](https://github.com/facebookresearch/EdgeTAM)
15
- - [EdgeTAM Huggingface](https://huggingface.co/facebook/EdgeTAM)
16
-
17
- ## 性能对比
18
-
19
- - 输入图片大小 512x512
20
-
21
- | Models | Latency (ms) | CMM Usage (MB) |
22
- | --------------------- | ---------------------- | -------------- |
23
- | edgetam_image_encoder | 22.348 | 29.124 |
24
- | edgetam_prompt_encoder | 0.055 | 0.023 |
25
- | edgetam_prompt_mask_encoder | 0.457 | 0.037 |
26
- | edgetam_mask_decoder | 4.729 | 16.730 |
27
-
28
- ## 模型转换
29
- - 模型转换工具链[Pulsar2](https://huggingface.co/AXERA-TECH/Pulsar2)
30
- - 转换文档[TODO]
31
-
32
- ## 环境准备
33
- - NPU Python API: [pyaxengine](https://github.com/AXERA-TECH/pyaxengine)
34
-
35
- 安装需要的python库
36
- ```pip install -r requirements.txt```
37
-
38
- ## 运行
39
-
40
- ```bash
41
- (myenv) root@ax650:~/EdgeTAM# python3 image_prediction_ax.py --input_box 75,275,1725,850
42
- [INFO] Available providers: ['AxEngineExecutionProvider']
43
- Loading EdgeTAM Onnx models...
44
- [INFO] Using provider: AxEngineExecutionProvider
45
- [INFO] Chip type: ChipType.MC50
46
- [INFO] VNPU type: VNPUType.DISABLED
47
- [INFO] Engine version: 2.12.0s
48
- [INFO] Model type: 2 (triple core)
49
- [INFO] Compiler version: 5.0-patch1-dirty a512c95e-dirty
50
- [INFO] Using provider: AxEngineExecutionProvider
51
- [INFO] Model type: 2 (triple core)
52
- [INFO] Compiler version: 5.0-patch1-dirty a512c95e-dirty
53
- [INFO] Using provider: AxEngineExecutionProvider
54
- [INFO] Model type: 2 (triple core)
55
- [INFO] Compiler version: 5.0-patch1-dirty a512c95e-dirty
56
- [INFO] Using provider: AxEngineExecutionProvider
57
- [INFO] Model type: 2 (triple core)
58
- [INFO] Compiler version: 5.0-patch1-dirty a512c95e-dirty
59
- Get prompts:
60
- input_box: [ 75 275 1725 850]
61
- input_point_coords: None
62
- input_point_labels: None
63
- Only box input provided
64
- Get dense_embeddings_no_mask
65
- [0.9777304]
66
- ✅ Saved: ./results/mask_1.png
67
- ```
68
-
69
- 保存结果在 `./results` 目录下：
70
- ![image](./results/mask_1.png)
71
-
72
- ```
73
- (myenv) root@ax650:~/EdgeTAM# python3 image_prediction_ax.py --image_path ./examples/images/truck.jpg --input_box 425,600,700,875 --input_point_coords 575,750 --input_point_labels 0
74
- [INFO] Available providers: ['AxEngineExecutionProvider']
75
- Loading EdgeTAM Onnx models...
76
- [INFO] Using provider: AxEngineExecutionProvider
77
- [INFO] Chip type: ChipType.MC50
78
- [INFO] VNPU type: VNPUType.DISABLED
79
- [INFO] Engine version: 2.12.0s
80
- [INFO] Model type: 2 (triple core)
81
- [INFO] Compiler version: 5.0-patch1-dirty a512c95e-dirty
82
- [INFO] Using provider: AxEngineExecutionProvider
83
- [INFO] Model type: 2 (triple core)
84
- [INFO] Compiler version: 5.0-patch1-dirty a512c95e-dirty
85
- [INFO] Using provider: AxEngineExecutionProvider
86
- [INFO] Model type: 2 (triple core)
87
- [INFO] Compiler version: 5.0-patch1-dirty a512c95e-dirty
88
- [INFO] Using provider: AxEngineExecutionProvider
89
- [INFO] Model type: 2 (triple core)
90
- [INFO] Compiler version: 5.0-patch1-dirty a512c95e-dirty
91
- ['575,750']
92
- 575,750
93
- Get prompts:
94
- input_box: [425 600 700 875]
95
- input_point_coords: [[575 750]]
96
- input_point_labels: [0]
97
- Get dense_embeddings_no_mask
98
- [0.90291053]
99
- ✅ Saved: ./results/mask_1.png
100
-
101
- ```
 
 
 
 
 
 
102
  ![image](./results/mask_5.png)
 
1
+ ---
2
+ license: afl-3.0
3
+ base_model:
4
+ - facebook/EdgeTAM
5
+ pipeline_tag: image-segmentation
6
+ ---
7
+ # EdgeTAM
8
+ 基于EdgeSAM的图像分割Pipeline，支持多种输入提示（框、点、掩码），支持650N系列平台的模型推理。
9
+
10
+ 支持芯片:
11
+ - AX650N
12
+
13
+
14
+ 支持硬件
15
+
16
+ - [M4N-Dock(爱芯派Pro)](https://wiki.sipeed.com/hardware/zh/maixIV/m4ndock/m4ndock.html)
17
+ - [M.2 Accelerator card](https://axcl-docs.readthedocs.io/zh-cn/latest/doc_guide_hardware.html)
18
+
19
+ 原始模型请参考
20
+ - [EdgeTAM Github](https://github.com/facebookresearch/EdgeTAM)
21
+ - [EdgeTAM Huggingface](https://huggingface.co/facebook/EdgeTAM)
22
+
23
+ ## 性能对比
24
+
25
+ - 输入图片大小 512x512
26
+
27
+ | Models | Latency (ms) | CMM Usage (MB) |
28
+ | --------------------- | ---------------------- | -------------- |
29
+ | edgetam_image_encoder | 22.348 | 29.124 |
30
+ | edgetam_prompt_encoder | 0.055 | 0.023 |
31
+ | edgetam_prompt_mask_encoder | 0.457 | 0.037 |
32
+ | edgetam_mask_decoder | 4.729 | 16.730 |
33
+
34
+ ## 模型转换
35
+ - 模型转换工具链[Pulsar2](https://huggingface.co/AXERA-TECH/Pulsar2)
36
+ - 转换文档[TODO]
37
+
38
+ ## 环境准备
39
+ - NPU Python API: [pyaxengine](https://github.com/AXERA-TECH/pyaxengine)
40
+
41
+ 安装需要的python库
42
+ ```pip install -r requirements.txt```
43
+
44
+ ## 运行
45
+
46
+ ```bash
47
+ (myenv) root@ax650:~/EdgeTAM# python3 image_prediction_ax.py --input_box 75,275,1725,850
48
+ [INFO] Available providers: ['AxEngineExecutionProvider']
49
+ Loading EdgeTAM Onnx models...
50
+ [INFO] Using provider: AxEngineExecutionProvider
51
+ [INFO] Chip type: ChipType.MC50
52
+ [INFO] VNPU type: VNPUType.DISABLED
53
+ [INFO] Engine version: 2.12.0s
54
+ [INFO] Model type: 2 (triple core)
55
+ [INFO] Compiler version: 5.0-patch1-dirty a512c95e-dirty
56
+ [INFO] Using provider: AxEngineExecutionProvider
57
+ [INFO] Model type: 2 (triple core)
58
+ [INFO] Compiler version: 5.0-patch1-dirty a512c95e-dirty
59
+ [INFO] Using provider: AxEngineExecutionProvider
60
+ [INFO] Model type: 2 (triple core)
61
+ [INFO] Compiler version: 5.0-patch1-dirty a512c95e-dirty
62
+ [INFO] Using provider: AxEngineExecutionProvider
63
+ [INFO] Model type: 2 (triple core)
64
+ [INFO] Compiler version: 5.0-patch1-dirty a512c95e-dirty
65
+ Get prompts:
66
+ input_box: [ 75 275 1725 850]
67
+ input_point_coords: None
68
+ input_point_labels: None
69
+ Only box input provided
70
+ Get dense_embeddings_no_mask
71
+ [0.9777304]
72
+ ✅ Saved: ./results/mask_1.png
73
+ ```
74
+
75
+ 保存结果在 `./results` 目录下：
76
+ ![image](./results/mask_1.png)
77
+
78
+ ```
79
+ (myenv) root@ax650:~/EdgeTAM# python3 image_prediction_ax.py --image_path ./examples/images/truck.jpg --input_box 425,600,700,875 --input_point_coords 575,750 --input_point_labels 0
80
+ [INFO] Available providers: ['AxEngineExecutionProvider']
81
+ Loading EdgeTAM Onnx models...
82
+ [INFO] Using provider: AxEngineExecutionProvider
83
+ [INFO] Chip type: ChipType.MC50
84
+ [INFO] VNPU type: VNPUType.DISABLED
85
+ [INFO] Engine version: 2.12.0s
86
+ [INFO] Model type: 2 (triple core)
87
+ [INFO] Compiler version: 5.0-patch1-dirty a512c95e-dirty
88
+ [INFO] Using provider: AxEngineExecutionProvider
89
+ [INFO] Model type: 2 (triple core)
90
+ [INFO] Compiler version: 5.0-patch1-dirty a512c95e-dirty
91
+ [INFO] Using provider: AxEngineExecutionProvider
92
+ [INFO] Model type: 2 (triple core)
93
+ [INFO] Compiler version: 5.0-patch1-dirty a512c95e-dirty
94
+ [INFO] Using provider: AxEngineExecutionProvider
95
+ [INFO] Model type: 2 (triple core)
96
+ [INFO] Compiler version: 5.0-patch1-dirty a512c95e-dirty
97
+ ['575,750']
98
+ 575,750
99
+ Get prompts:
100
+ input_box: [425 600 700 875]
101
+ input_point_coords: [[575 750]]
102
+ input_point_labels: [0]
103
+ Get dense_embeddings_no_mask
104
+ [0.90291053]
105
+ ✅ Saved: ./results/mask_1.png
106
+
107
+ ```
108
  ![image](./results/mask_5.png)