Release AI-ModelZoo-4.0.0
Browse files
README.md
CHANGED
|
@@ -64,85 +64,28 @@ For an image resolution of NxM and P classes
|
|
| 64 |
|
| 65 |
## Metrics
|
| 66 |
|
| 67 |
-
Measures are done with default
|
| 68 |
|
| 69 |
### Reference **NPU** memory footprint based on Person PASCAL VOC 2012 + COCO 2017 segmentation dataset 21 classes and a derivative person dataset from it (see Accuracy for details on dataset)
|
| 70 |
|
| 71 |
-
| Model
|
| 72 |
-
|
| 73 |
-
|
| 74 |
-
| [DeepLabv3 MobileNetv2 ASPPv2](https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/semantic_segmentation/
|
| 75 |
-
| [DeepLabv3 MobileNetv2 ASPPv2](https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/semantic_segmentation/
|
| 76 |
|
| 77 |
|
| 78 |
|
| 79 |
### Reference **NPU** inference time based on Person COCO 2017 + PASCAL VOC 2012 segmentation dataset 21 classes and a derivative person dataset from it (see Accuracy for details on dataset)
|
| 80 |
|
| 81 |
|
| 82 |
-
| Model
|
| 83 |
-
|
| 84 |
-
|
| 85 |
-
| [DeepLabv3 MobileNetv2 ASPPv2](https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/semantic_segmentation/
|
| 86 |
-
| [DeepLabv3 MobileNetv2 ASPPv2](https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/semantic_segmentation/
|
| 87 |
|
| 88 |
|
| 89 |
-
### Reference **MPU** inference time based on COCO 2017 + PASCAL VOC 2012 segmentation dataset 21 classes and a derivative person dataset from it (see Accuracy for details on dataset)
|
| 90 |
-
| Model | Dataset | Format | Resolution | Quantization | Board| Execution Engine | Frequency | Inference time (ms) | %NPU | %GPU | %CPU | X-LINUX-AI version |Framework |
|
| 91 |
-
|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------|--------|------------|----------------|-------------------|------------------|-----------|---------------------|-------|--------|------|--------------------|-----------------------|
|
| 92 |
-
| [DeepLabV3 per tensor (no ASPP)](https://www.st.com/en/embedded-software/x-linux-ai.html) | COCO 2017 + PASCAL VOC 2012 | Int8 | 257x257x3 | per-tensor | STM32MP257F-DK2 | NPU/GPU | 1500 MHz | 52.75 | 99.2 | 0.80 | 0 | v6.1.0 | OpenVX | | | | | v6.1.0
|
| 93 |
-
| [DeepLabV3 MobileNetv2 ASPPv1 per channel](https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/semantic_segmentation/deeplab_v3/ST_pretrainedmodel_public_dataset/coco_2017_pascal_voc_2012/deeplab_v3_mobilenetv2_05_16_512/deeplab_v3_mobilenetv2_05_16_512_asppv1_int8.tflite) | COCO 2017 + PASCAL VOC 2012 | Int8 (tflite) | 512x512x3 | per-channel ** | STM32MP257F-DK2 | NPU/GPU | 1500 MHz | 830.50 | 7.38| 92.62 | 0 | v6.1.0 | OpenVX |
|
| 94 |
-
| [DeepLabV3 MobileNetv2 ASPPv1 mixed precision](https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/semantic_segmentation/deeplab_v3/ST_pretrainedmodel_public_dataset/coco_2017_pascal_voc_2012/deeplab_v3_mobilenetv2_05_16_512/deeplab_v3_mobilenetv2_05_16_512_asppv1_int8_f32.tflite) | COCO 2017 + PASCAL VOC 2012 | Int8 & float32 (tflite) | 512x512x3 | per-channel ** | STM32MP257F-DK2 | NPU/GPU | 1500 MHz | 939.8 | 6.29 | 93.71 | 0 | v6.1.0 | OpenVX |
|
| 95 |
-
| [DeepLabV3 MobileNetv2 ASPPv1 per channel](https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/semantic_segmentation/deeplab_v3/ST_pretrainedmodel_public_dataset/coco_2017_pascal_voc_2012/deeplab_v3_mobilenetv2_05_16_512/deeplab_v3_mobilenetv2_05_16_512_asppv1_qdq_int8.onnx) | COCO 2017 + PASCAL VOC 2012 | Int8 (onnx) | 512x512x3 | per-channel ** | STM32MP257F-DK2 | NPU/GPU | 1500 MHz | 729.62 | 3.0 | 97.0 | 0 | v6.1.0| OpenVX |
|
| 96 |
-
|
| 97 |
-
- **DeepLabV3 per tensor**:
|
| 98 |
-
This model, which does not include ASPP (Atrous Spatial Pyramid Pooling), was downloaded from the TensorFlow DeepLabV3 page on [Kaggle](https://www.kaggle.com/models/tensorflow/deeplabv3/).
|
| 99 |
-
|
| 100 |
-
- **The onnx DeepLabv3 MobileNetv2 ASPPv1 per channel**:
|
| 101 |
-
The quantized TFLite model is derived from the DeepLabV3 float precision model. The ONNX quantized model is obtained by quantizing the DeepLabV3 float model using the [deeplab_v3_mobilenetv2_05_16_512_asppv1_onnx_config](https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/semantic_segmentation/deeplab_v3/ST_pretrainedmodel_public_dataset/coco_2017_pascal_voc_2012/deeplab_v3_mobilenetv2_05_16_512/deeplab_v3_mobilenetv2_05_16_512_asppv1_onnx_config.yaml) YAML file.
|
| 102 |
-
|
| 103 |
-
**Note:** These results were obtained using the exact YAML files mentioned above and a specific quantization set containing 4 images from the PASCAL VOC dataset with the following IDs:
|
| 104 |
-
- 2008_004804
|
| 105 |
-
- 2010_005620
|
| 106 |
-
- 2010_004290
|
| 107 |
-
- 2008_000183
|
| 108 |
-
|
| 109 |
-
- **DeepLabV3 MobileNetv2 ASPPv1 mixed precision**:
|
| 110 |
-
This model is a mixed precision version of the DeepLabV3 float precision. The backbone is fully quantized to 8 bits, while the ASPP head remains partially in float precision. Some layers were too sensitive to 8-bit quantization, resulting in unacceptable accuracy degradation. Therefore, we instructed TFLite not to quantize those specific layers.
|
| 111 |
-
|
| 112 |
-
** **To get the most out of MP25 NPU hardware acceleration, please use per-tensor quantization**
|
| 113 |
-
|
| 114 |
-
### Accuracy with COCO 2017 + PASCAL VOC 2012
|
| 115 |
-
|
| 116 |
-
**Pascal VOC Dataset Details:**
|
| 117 |
-
|
| 118 |
-
- **Link:** [VOC 2012 Dataset](http://host.robots.ox.ac.uk/pascal/VOC/voc2012/)
|
| 119 |
-
- **License:** [Database Contents License (DbCL) v1.0](https://opendatacommons.org/licenses/dbcl/1-0/)
|
| 120 |
-
- **Number of Classes:** 21
|
| 121 |
-
- **Contents:**
|
| 122 |
-
- 1464 training images and masks
|
| 123 |
-
- 1449 validation images and masks
|
| 124 |
-
|
| 125 |
-
|
| 126 |
-
**Please follow the [PASCAL VOC 2012 tutorial](https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/semantic_segmentation/datasets) to have more training masks (about 10,582) and a `trainaug.txt` file containing the IDs of the new training masks.**
|
| 127 |
-
|
| 128 |
-
|
| 129 |
-
**COCO Dataset Details:**
|
| 130 |
-
|
| 131 |
-
- **Link:** [COCO 2017 Dataset](https://cocodataset.org/#home)
|
| 132 |
-
- **License:** [COCO License](https://cocodataset.org/#termsofuse)
|
| 133 |
-
|
| 134 |
-
Please note, that the following accuracies are obtained after training the model with the augmented Pascal VOC + COCO data and evaluated on Pascal VOC 2012 validation set (val.txt), and with a preprocessing resize with interpolation method 'bilinear'.
|
| 135 |
-
Moreover, IoU are averaged on all classes including background.
|
| 136 |
-
|
| 137 |
-
**Please use the [COCO 2017 PASCAL VOC 2012 tutorial](https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/semantic_segmentation/datasets/coco_2017_pascal_voc_2012) to create COCO 2017 + PASCAL VOC 2012 dataset to do the needed filtering. Only images containing one or more classes from the 21 Pascal VOC dataset classes should be used. Additionally, the masks need to be converted to the Pascal VOC masks format.**
|
| 138 |
-
|
| 139 |
-
| Model Description | Resolution | Format | Accuracy | Averaed IoU |
|
| 140 |
-
|--------------------------------------------------------------------------------------------------------------------------------------------------------------|------------|------------|----------|--------------|
|
| 141 |
-
| [DeepLabV3 per tensor (no ASPP)](https://www.st.com/en/embedded-software/x-linux-ai.html) | 257x257x3 | Int8 (tflite)| 88.6% | 59.33% |
|
| 142 |
-
| [Deeplabv3 MobileNetv2 ASPPv1 float precision](https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/semantic_segmentation/deeplab_v3/ST_pretrainedmodel_public_dataset/coco_2017_pascal_voc_2012/deeplab_v3_mobilenetv2_05_16_512/deeplab_v3_mobilenetv2_05_16_512_asppv1.h5) | 512x512x3 | Float | 93.29% | 73.44% |
|
| 143 |
-
| [DeepLabv3 MobileNetv2 ASPPv1 per channel](https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/semantic_segmentation/deeplab_v3/ST_pretrainedmodel_public_dataset/coco_2017_pascal_voc_2012/deeplab_v3_mobilenetv2_05_16_512/deeplab_v3_mobilenetv2_05_16_512_asppv1_int8.tflite) | 512x512x3 | Int8 (tflite) | 91.3% | 67.32% |
|
| 144 |
-
| [DDeepLabv3 MobileNetv2 ASPPv1 mixed precision](https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/semantic_segmentation/deeplab_v3/ST_pretrainedmodel_public_dataset/coco_2017_pascal_voc_2012/deeplab_v3_mobilenetv2_05_16_512/deeplab_v3_mobilenetv2_05_16_512_asppv1_int8_f32.tflite) | 512x512x3 | Int8/Float (tflite)| 92.83% | 71.93% |
|
| 145 |
-
| [DeepLabv3 MobileNetv2 ASPPv1 per channel](https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/semantic_segmentation/deeplab_v3/ST_pretrainedmodel_public_dataset/coco_2017_pascal_voc_2012/deeplab_v3_mobilenetv2_05_16_512/deeplab_v3_mobilenetv2_05_16_512_asppv1_qdq_int8.onnx) | 512x512x3 | Int8 (onnx) | 93.15%| 72.39% |
|
| 146 |
|
| 147 |
|
| 148 |
### Accuracy with Person COCO 2017 + PASCAL VOC 2012
|
|
@@ -151,12 +94,11 @@ Moreover, IoU are averaged on all classes including background.
|
|
| 151 |
|
| 152 |
| Models Description | Resolution | Format | Accuracy (%) | average IoU |
|
| 153 |
|--------------------------------------------|-----------|---------------|--------------|-------------|
|
| 154 |
-
| [Deeplabv3 MobileNetv2 ASPPv2 float precision](https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/semantic_segmentation/
|
| 155 |
-
| [DeepLabv3 MobileNetv2 ASPPv2 per channel](https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/semantic_segmentation/
|
| 156 |
-
| [Deeplabv3 MobileNetv2 ASPPv2 float precision](https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/semantic_segmentation/
|
| 157 |
-
| [DeepLabv3 MobileNetv2 ASPPv2 per channel](https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/semantic_segmentation/
|
| 158 |
-
| [Deeplabv3 MobileNetv2 ASPPv2 float precision](https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/semantic_segmentation/
|
| 159 |
-
| [DeepLabv3 MobileNetv2 ASPPv2 per channel](https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/semantic_segmentation/
|
| 160 |
-
|
| 161 |
-
|
| 162 |
-
Please refer to the stm32ai-modelzoo-services GitHub [here](https://github.com/STMicroelectronics/stm32ai-modelzoo-services)
|
|
|
|
| 64 |
|
| 65 |
## Metrics
|
| 66 |
|
| 67 |
+
Measures are done with default STEdgeAI Core version configuration with enabled input / output allocated option.
|
| 68 |
|
| 69 |
### Reference **NPU** memory footprint based on Person PASCAL VOC 2012 + COCO 2017 segmentation dataset 21 classes and a derivative person dataset from it (see Accuracy for details on dataset)
|
| 70 |
|
| 71 |
+
| Model | Dataset | Format | Resolution | Series | Internal RAM (KiB) | External RAM (KiB) | Weights Flash (KiB) | STEdgeAI Core version |
|
| 72 |
+
|-------|---------|--------|------------|--------|--------------------|--------------------|---------------------|----------------------|
|
| 73 |
+
[DeepLabv3 MobileNetv2 ASPPv2](https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/semantic_segmentation/deeplabv3/ST_pretrainedmodel_public_dataset/person_coco_2017_pascal_voc_2012/st_deeplabv3_mnv2_a050_s16_asppv2_256/st_deeplabv3_mnv2_a050_s16_asppv2_256_qdq_int8.onnx) | person COCO 2017 + PASCAL VOC 2012 | Int8 | 256x256x3 | STM32N6 | 1869.88 | 0.0 | 882.33 | 3.0.0 |
|
| 74 |
+
| [DeepLabv3 MobileNetv2 ASPPv2](https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/semantic_segmentation/deeplabv3/ST_pretrainedmodel_public_dataset/person_coco_2017_pascal_voc_2012/st_deeplabv3_mnv2_a050_s16_asppv2_320/st_deeplabv3_mnv2_a050_s16_asppv2_320_qdq_int8.onnx) | person COCO 2017 + PASCAL VOC 2012 | Int8 | 320x320x3 | STM32N6 | 2421 | 0.0 | 893.3 | 3.0.0 |
|
| 75 |
+
| [DeepLabv3 MobileNetv2 ASPPv2](https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/semantic_segmentation/deeplabv3/ST_pretrainedmodel_public_dataset/person_coco_2017_pascal_voc_2012/st_deeplabv3_mnv2_a050_s16_asppv2_416/st_deeplabv3_mnv2_a050_s16_asppv2_416_qdq_int8.onnx) | person COCO 2017 + PASCAL VOC 2012 | Int8 | 416x416x3 | STM32N6 | 2802.28 | 2028.0 | 894.14 | 3.0.0 |
|
| 76 |
|
| 77 |
|
| 78 |
|
| 79 |
### Reference **NPU** inference time based on Person COCO 2017 + PASCAL VOC 2012 segmentation dataset 21 classes and a derivative person dataset from it (see Accuracy for details on dataset)
|
| 80 |
|
| 81 |
|
| 82 |
+
| Model | Dataset | Format | Resolution | Board | Execution Engine | Inference time (ms) | Inf / sec | STEdgeAI Core version |
|
| 83 |
+
|-------|---------|--------|------------|-------|------------------|---------------------|-----------|----------------------
|
| 84 |
+
[DeepLabv3 MobileNetv2 ASPPv2](https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/semantic_segmentation/deeplabv3/ST_pretrainedmodel_public_dataset/person_coco_2017_pascal_voc_2012/st_deeplabv3_mnv2_a050_s16_asppv2_256/st_deeplabv3_mnv2_a050_s16_asppv2_256_qdq_int8.onnx) | person COCO 2017 + PASCAL VOC 2012 | Int8 | 256x256x3 | STM32N6570-DK | NPU/MCU | 26.62 | 37.55 | 3.0.0 |
|
| 85 |
+
| [DeepLabv3 MobileNetv2 ASPPv2](https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/semantic_segmentation/deeplabv3/ST_pretrainedmodel_public_dataset/person_coco_2017_pascal_voc_2012/st_deeplabv3_mnv2_a050_s16_asppv2_320/st_deeplabv3_mnv2_a050_s16_asppv2_320_qdq_int8.onnx) | person COCO 2017 + PASCAL VOC 2012 | Int8 | 320x320x3 | STM32N6570-DK | NPU/MCU | 40.83 | 24.49 | 3.0.0 |
|
| 86 |
+
| [DeepLabv3 MobileNetv2 ASPPv2](https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/semantic_segmentation/deeplabv3/ST_pretrainedmodel_public_dataset/person_coco_2017_pascal_voc_2012/st_deeplabv3_mnv2_a050_s16_asppv2_416/st_deeplabv3_mnv2_a050_s16_asppv2_416_qdq_int8.onnx) | person COCO 2017 + PASCAL VOC 2012 | Int8 | 416x416x3 | STM32N6570-DK | NPU/MCU | 227.02 | 4.41 | 3.0.0 |
|
| 87 |
|
| 88 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 89 |
|
| 90 |
|
| 91 |
### Accuracy with Person COCO 2017 + PASCAL VOC 2012
|
|
|
|
| 94 |
|
| 95 |
| Models Description | Resolution | Format | Accuracy (%) | average IoU |
|
| 96 |
|--------------------------------------------|-----------|---------------|--------------|-------------|
|
| 97 |
+
| [Deeplabv3 MobileNetv2 ASPPv2 float precision](https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/semantic_segmentation/deeplabv3/ST_pretrainedmodel_public_dataset/person_coco_2017_pascal_voc_2012/st_deeplabv3_mnv2_a050_s16_asppv2_256/st_deeplabv3_mnv2_a050_s16_asppv2_256.keras) | 256x256x3 | TensorFlow | 94.46 % | 76.58 % |
|
| 98 |
+
| [DeepLabv3 MobileNetv2 ASPPv2 per channel](https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/semantic_segmentation/deeplabv3/ST_pretrainedmodel_public_dataset/person_coco_2017_pascal_voc_2012/st_deeplabv3_mnv2_a050_s16_asppv2_256/st_deeplabv3_mnv2_a050_s16_asppv2_256_qdq_int8.onnx) | 256x256x3 | ONNX | 94.42 % | 76.25 % |
|
| 99 |
+
| [Deeplabv3 MobileNetv2 ASPPv2 float precision](https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/semantic_segmentation/deeplabv3/ST_pretrainedmodel_public_dataset/person_coco_2017_pascal_voc_2012/st_deeplabv3_mnv2_a050_s16_asppv2_320/st_deeplabv3_mnv2_a050_s16_asppv2_320.keras) | 320x320x3 | TensorFlow | 94.8 % | 77.87 % |
|
| 100 |
+
| [DeepLabv3 MobileNetv2 ASPPv2 per channel](https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/semantic_segmentation/deeplabv3/ST_pretrainedmodel_public_dataset/person_coco_2017_pascal_voc_2012/st_deeplabv3_mnv2_a050_s16_asppv2_320/st_deeplabv3_mnv2_a050_s16_asppv2_320_qdq_int8.onnx) | 320x320x3 | ONNX | 94.71 % | 77.45 % |
|
| 101 |
+
| [Deeplabv3 MobileNetv2 ASPPv2 float precision](https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/semantic_segmentation/deeplabv3/ST_pretrainedmodel_public_dataset/person_coco_2017_pascal_voc_2012/st_deeplabv3_mnv2_a050_s16_asppv2_416/st_deeplabv3_mnv2_a050_s16_asppv2_416.keras) | 416x416x3 | TensorFlow | 95.25 % | 79.9 % |
|
| 102 |
+
| [DeepLabv3 MobileNetv2 ASPPv2 per channel](https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/semantic_segmentation/deeplabv3/ST_pretrainedmodel_public_dataset/person_coco_2017_pascal_voc_2012/st_deeplabv3_mnv2_a050_s16_asppv2_416/st_deeplabv3_mnv2_a050_s16_asppv2_416_qdq_int8.onnx) | 416x416x3 | ONNX | 95.1 % | 79.18 % |
|
| 103 |
+
|
| 104 |
+
**The ASPPv2 is an improved version of the Atrous Spatial Pyramid Pooling (ASPP) module. In ASPPv2, the standard atrous (dilated) convolutions are replaced by separable depthwise dilated convolutions. This change reduces computational complexity and model size, while still capturing multi-scale context efficiently. Please refer to the stm32ai-modelzoo-services GitHub [here](https://github.com/STMicroelectronics/stm32ai-modelzoo-services)**
|
|
|