Release AI-ModelZoo-4.0.0

README.md

@@ -4,7 +4,7 @@ license_link: >-
   https://github.st.com/AIS/stm32ai-modelzoo/raw/master/audio_event_detection/LICENSE.md
 pipeline_tag: audio-classification
 ---
-# Quantized miniresnet
+# Quantized miniresnetv2
 
 ## **Use case** : `AED`
 
@@ -14,20 +14,21 @@ ResNets are well known image classification models, that use skip-connections be
 
 However, they are also widely used in AED and Audio classification, by converting the audio to a mel-spectrogram, and passing that as input to the model.
 
+ResNetv2 changes the order of the skip-connections and ReLU activations in the ordinary ResNet architecture, with the main benefit being faster convergence during training.
 
-MiniResNet is based on the ResNet implementation found in tensorflow, and is a resized version of a ResNet18 with a custom block function. These blocks are then assembled in stacks, and the user can specify the number of stacks desired, with more stacks resulting in a larger network.
+miniresnetv2 is based on the ResNetv2 implementation found in tensorflow, and is a resized version of a ResNet18v2 with a custom block function. These blocks are then assembled in stacks, and the user can specify the number of stacks desired, with more stacks resulting in a larger network.
 
 A note on pooling : In some of our pretrained models, we do not use a pooling function at the end of the convolutional backbone, as is traditionally done. Because of the small number of convolutional blocks, the number of filters is low even for larger model sizes, leading to a low embedding size after pooling.
 We found that in many cases we obtain a better performance / model size / inference time tradeoff by not performing any pooling. This makes the linear classification layer larger, but in cases with a relatively low number of classes, this remains cheaper than adding more convolutional blocks.
 
 Naturally, you are able to set the type of pooling you wish to use when training a model, whether from scratch or using transfer learning.
 
-The MiniResNet backbones provided in the model zoo are pretrained on [FSD50K](https://zenodo.org/records/4060432)
-
+The MiniResNetv2 backbones provided in the model zoo are pretrained on [FSD50K](https://zenodo.org/records/4060432)
 
 Source implementation : https://keras.io/api/applications/resnet/
 
 Papers : [ResNet](https://arxiv.org/abs/1512.03385)
+         [ResNetv2](https://arxiv.org/abs/1603.05027)
 
 ## Network information
 
@@ -35,8 +36,8 @@ Papers : [ResNet](https://arxiv.org/abs/1512.03385)
 | Network Information     |  Value          |
 |-------------------------|-----------------|
 |  Framework              | TensorFlow Lite |
-|  Params 1 stack          | 135K            |
-|  Params 2 stacks         | 450K            |
+|  Params 1 stack          | 125K            |
+|  Params 2 stacks         | 440K            |
 |  Quantization           | int8            |
 |  Provenance             | https://keras.io/api/applications/resnet/ |
 
@@ -45,7 +46,7 @@ The pre-trained networks expects patches of shape (64, 50, 1), with 64 mels and
 
 When training from scratch, you can specify whichever input shape you desire.
 
-It outputs embedding vectors of size 2048 for the 2 stacks version, and 3548 for the 1 stack version. If you use the model zoo scripts to perform transfer learning or training from scratch, a classification head with the specified number of classes will automatically be added to the network.
+It outputs embedding vectors of size 2048 for the 2 stacks version, and 3548 for the 1 stack version. If you use the train.py script to perform transfer learning or training from scratch, a classification head with the specified number of classes will automatically be added to the network.
 
 ## Recommended platforms
 
@@ -58,26 +59,27 @@ It outputs embedding vectors of size 2048 for the 2 stacks version, and 3548 for
 
 ## Metrics
 
+* Measures are done with default STEdgeAI Core configuration with enabled input / output allocated option.
 
-Measures are done with default STEdgeAI Core configuration with enabled input / output allocated option.
+* `tl` stands for "transfer learning", meaning that the model backbone weights were initialized from a pre-trained model, then only the last layer was unfrozen during the training.
 
 
 ### Reference MCU memory footprint based on ESC-10 dataset
 
 
-| Model             | Format | Resolution | Series  | Activation RAM (KiB) | Runtime RAM (KiB)| Weights Flash (KiB) | Code Flash (KiB) | Total RAM (KiB) | Total Flash (KiB)| STEdgeAI Core version  |
+| Model             | Format | Resolution | Series  | Activation RAM (KiB) | Runtime RAM (KiB) | Weights Flash (KiB) | Code Flash (KiB) | Total RAM (KiB)  | Total Flash (kB) | STEdgeAI Core version  |
 |-------------------|--------|------------|---------|----------------|-------------|---------------|------------|-------------|-------------|-----------------------|
-| [MiniResNet 1 stack ](https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/audio_event_detection/miniresnetv1/esc10/miniresnetv1_s1_64x50_tl/miniresnetv1_s1_64x50_tl_int8.tflite) | int8 | 64x50x1 | B-U585I-IOT02A    | 59.89 | 1.08               |   123.6       |   32.36            | 60.97 | 155.96 | 3.0.0   |
-| [MiniResNet 2 stacks ](https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/audio_event_detection/miniresnetv1/esc10/miniresnetv1_s2_64x50_tl/miniresnetv1_s2_64x50_tl_int8.tflite) | int8 | 64x50x1 | B-U585I-IOT02A    | 59.89 |   	1.69      |   431.1           |   	36.81    | 61.58 | 467.91 | 3.0.0     | 
+| [miniresnet v2 1 stack ](https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/audio_event_detection/miniresnetv2/esc10/miniresnetv2_s1_64x50_tl/miniresnetv2_s1_64x50_tl_int8.tflite) | int8 | 64x50x1 | B-U585I-IOT02A    | 59.89        |   2.84       |   	123.98       |   42.76     | 62.73| 166.74 | 3.0.0   |
+| [miniresnet v2 2 stacks ](https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/audio_event_detection/miniresnetv2/esc10/miniresnetv2_s2_64x50_tl/miniresnetv2_s2_64x50_tl_int8.tflite) | int8 | 64x50x1 | B-U585I-IOT02A    | 59.89      |   	4.59        |   431.98       |   49.22   | 64.48 | 	481.2 | 3.0.0  |
 
 
 ### Reference inference time based on ESC-10 dataset
 
 
-| Model             | Format | Resolution | Board            | Execution Engine | Frequency   | Inference time (ms) | STEdgeAI Core version  |
-|-------------------|--------|------------|------------------|------------------|-------------|-----------------|-----------------------|
-| [MiniResNet 1 stack ](https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/audio_event_detection/miniresnetv1/esc10/miniresnetv1_s1_64x50_tl/miniresnetv1_s1_64x50_tl_int8.tflite) | int8 | 64x50x1 | B-U585I-IOT02A | 1 CPU | 160 MHz | 91.45 | 3.0.0   |
-| [MiniResNet 2 stacks ](https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/audio_event_detection/miniresnetv1/esc10/miniresnetv1_s2_64x50_tl/miniresnetv1_s2_64x50_tl_int8.tflite) | int8 | 64x50x1 | B-U585I-IOT02A | 1 CPU | 160 MHz | 141.82 | 3.0.0   |
+| Model             | Format | Resolution | Board            | Execution Engine |  Frequency   | Inference time (ms) | STEdgeAI Core version  |
+|-------------------|--------|------------|------------------|------------------|--------------|-------|-----------------------|
+| [miniresnet v2 1 stack ](https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/audio_event_detection/miniresnetv2/esc10/miniresnetv2_s1_64x50_tl/miniresnetv2_s1_64x50_tl_int8.tflite) | int8 | 64x50x1 | B-U585I-IOT02A |  1 CPU | 160 | 	187.26 | 3.0.0                |
+| [miniresnet v2 2 stacks ](https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/audio_event_detection/miniresnetv2/esc10/miniresnetv2_s2_64x50_tl/miniresnetv2_s2_64x50_tl_int8.tflite) | int8 | 64x50x1 | B-U585I-IOT02A |  1 CPU | 160 | 	307.34 | 3.0.0                 |
 
 
 ### Accuracy with ESC-10 dataset
@@ -88,10 +90,12 @@ The reason this metric is used instead of patch-level accuracy is because patch-
 
 | Model | Format | Resolution | Clip-level Accuracy |
 |-------|--------|------------|----------------|
-| [MiniResNet 1 stack ](https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/audio_event_detection/miniresnetv1/esc10/miniresnetv1_s1_64x50_tl/miniresnetv1_s1_64x50_tl.keras) | float32 | 64x50x1 | 90.0% |
-| [MiniResNet 1 stack ](https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/audio_event_detection/miniresnetv1/esc10/miniresnetv1_s1_64x50_tl/miniresnetv1_s1_64x50_tl_int8.tflite) | int8 | 64x50x1 | 90.0% |
-| [MiniResNet 2 stacks ](https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/audio_event_detection/miniresnetv1/esc10/miniresnetv1_s2_64x50_tl/miniresnetv1_s2_64x50_tl.keras) | float32 | 64x50x1 | 92.5% |
-| [MiniResNet 2 stacks ](https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/audio_event_detection/miniresnetv1/esc10/miniresnetv1_s2_64x50_tl/miniresnetv1_s2_64x50_tl_int8.tflite) | int8 | 64x50x1 | 92.5% |
+| [miniresnet v2 1 stack ](https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/audio_event_detection/miniresnetv2/esc10/miniresnetv2_s1_64x50_tl/miniresnetv2_s1_64x50_tl.keras) | float32 | 64x50x1 | 91.25% |
+| [miniresnet v2 1 stack ](https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/audio_event_detection/miniresnetv2/esc10/miniresnetv2_s1_64x50_tl/miniresnetv2_s1_64x50_tl_int8.tflite) | int8 | 64x50x1 | 92.5% |
+| [miniresnet v2 2 stacks ](https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/audio_event_detection/miniresnetv2/esc10/miniresnetv2_s2_64x50_tl/miniresnetv2_s2_64x50_tl.keras) | float32 | 64x50x1 | 93.75% |
+| [miniresnet v2 2 stacks ](https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/audio_event_detection/miniresnetv2/esc10/miniresnetv2_s2_64x50_tl/miniresnetv2_s2_64x50_tl_int8.tflite) | int8 | 64x50x1 | 93.75% |
+
+
 
 ## Retraining and Integration in a simple example: