Update README.md
Browse files
README.md
CHANGED
|
@@ -1,5 +1,91 @@
|
|
| 1 |
-
---
|
| 2 |
-
license: other
|
| 3 |
-
license_name: sla0081
|
| 4 |
-
license_link: https://github.com/STMicroelectronics/stm32ai-modelzoo/hand_posture/LICENSE.md
|
| 5 |
-
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: other
|
| 3 |
+
license_name: sla0081
|
| 4 |
+
license_link: https://github.com/STMicroelectronics/stm32ai-modelzoo/hand_posture/LICENSE.md
|
| 5 |
+
---
|
| 6 |
+
# CNN2D_ST_HandPosture model
|
| 7 |
+
|
| 8 |
+
## **Use case** : `Hand posture recognition`
|
| 9 |
+
|
| 10 |
+
# Model description
|
| 11 |
+
|
| 12 |
+
CNN2D_ST_HandPosture is a network topology designed by ST Teams to solve basic Hand Posture recognition use cases based on ST multi-zone Time-of-Flight sensor data. It is a convolutional neural network based model before feeding the data to the fully-connected (Dense) layer. It uses the distance and signal per spad 8x8 data. This is a very light model with very small foot prints in terms of FLASH and RAM as well as computational requirements.
|
| 13 |
+
|
| 14 |
+
We recommend to use input size (8 x 8 x 2) but this network can support greater input size.
|
| 15 |
+
|
| 16 |
+
The only input required to the model is the input shape and the number of outputs.
|
| 17 |
+
|
| 18 |
+
In this folder you will find multiple copies of the CNN2D_ST_HandPosture model pretrained on a ST custom datasets - Please refer to th [stm32ai-modelzoo-services](https://github.com/STMicroelectronics/stm32ai-modelzoo-services) GitHub for more informations
|
| 19 |
+
|
| 20 |
+
## Network information (for 8 hand postures)
|
| 21 |
+
|
| 22 |
+
|
| 23 |
+
| Network Information | Value |
|
| 24 |
+
|:-----------------------:|:---------------:|
|
| 25 |
+
| Framework | TensorFlow |
|
| 26 |
+
| Params | 2,752 |
|
| 27 |
+
|
| 28 |
+
|
| 29 |
+
## Network inputs / outputs
|
| 30 |
+
|
| 31 |
+
|
| 32 |
+
For an Time of Flight frame resolution of 8x8 and P classes
|
| 33 |
+
|
| 34 |
+
| Input Shape | Description |
|
| 35 |
+
| :----:| :-----------: |
|
| 36 |
+
| (N, 8, 8, 2) | Batch ( 8 x 8 x 2 ) matrix of Time of Flight values (distance, signal per spad) for a 8x8 frame in FLOAT32.|
|
| 37 |
+
|
| 38 |
+
| Output Shape | Description |
|
| 39 |
+
| :----:| :-----------: |
|
| 40 |
+
| (N, P) | Batch of per-class confidence for P classes in FLOAT32|
|
| 41 |
+
|
| 42 |
+
|
| 43 |
+
## Recommended platforms
|
| 44 |
+
|
| 45 |
+
|
| 46 |
+
| Platform | Supported | Recommended |
|
| 47 |
+
|:--------:|:---------:|:-----------:|
|
| 48 |
+
| STM32F4 | [x] | [x] |
|
| 49 |
+
| STM32L4 | [x] | [x] |
|
| 50 |
+
| STM32U5 | [x] | [] |
|
| 51 |
+
|
| 52 |
+
|
| 53 |
+
# Performances
|
| 54 |
+
|
| 55 |
+
## Metrics
|
| 56 |
+
|
| 57 |
+
Measures are done with default STM32Cube.AI configuration with enabled input / output allocated option.
|
| 58 |
+
|
| 59 |
+
|
| 60 |
+
### Reference memory footprint based on ST_VL53LxCX_handposture_dataset (see Accuracy for details on dataset)
|
| 61 |
+
|
| 62 |
+
| Model | Format | Input Shape | Series | Activation RAM (KiB) | Runtime RAM (KiB) | Weights Flash (KiB) | Code Flash (KiB) | Total RAM (KiB) | Total Flash (KiB) | STM32Cube.AI version |
|
| 63 |
+
|:-----------------:|:------:|:-----------:|:-------:|:--------------:|:-----------:|:-------------:|:----------:|:-----------:|:-----------:|:---------------------:|
|
| 64 |
+
| [CNN2D_ST_HandPosture](https://github.com/STMicroelectronics/stm32ai-modelzoo/ST_pretrainedmodel_custom_dataset/ST_VL53L8CX_handposture_dataset/CNN2D_ST_HandPosture_8classes/CNN2D_ST_HandPosture_8classes.h5) | FLOAT32 | 8 x 8 x 2 | STM32F4 | 1.07 | 2.08 | 10.75 | 14.37 | 3.15 | 25.12 | 10.0.0 |
|
| 65 |
+
| [CNN2D_ST_HandPosture](https://github.com/STMicroelectronics/stm32ai-modelzoo/ST_pretrainedmodel_custom_dataset/ST_VL53L5CX_handposture_dataset/CNN2D_ST_HandPosture_8classes/CNN2D_ST_HandPosture_8classes.h5) | FLOAT32 | 8 x 8 x 2 | STM32F4 | 1.07 | 2.08 | 10.75 | 14.37 | 3.15 | 25.12 | 10.0.0 |
|
| 66 |
+
|
| 67 |
+
|
| 68 |
+
### Reference inference time based on ST_VL53LxCX_handposture_dataset (see Accuracy for details on dataset)
|
| 69 |
+
|
| 70 |
+
|
| 71 |
+
| Model | Format | Resolution | Board | Frequency | Inference time (ms) | STM32Cube.AI version |
|
| 72 |
+
|:-----------------:|:------:|:----------:|:----------------:|:-------------:|:-------------------:|:---------------------:|
|
| 73 |
+
| [CNN2D_ST_HandPosture](https://github.com/STMicroelectronics/stm32ai-modelzoo/ST_pretrainedmodel_custom_dataset/ST_VL53L8CX_handposture_dataset/CNN2D_ST_HandPosture_8classes/CNN2D_ST_HandPosture_8classes.h5) | FLOAT32 | 8 x 8 x 2 | STM32F401 | 84 MHz | 1.54 ms | 10.0.0 |
|
| 74 |
+
| [CNN2D_ST_HandPosture](https://github.com/STMicroelectronics/stm32ai-modelzoo/ST_pretrainedmodel_custom_dataset/ST_VL53L5CX_handposture_dataset/CNN2D_ST_HandPosture_8classes/CNN2D_ST_HandPosture_8classes.h5) | FLOAT32 | 8 x 8 x 2 | STM32F401 | 84 MHz | 1.53 ms | 10.0.0 |
|
| 75 |
+
|
| 76 |
+
### Accuracy with ST_VL53LxCX_handposture_dataset
|
| 77 |
+
|
| 78 |
+
Number of classes: 8 [None, FlatHand, Like, Dislike, Fist, Love, BreakTime, CrossHands]. Training dataset number of frames: 3,031. Test dataset number of frames: 1146.
|
| 79 |
+
|
| 80 |
+
|
| 81 |
+
| Model | Format | Resolution | Accuracy |
|
| 82 |
+
|:-----------------:|:------:|:----------:|:----------------:|
|
| 83 |
+
| [CNN2D_ST_HandPosture](https://github.com/STMicroelectronics/stm32ai-modelzoo/ST_pretrainedmodel_custom_dataset/ST_VL53L8CX_handposture_dataset/CNN2D_ST_HandPosture_8classes/CNN2D_ST_HandPosture_8classes.h5) | FLOAT32 | 8 x 8 x 2 | 99.43 % |
|
| 84 |
+
| [CNN2D_ST_HandPosture](https://github.com/STMicroelectronics/stm32ai-modelzoo/ST_pretrainedmodel_custom_dataset/ST_VL53L5CX_handposture_dataset/CNN2D_ST_HandPosture_8classes/CNN2D_ST_HandPosture_8classes.h5) | FLOAT32 | 8 x 8 x 2 | 97.17 % |
|
| 85 |
+
|
| 86 |
+
|
| 87 |
+
## Retraining and Integration in a simple example:
|
| 88 |
+
|
| 89 |
+
Please refer to the stm32ai-modelzoo-services GitHub [here](https://github.com/STMicroelectronics/stm32ai-modelzoo-services)
|
| 90 |
+
|
| 91 |
+
|