---
license: agpl-3.0
language:
- en
pipeline_tag: object-detection
tags:
- Axera
- YOLOv7-pose
- NPU
- Object Detection
- Keypoint Detection
---

# fall-axera

This version of **fall-axera** has been converted to run on the Axera NPU using **w8a16** quantization. It is trained with modified yolov7-pose model to detect bbox and 14 keypoints of human, and to determine whether a fall behavior is likely to occur.

## Supported Classes

This model is trained to detect the following classes:
1. **normal**
2. **fall**

## Supported keypoints
This model is trained to detect the following 14 keypoints:
```
"keypoints": { 0: "right shoulder", 1: "right elbow", 2: "right wrist", 3: "left shoulder", 4: "left elbow", 5: "left wrist", 6: "right hip", 7: "right knee", 8: "right ankle", 9: "left hip", 10: "left knee", 11: "left ankle", 12: "head tops", 13: "upper neck" }"
```

Compatible with Pulsar2 version: 5.2.

## Convert tools links:

For those who are interested in model conversion, you can try to export axmodel through:
- [The repo of AXera Platform](https://github.com/AXERA-TECH/ax-samples), where you can get the detailed guide.
- [Pulsar2 Link, How to Convert ONNX to axmodel](https://pulsar2-docs.readthedocs.io/en/latest/pulsar2/introduction.html)

## Support Platform

https://docs.m5stack.com/zh_CN/ai_hardware/AI_Pyramid-Pro

- **AX650N/AX8850**
  - [M4N-Dock(爱芯派Pro)](https://wiki.sipeed.com/hardware/zh/maixIV/m4ndock/m4ndock.html)
  - [AI Pyramid](https://docs.m5stack.com/zh_CN/ai_hardware/AI_Pyramid-Pro)
  - [M.2 Accelerator card](https://docs.m5stack.com/en/ai_hardware/LLM-8850_Card)

## How to use

Download all files from this repository to the device.

### python env requirement

#### pyaxengine

https://github.com/AXERA-TECH/pyaxengine

```bash
wget https://github.com/AXERA-TECH/pyaxengine/releases/download/0.1.3.rc2/axengine-0.1.3-py3-none-any.whl
pip install axengine-0.1.3-py3-none-any.whl
```

### Inference with AX650 Host, such as M4N-Dock(爱芯派Pro)

Input image:
![](fall4.png)

run
```
python3 axmodel_infer_fall.py
```

```
root@ax650:~/fall-axera# python3 axmodel_infer_fall.py
[INFO] Available providers:  ['AxEngineExecutionProvider', 'AXCLRTExecutionProvider']
[INFO] Using provider: AxEngineExecutionProvider
[INFO] Chip type: ChipType.MC50
[INFO] VNPU type: VNPUType.DISABLED
[INFO] Engine version: 2.12.0s
[INFO] Model type: 2 (triple core)
[INFO] Compiler version: 5.2 eccb31f5
class: fall left:281 top:396 right:734 bottom:629 conf: 74%
Result saved to axmodel_res.jpg

```

Output image:
![](axmodel_res.jpg)

### Extra
This example only shows the model's predicted bounding boxes and keypoints. You can further assist in determining human falls based on the physical information of the boxes and keypoints, or by adding tracking and action recognition models like st-gcn. From the experiments I have conducted, factors such as occlusion, direction of falling, camera angle, and even the scene (such as on the bed or on the floor) can affect the results of fall detection.