| --- |
| license: apache-2.0 |
| language: |
| - en |
| base_model: |
| - RTMPose-M |
| pipeline_tag: image-classification |
| tags: |
| - Axera |
| - RTMPose |
| - Pose Estimation |
| - Keypoint Detection |
| - SimCC |
| - OpenMMLab |
| --- |
| |
| # RTMPose-M |
|
|
| This version of **RTMPose-M** (256x192) has been converted to run on the Axera NPU using **mixed w16/fp32** quantization. It is optimized for real-time human pose estimation with 17 COCO keypoints using the SimCC decoding approach. |
|
|
| Compatible with Pulsar2 version: 6.0. |
|
|
| ## Model Info |
|
|
| | Item | Value | |
| | :--- | :--- | |
| | **Architecture** | RTMPose-M (CSPNeXt + SimCC Head) | |
| | **Parameters** | 13.58M | |
| | **Input** | 1x256x192x3 (NHWC, uint8, BGR) | |
| | **Output** | simcc_x (1,17,384), simcc_y (1,17,512) | |
| | **Keypoints** | 17 (COCO format) | |
| | **Source** | [OpenMMLab MMPose](https://github.com/open-mmlab/mmpose) | |
|
|
| ## Convert tools links: |
|
|
| For those who are interested in model conversion, you can try to export axmodel through: |
| - [The repo of AXera Platform](https://github.com/AXERA-TECH/ax-samples), where you can get the detailed guide. |
| - [Pulsar2 Link, How to Convert ONNX to axmodel](https://pulsar2-docs.readthedocs.io/en/latest/pulsar2/introduction.html) |
|
|
| ## Support Platform |
|
|
| - **AX650N/AX8850** |
| - [M4N-Dock(爱芯派Pro)](https://wiki.sipeed.com/hardware/zh/maixIV/m4ndock/m4ndock.html) |
| - [M.2 Accelerator card](https://docs.m5stack.com/en/ai_hardware/LLM-8850_Card) |
|
|
| ### Performance Statistics |
|
|
| #### AX650N |
|
|
| | Model | Latency(ms) npu3 | |
| | :--- | :---: | |
| | **rtmpose_m** | 2.881 | |
| |
| ## Conversion Pipeline |
| |
| 1. **Export ONNX** — Download official RTMPose-M from OpenMMLab and fix batch dim: |
| |
| ```bash |
| python export_onnx.py |
| ``` |
| |
| 2. **Replace HardSigmoid** — Replace HardSigmoid ops with Mul+Add+Clip for better NPU quantization: |
|
|
| ```bash |
| python replace_hardsigmoid.py |
| ``` |
|
|
| 3. **Compile axmodel** — Use Pulsar2 with the provided `config.json` to quantize and compile: |
|
|
| ```bash |
| pulsar2 build --target_hardware AX650 --config config.json --input rtmpose_m_256x192_no_hs.onnx --output_dir AX650 |
| ``` |
|
|
| ## How to use |
|
|
| Download all files from this repository to the device. |
|
|
| ### python env requirement |
|
|
| #### pyaxengine |
|
|
| https://github.com/AXERA-TECH/pyaxengine |
|
|
| ```bash |
| wget https://github.com/AXERA-TECH/pyaxengine/releases/download/0.1.3.rc2/axengine-0.1.3-py3-none-any.whl |
| pip install axengine-0.1.3-py3-none-any.whl |
| ``` |
|
|
| ### Inference with AX650 Host, such as M4N-Dock(爱芯派Pro) |
|
|
| Input image: |
|
|
|  |
|
|
| run |
| ```bash |
| python3 ax_infer.py -m rtmpose_m_npu3.axmodel -i test.jpg |
| ``` |
|
|
| ```bash |
| root@ax650:~/data# python3 ax_infer.py -m rtmpose_m_npu3.axmodel -i test.jpg |
| [INFO] Available providers: ['AxEngineExecutionProvider'] |
| [INFO] Using provider: AxEngineExecutionProvider |
| [INFO] Chip type: ChipType.MC50 |
| [INFO] VNPU type: VNPUType.DISABLED |
| [INFO] Engine version: 2.10.1s |
| [INFO] Model type: 2 (triple core) |
| [INFO] Compiler version: 6.0 93b95f7f |
| Model input: name=input, shape=[1, 256, 192, 3], dtype=uint8 |
| Forward: 3.38 ms (avg of 10 runs) |
| simcc_x: shape=(1, 17, 384), range=[-0.58, 0.88] |
| simcc_y: shape=(1, 17, 512), range=[-0.49, 0.88] |
| kpts above 0.3: 17/17 |
| kp00: ( 359.6, 83.3) score=0.6773 |
| kp01: ( 370.0, 79.2) score=0.6950 |
| kp02: ( 359.6, 77.1) score=0.6878 |
| kp03: ( 384.6, 79.2) score=0.7398 |
| kp04: ( 359.6, 79.2) score=0.6385 |
| kp05: ( 403.3, 106.3) score=0.7596 |
| kp06: ( 367.9, 116.7) score=0.7683 |
| kp07: ( 432.5, 152.1) score=0.4699 |
| kp08: ( 342.9, 158.3) score=0.6831 |
| kp09: ( 445.0, 177.1) score=0.3021 |
| kp10: ( 305.4, 179.2) score=0.5798 |
| kp11: ( 432.5, 212.5) score=0.7872 |
| kp12: ( 399.2, 218.7) score=0.8110 |
| kp13: ( 432.5, 289.6) score=0.7358 |
| kp14: ( 372.1, 279.2) score=0.8252 |
| kp15: ( 470.0, 356.2) score=0.6704 |
| kp16: ( 399.2, 345.8) score=0.8183 |
| Saved: ax_result.jpg |
| ``` |
|
|
| Output image: |
|
|
|  |
|
|
|
|
