README.md

---
license: apache-2.0
language:
- en
base_model:
- RTMPose-M
pipeline_tag: image-classification
tags:
- Axera
- RTMPose
- Pose Estimation
- Keypoint Detection
- SimCC
- OpenMMLab
---

# RTMPose-M

This version of **RTMPose-M** (256x192) has been converted to run on the Axera NPU using **mixed w16/fp32** quantization. It is optimized for real-time human pose estimation with 17 COCO keypoints using the SimCC decoding approach.

Compatible with Pulsar2 version: 6.0.

## Model Info

| Item | Value |
| :--- | :--- |
| **Architecture** | RTMPose-M (CSPNeXt + SimCC Head) |
| **Parameters** | 13.58M |
| **Input** | 1x256x192x3 (NHWC, uint8, BGR) |
| **Output** | simcc_x (1,17,384), simcc_y (1,17,512) |
| **Keypoints** | 17 (COCO format) |
| **Source** | [OpenMMLab MMPose](https://github.com/open-mmlab/mmpose) |

## Convert tools links:

For those who are interested in model conversion, you can try to export axmodel through:
- [The repo of AXera Platform](https://github.com/AXERA-TECH/ax-samples), where you can get the detailed guide.
- [Pulsar2 Link, How to Convert ONNX to axmodel](https://pulsar2-docs.readthedocs.io/en/latest/pulsar2/introduction.html)

## Support Platform

- **AX650N/AX8850**
  - [M4N-Dock(爱芯派Pro)](https://wiki.sipeed.com/hardware/zh/maixIV/m4ndock/m4ndock.html)
  - [M.2 Accelerator card](https://docs.m5stack.com/en/ai_hardware/LLM-8850_Card)

### Performance Statistics

#### AX650N

| Model | Latency(ms) npu3 |
| :--- | :---: |
| **rtmpose_m** | 2.881 |

## Conversion Pipeline

1. **Export ONNX** — Download official RTMPose-M from OpenMMLab and fix batch dim:

```bash
python export_onnx.py
```

2. **Replace HardSigmoid** — Replace HardSigmoid ops with Mul+Add+Clip for better NPU quantization:

```bash
python replace_hardsigmoid.py
```

3. **Compile axmodel** — Use Pulsar2 with the provided `config.json` to quantize and compile:

```bash
pulsar2 build --target_hardware AX650 --config config.json --input rtmpose_m_256x192_no_hs.onnx --output_dir AX650
```

## How to use

Download all files from this repository to the device.

### python env requirement

#### pyaxengine

https://github.com/AXERA-TECH/pyaxengine

```bash
wget https://github.com/AXERA-TECH/pyaxengine/releases/download/0.1.3.rc2/axengine-0.1.3-py3-none-any.whl
pip install axengine-0.1.3-py3-none-any.whl
```

### Inference with AX650 Host, such as M4N-Dock(爱芯派Pro)

Input image:

![](test.jpg)

run
```bash
python3 ax_infer.py -m rtmpose_m_npu3.axmodel -i test.jpg
```

```bash
root@ax650:~/data# python3 ax_infer.py -m rtmpose_m_npu3.axmodel -i test.jpg
[INFO] Available providers:  ['AxEngineExecutionProvider']
[INFO] Using provider: AxEngineExecutionProvider
[INFO] Chip type: ChipType.MC50
[INFO] VNPU type: VNPUType.DISABLED
[INFO] Engine version: 2.10.1s
[INFO] Model type: 2 (triple core)
[INFO] Compiler version: 6.0 93b95f7f
Model input: name=input, shape=[1, 256, 192, 3], dtype=uint8
Forward: 3.38 ms (avg of 10 runs)
simcc_x: shape=(1, 17, 384), range=[-0.58, 0.88]
simcc_y: shape=(1, 17, 512), range=[-0.49, 0.88]
kpts above 0.3: 17/17
  kp00: ( 359.6,   83.3)  score=0.6773
  kp01: ( 370.0,   79.2)  score=0.6950
  kp02: ( 359.6,   77.1)  score=0.6878
  kp03: ( 384.6,   79.2)  score=0.7398
  kp04: ( 359.6,   79.2)  score=0.6385
  kp05: ( 403.3,  106.3)  score=0.7596
  kp06: ( 367.9,  116.7)  score=0.7683
  kp07: ( 432.5,  152.1)  score=0.4699
  kp08: ( 342.9,  158.3)  score=0.6831
  kp09: ( 445.0,  177.1)  score=0.3021
  kp10: ( 305.4,  179.2)  score=0.5798
  kp11: ( 432.5,  212.5)  score=0.7872
  kp12: ( 399.2,  218.7)  score=0.8110
  kp13: ( 432.5,  289.6)  score=0.7358
  kp14: ( 372.1,  279.2)  score=0.8252
  kp15: ( 470.0,  356.2)  score=0.6704
  kp16: ( 399.2,  345.8)  score=0.8183
Saved: ax_result.jpg
```

Output image:

![](ax_result.jpg)