README.md

---
license: mit
language:
- en
base_model:
- tianweiy/CenterPoint
pipeline_tag: object-detection
tags:
- Axera
- NPU
- Pulsar2
- CenterPoint
- 3D-Object-Detection
- LiDAR
---

# CenterPoint on Axera NPU

This repository contains the [CenterPoint](https://arxiv.org/abs/2006.11275) model converted for high-performance inference on the Axera NPU. CenterPoint is a center-based framework for 3D object detection and tracking that represents objects as points, significantly simplifying the detection pipeline on LiDAR point clouds.

This version is optimized with **w8a16** quantization and is compatible with **Pulsar2 version 4.2**.

## Convert Tools Links

For model conversion and deployment guidance:
- [AXera Platform GitHub Repo](https://github.com/AXERA-TECH/centerpoint.axera): Sample code and optimization guides for Axera NPU.
- [Pulsar2 Documentation](https://pulsar2-docs.readthedocs.io/en/latest/pulsar2/introduction.html): Guide for converting ONNX models to `.axmodel`.

## Support Platforms

- **AX650**
  - [M4N-Dock (爱芯派Pro)](https://wiki.sipeed.com/hardware/zh/maixIV/m4ndock/m4ndock.html)
  - [M.2 Accelerator card](https://docs.m5stack.com/zh_CN/ai_hardware/LLM-8850_Card)

| Chips | Model Variant  | NPU3 Latency (ms) |
|---|---|---|
| AX650 | CenterPoint-Pillar |  88.334 |

## How to Use

Download the repository and ensure the directory structure is organized as follows:

```text
.
├── centerpoint.axmodel       # The compiled Axera model
├── inference_axmodel.py      # Main inference script
└── extracted_data/           # Input directory
    ├── config.json           # Configuration files (e.g., inference_config.json)
    ├── sample_index.json
    ├── gt_annotations/
    └── points/                 
```

### Prerequisites

1.  **Environment:** Ensure you have the required Python environment activated with the following core packages installed:
    * **NPU Runtime:** `axengine` (PyAXEngine)
    * **Core Libraries:** `numba` , `opencv-python` and `tqdm`.

2.  **Model/Data:** Ensure the compiled `.axmodel`, `inference_config.json`, and input data (`inference_data/`) are available on the host.

### Inference Command

Run the inference script by providing the compiled model, configuration, and data directory.

```bash
python inference_axmodel.py ./centerpoint.axmodel ./extracted_data/config.json ./extracted_data --output-dir ./inference_results --visualize --num-samples 50 --score-thr 0.5

```

### Inference with AX650 Host

### Results
The model generates a 3D detection map with bounding boxes oriented in 3D space. Results are saved as images and videos which visualize the ego-vehicle, point cloud data, and detected objects.
```
(ax_env) root@ax650:~/data# python inference_axmodel.py ./centerpoint.axmodel ./extracted_data/config.json ./extracted_data --output-dir ./inference_results --visualize --num-samples 50 --score-thr 0.5
[INFO] Available providers:  ['AxEngineExecutionProvider']
[INFO] Using provider: AxEngineExecutionProvider
[INFO] Chip type: ChipType.MC50
[INFO] VNPU type: VNPUType.DISABLED
[INFO] Engine version: 2.12.0s
[INFO] Model type: 2 (triple core)
[INFO] Compiler version: 5.1-patch1 ed388aa0
Processing 50 samples...
Inference: 100%|████████████████████████████████████████████████████████████████████████| 50/50 [00:47<00:00,  1.06it/s]
Creating video: 100%|███████████████████████████████████████████████████████████████████| 50/50 [00:02<00:00, 23.32it/s]
Done! 50 frames, 12836 detections, saved to ./inference_results
```

### Example Visualization
![CenterPoint Detection Result GIF](output.gif)