File size: 3,413 Bytes
1765773 c3f9f17 d7e5460 1765773 c3f9f17 1765773 d7e5460 1765773 d7e5460 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 | ## Model Information
### Source model
- Input shape: [[1,6,3,480,800]]
- Number of parameters: 33.7M
- Model size: 128.5 FP32
- Output shape: boxes [[300,9]], scores [[300]], labels [[300]]
Source model repository: [BEVFormer](https://github.com/fundamentalvision/bevformer)
## Inference with AidLite SDK
### SDK installation
Model Farm uses AidLite SDK as the model inference SDK. For details, please refer to the [AidLite Developer Documentation](https://docs.aidlux.com/software/ai-sdk/aidlite_guide)
- Install AidLite SDK
```bash
# Install the appropriate version of the aidlite sdk
sudo aid-pkg update
sudo aid-pkg install aidlite-sdk
# Download the qnn version that matches the above backend. Eg Install QNN2.23 Aidlite: sudo aid-pkg install aidlite-qnn223
sudo aid-pkg install aidlite-{QNN VERSION}
```
- Verify AidLite SDK
```bash
# aidlite sdk c++ check
python3 -c "import aidlite ; print(aidlite.get_library_version())"
# aidlite sdk python check
python3 -c "import aidlite ; print(aidlite.get_py_library_version())"
```
### Environment Dependencies
Except for AidLite SDK, this BEVFormer demo also requires the following board-side Python environment.
#### Verified environment:
```bash
Python 3.10.12
numpy 1.26.4
opencv-python / cv2 4.13.0
Pillow / PIL 10.4.0
```
#### Required Python packages:
```bash
numpy
opencv-python
Pillow
```
#### Quick check:
```bash
python3 -c "import numpy, cv2; from PIL import Image; print('python env ok')"
```
The demo should be run on the QCS8550 / HTP v73 board environment. A normal host or development container can only run dry-run package checks and cannot execute real QNN inference without AidLite.
### Run Demo
#### python
```bash
cd /home/aidlux/bevformer_delivery_demo_2026_07_08
python3 code/python/run_test.py \
--backbone_model ./models/QCS8550/FP16/backbone_context.bin.aidem \
--scene_start_encoder_model ./models/QCS8550/FP16/scene_start_encoder_context.bin.aidem \
--encoder_model ./models/QCS8550/FP16/temporal_encoder_context.bin.aidem \
--decoder_model ./models/QCS8550/FP16/decoder_context.bin.aidem \
--asset_manifest ./code/python/datasets/sample4/asset_manifest.json \
--nms_contract ./code/python/configs/nms_runtime_contract.json \
--model_type QNN240 \
--invoke_nums 4 \
--output_dir ./outputs/final_sample4
```
[model_file_path] is the path of the QNN240 context binary file.
For this BEVFormer demo, the model is split into four QNN240 context files:
```bash
./models/QCS8550/FP16/backbone_context.bin.aidem
./models/QCS8550/FP16/scene_start_encoder_context.bin.aidem
./models/QCS8550/FP16/temporal_encoder_context.bin.aidem
./models/QCS8550/FP16/decoder_context.bin.aidem
```
The demo input is specified by:
```bash
./code/python/datasets/sample4/asset_manifest.json
```
The manifest describes four continuous BEVFormer frames. Each frame contains six raw camera JPG images and the required calibration / temporal auxiliary tensors. The demo starts from original six-camera images, not preprocessed image tensors.
Frame execution order:
```bash
frame 000: scene-start encoder
frame 001: temporal encoder
frame 002: temporal encoder
frame 003: temporal encoder
```
The output files are written to:
```bash
./outputs/final_sample4
```
Main output files include:
```bash
bevformer_demo_summary.json
run.log
frameXXX_final_coordinates.npz
frameXXX_camera_grid.png
sample4_camera_grid.gif
```
|