Model Information

Source model

Input shape: [[1,6,3,480,800]]
Number of parameters: 33.7M
Model size: 128.5 FP32
Output shape: boxes [[300,9]], scores [[300]], labels [[300]]

Source model repository: BEVFormer

Inference with AidLite SDK

SDK installation

Model Farm uses AidLite SDK as the model inference SDK. For details, please refer to the AidLite Developer Documentation

Install AidLite SDK

# Install the appropriate version of the aidlite sdk
sudo aid-pkg update
sudo aid-pkg install aidlite-sdk
# Download the qnn version that matches the above backend. Eg Install QNN2.23 Aidlite: sudo aid-pkg install aidlite-qnn223
sudo aid-pkg install aidlite-{QNN VERSION}

Verify AidLite SDK

# aidlite sdk c++ check
python3 -c "import aidlite ; print(aidlite.get_library_version())"

# aidlite sdk python check
python3 -c "import aidlite ; print(aidlite.get_py_library_version())"

Environment Dependencies

Except for AidLite SDK, this BEVFormer demo also requires the following board-side Python environment.

Verified environment:

Python 3.10.12
numpy 1.26.4
opencv-python / cv2 4.13.0
Pillow / PIL 10.4.0

Required Python packages:

numpy
opencv-python
Pillow

Quick check:

python3 -c "import numpy, cv2; from PIL import Image; print('python env ok')"

The demo should be run on the QCS8550 / HTP v73 board environment. A normal host or development container can only run dry-run package checks and cannot execute real QNN inference without AidLite.

Run Demo

python

cd /home/aidlux/bevformer_delivery_demo_2026_07_08

python3 code/python/run_test.py \
  --backbone_model ./models/QCS8550/FP16/backbone_context.bin.aidem \
  --scene_start_encoder_model ./models/QCS8550/FP16/scene_start_encoder_context.bin.aidem \
  --encoder_model ./models/QCS8550/FP16/temporal_encoder_context.bin.aidem \
  --decoder_model ./models/QCS8550/FP16/decoder_context.bin.aidem \
  --asset_manifest ./code/python/datasets/sample4/asset_manifest.json \
  --nms_contract ./code/python/configs/nms_runtime_contract.json \
  --model_type QNN240 \
  --invoke_nums 4 \
  --output_dir ./outputs/final_sample4

[model_file_path] is the path of the QNN240 context binary file.

For this BEVFormer demo, the model is split into four QNN240 context files:

./models/QCS8550/FP16/backbone_context.bin.aidem
./models/QCS8550/FP16/scene_start_encoder_context.bin.aidem
./models/QCS8550/FP16/temporal_encoder_context.bin.aidem
./models/QCS8550/FP16/decoder_context.bin.aidem

The demo input is specified by:

./code/python/datasets/sample4/asset_manifest.json

The manifest describes four continuous BEVFormer frames. Each frame contains six raw camera JPG images and the required calibration / temporal auxiliary tensors. The demo starts from original six-camera images, not preprocessed image tensors.

Frame execution order:

frame 000: scene-start encoder
frame 001: temporal encoder
frame 002: temporal encoder
frame 003: temporal encoder

The output files are written to:

./outputs/final_sample4

Main output files include:

bevformer_demo_summary.json
run.log
frameXXX_final_coordinates.npz
frameXXX_camera_grid.png
sample4_camera_grid.gif