BEVDet for Autoware (tensorrt_bevdet)

Multi-view camera 3D object detection model, used by the autoware_tensorrt_bevdet node in Autoware.

The model follows the BEVDet architecture: it unifies six surround-view camera images into a bird's-eye-view (BEV) representation for the 3D object detection task, deployed with the BEVPoolv2 [1] TensorRT/C++ implementation. It is distributed as ONNX; Autoware builds the TensorRT engine from the ONNX file on first launch.

Model overview

Task	3D object detection (oriented bounding boxes) from 6 surround-view camera images
Architecture	BEVDet (configuration bevdet_r50_4dlongterm_depth: depth-aware BEV pooling with temporal fusion of 8 adjacent frames)
Detected classes	car, truck, construction_vehicle, bus, trailer, barrier, motorcycle, bicycle, pedestrian, traffic_cone
Cameras	CAM_FRONT_LEFT, CAM_FRONT, CAM_FRONT_RIGHT, CAM_BACK_LEFT, CAM_BACK, CAM_BACK_RIGHT
Network input size	256 x 704 (resized/cropped from 900 x 1600 source images)
BEV range	[-51.2, -51.2, -5.0, 51.2, 51.2, 3.0] m, 0.8 m grid
Runtime	TensorRT (FP16 by default, FP32 selectable) via the autoware_tensorrt_bevdet ROS 2 node
Format	ONNX (Autoware builds the TensorRT engine locally on first launch)
License	Apache-2.0 (see Legal Notice for training-data terms)

The node also requires two configuration files that ship inside the autoware_tensorrt_bevdet package, not in this repository:

• config/bevdet.param.yaml: ROS parameters (precision, score threshold, class names, camera list)
• config/bevdet_r50_4dlongterm_depth.yaml: model configuration (BEV range, grid, image pre-processing, NMS)

Files

File	Description
bevdet_one_lt_d.onnx	BEVDet network (long-term temporal fusion, depth branch), all six camera views
deploy_metadata.yaml	Deployment metadata recording the artifact version of this repository

TensorRT engines are not distributed here. TensorRT engines are specific to the GPU architecture and TensorRT version they are built on and are not portable, so Autoware builds them locally from the ONNX file on first launch (the node appends _fp16.engine or _fp32.engine to the configured engine path depending on the precision, so with the default config the engine lands next to the ONNX file).

Inputs and outputs (as used by the node)

Inputs: six sensor_msgs/msg/Image topics (~/input/topic_img_front_left, ~/input/topic_img_front, ~/input/topic_img_front_right, ~/input/topic_img_back_left, ~/input/topic_img_back, ~/input/topic_img_back_right) plus the matching six sensor_msgs/msg/CameraInfo topics.

Outputs:

• ~/output/boxes (autoware_perception_msgs/msg/DetectedObjects): detected 3D objects
• ~/output_bboxes (visualization_msgs/msg/MarkerArray): bounding-box markers for nuScenes visualization, published when debug_mode is enabled

Usage in Autoware

The node loads the model from ~/autoware_data/ml_models/tensorrt_bevdet/ by default and launches with:

ros2 launch autoware_tensorrt_bevdet tensorrt_bevdet.launch.xml

Key launch arguments: model_name (default bevdet_one_lt_d), model_path (default $HOME/autoware_data/ml_models/tensorrt_bevdet), model_config, and param_file. Precision (fp16/fp32) and debug_mode are set in config/bevdet.param.yaml. The package README documents TensorRT 10.8.0.43 and CUDA 12.4 as prerequisites, and describes how to test the node against nuScenes data played back with ros2_dataset_bridge. See the package README for the full instructions.

Training

The model was trained on the nuScenes dataset for 20 epochs. Training, export, and deployment code:

• Original BEVDet: https://github.com/HuangJunJie2017/BEVDet/tree/dev2.1
• TensorRT C++ implementation: https://github.com/LCH1238/bevdet-tensorrt-cpp/tree/one
• ONNX export fork: https://github.com/LCH1238/BEVDet/tree/export
• Training/export fork adapted to the TIER IV dataset: https://github.com/cyn-liu/BEVDet/tree/train_export
• Autoware vendor package (modified TensorRT implementation): https://github.com/autowarefoundation/bevdet_vendor

Limitations

• Trained only on nuScenes; the package README notes poor generalization to other data. To use this model on your own vehicle and camera setup, retraining is required (see the train_export fork above).
• Expects a six-camera surround-view rig matching the nuScenes camera layout.
• Only the ten nuScenes classes listed above are detected.

Provenance and versioning

Original source	tensorrt_bevdet.tar.gz from the Autoware artifacts S3 bucket (https://autoware-files.s3.us-west-2.amazonaws.com/models/tensorrt_bevdet.tar.gz), unversioned
This repository	AutowareFoundation/tensorrt_bevdet, extracted ONNX published as tag v1.0

Citation

@article{huang2022bevpoolv2,
  title   = {BEVPoolv2: A Cutting-edge Implementation of BEVDet Toward Deployment},
  author  = {Huang, Junjie and Huang, Guan},
  journal = {arXiv preprint arXiv:2211.17111},
  year    = {2022}
}
@article{huang2021bevdet,
  title   = {BEVDet: High-performance Multi-camera 3D Object Detection in Bird-Eye-View},
  author  = {Huang, Junjie and Huang, Guan and Zhu, Zheng and Ye, Yun and Du, Dalong},
  journal = {arXiv preprint arXiv:2112.11790},
  year    = {2021}
}

References

• [1] Huang and Huang, "BEVPoolv2: A Cutting-edge Implementation of BEVDet Toward Deployment", arXiv:2211.17111, 2022.
• [2] Huang et al., "BEVDet: High-performance Multi-camera 3D Object Detection in Bird-Eye-View", arXiv:2112.11790, 2021.
• [3] nuScenes: https://www.nuscenes.org/nuscenes

Legal Notice

The nuScenes dataset is released publicly for non-commercial use under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International Public License. Additional Terms of Use can be found at https://www.nuscenes.org/terms-of-use. To inquire about a commercial license please contact nuscenes@motional.com.