FRNet for Autoware (lidar_frnet)

LiDAR semantic segmentation models used by the autoware_lidar_frnet node in Autoware.

The models follow the FRNet (Frustum-Range Network) [1] architecture and perform per-point 3D semantic segmentation on LiDAR data (x, y, z, intensity). They are exported as ONNX and run with TensorRT inside Autoware; the node builds the TensorRT engine from the ONNX file on first launch.

Model overview

Task	3D semantic segmentation of LiDAR point clouds (per-point class ID and probability)
Architecture	FRNet (Frustum-Range Network)
Classes	27 semantic classes (see list below)
Runtime	TensorRT (FP16 by default) via the autoware_lidar_frnet ROS 2 node
Format	ONNX (Autoware builds the TensorRT engine locally on first launch)
License	Apache-2.0

Segmented classes (as defined in the ml_package_frnet_*.param.yaml files):

drivable_surface, other_flat_surface, sidewalk, manmade, vegetation, car, bus, emergency_vehicle, train, truck, tractor_unit, semi_trailer, construction_vehicle, forklift, kart, motorcycle, bicycle, pedestrian, personal_mobility, animal, pushable_pullable, traffic_cone, debris, stroller, other_stuff, noise+ghost_point, unknown

Pre-processing (range-image projection, frustum construction) and post-processing (per-point argmax, probability-based filtering, visualization coloring) run in the node, not in the ONNX graph.

Variants in this repository

FRNet uses range images, so each variant is tied to the field of view and resolution of a specific LiDAR sensor:

Variant	Sensor	FOV up / down [deg]	Frustum (W x H)	Interpolation (W x H)	Points (min / opt / max)
frnet_ot128	HESAI OT128	15.0 / -25.0	1024 x 128	4096 x 128	5000 / 80000 / 160000
frnet_qt128	HESAI QT128	52.6 / -52.6	256 x 128	1024 x 128	5000 / 60000 / 120000

Both variants share the same 27-class label set and color palette. The point count ranges define the dynamic TensorRT optimization profile for the input point cloud.

Files

File	Description
frnet_ot128.onnx	FRNet network, HESAI OT128 variant
frnet_qt128.onnx	FRNet network, HESAI QT128 variant
ml_package_frnet_ot128.param.yaml	Model parameters for frnet_ot128 (FOV, frustum/interpolation size, classes, palette, profile ranges)
ml_package_frnet_qt128.param.yaml	Model parameters for frnet_qt128 (FOV, frustum/interpolation size, classes, palette, profile ranges)
deploy_metadata.yaml	Deployment metadata recording the artifact version of this repository

TensorRT engines are not distributed here. TensorRT engines are specific to the GPU architecture and TensorRT version they are built on and are not portable, so Autoware builds them locally from the ONNX files on first launch (or via build_only:=true).

Inputs and outputs (as used by the node)

Input: ~/input/pointcloud (sensor_msgs/msg/PointCloud2). The node operates on the x, y, z and intensity fields and auto-detects the input layout on the first received message; supported layouts are XYZIRCAEDT, XYZIRADRT, XYZIRC and XYZI.

Outputs:

Topic	Type	Description
~/output/pointcloud/segmentation	sensor_msgs/msg/PointCloud2	XYZ cloud with class ID and probability fields
~/output/pointcloud/visualization	sensor_msgs/msg/PointCloud2	XYZ cloud with RGB field (palette colors)
~/output/pointcloud/filtered	sensor_msgs/msg/PointCloud2	Cloud with the configured filter classes removed (default: drivable_surface), in the requested filter.output_format

The node also publishes processing time and latency debug topics and /diagnostics.

Usage in Autoware

Autoware's setup tooling (the Ansible artifacts role) downloads these artifacts to ~/autoware_data/ml_models/lidar_frnet/. The node then loads frnet_<sensor_model>.onnx together with the matching ml_package_frnet_<sensor_model>.param.yaml from model_path:

ros2 launch autoware_lidar_frnet lidar_frnet.launch.xml \
  sensor_model:=ot128 \
  model_path:=$HOME/autoware_data/ml_models/lidar_frnet

sensor_model accepts ot128 (HESAI OT128) or qt128 (HESAI QT128). Add build_only:=true to build the TensorRT engine from the ONNX as a one-off pre-task. See the package README for the full parameter reference.

Training

The models were trained with AWML (https://github.com/tier4/AWML) on the T4Dataset, using approximately 16,000 frames (4,000 frames x 4 surrounding sensors), as documented in the package README.

• FRNet project page: https://xiangxu-0103.github.io/FRNet
• FRNet paper: arXiv:2312.04484

Provenance and versioning

HF tag	Original artifact source
v2.0	https://awf.ml.dev.web.auto/perception/models/frnet/v2/

The model and parameter files (frnet_*.onnx, ml_package_frnet_*.param.yaml) byte-match the artifacts previously hosted at the source path above; deploy_metadata.yaml is generated for this repository and has no upstream counterpart. Consumers should pin a tag (--revision v2.0), never main.

Limitations

• FRNet uses range images, so the input point cloud must be referenced to the sensor's origin.
• Each variant is tuned to its sensor's FOV and resolution; differences in your sensor's specifications (FOV, horizontal/vertical resolution) may affect performance.
• Only the two sensor variants above are provided; other LiDAR models require retraining or fine-tuning.

Citation

@article{xu2025frnet,
  title   = {FRNet: Frustum-Range Networks for Scalable LiDAR Segmentation},
  author  = {Xu, Xiang and Kong, Lingdong and Shuai, Hui and Liu, Qingshan},
  journal = {IEEE Transactions on Image Processing},
  volume  = {34},
  pages   = {2173--2186},
  year    = {2025},
  doi     = {10.1109/TIP.2025.3550011}
}

References

• [1] X. Xu, L. Kong, H. Shuai and Q. Liu, "FRNet: Frustum-Range Networks for Scalable LiDAR Segmentation", IEEE Transactions on Image Processing, vol. 34, pp. 2173-2186, 2025. arXiv:2312.04484.
• [2] AWML: https://github.com/tier4/AWML
• [3] FRNet project page: https://xiangxu-0103.github.io/FRNet