OnePlanner v1.0

f5a5b86 2 days ago

11.1 kB

---
license: apache-2.0
pipeline_tag: robotics
tags:
  - autoware
  - ros2
  - autonomous-driving
  - planning
  - end-to-end
  - e2e
  - trajectory-generation
  - lidar
  - bird-eye-view
  - 3d-object-detection
  - tensorrt
  - onnx
  - oneplanner
---
# OnePlanner for Autoware (`oneplanner`)

End-to-end planning model for autonomous driving, run in [Autoware](https://github.com/autowarefoundation/autoware)
by the [`autoware_oneplanner`](https://github.com/autowarefoundation/autoware_oneplanner) ROS 2 node.
OnePlanner turns the LiDAR point cloud, the Lanelet2 vector map, the route, traffic-light states and the ego
state into a 6 s ego trajectory, at the LiDAR's rate. A frozen BEVFusion-L backbone turns each point cloud into
a bird's-eye-view (BEV) feature map, and the planner reads a temporal history of those maps together with the
vector-map, route and ego context.

<p align="center">
  <img src="media/oneplanner_scenes.png" width="100%" alt="Six scenes: crossing a large intersection on green, a right turn at a signalized intersection, a sweeping curve at 40 km/h, holding at a red light and turning left on green, stopping for a red light, and slowing behind traffic queued at a red light">
</p>

## Model overview

| | |
| --- | --- |
| Task | Ego trajectory planning: a 6 s trajectory at 0.1 s steps, a second (prior) trajectory and turn-indicator logits, plus 3D object detection from the backbone's head |
| Inputs | Concatenated LiDAR point cloud; 3 s of ego history and the ego state; the nearest lanes, route lanes, polygons and line strings within ±100 m with their traffic-light states and speed limits; the route goal |
| Architecture | BEVFusion-L LiDAR backbone (sparse-convolution voxel encoder with a detection head, 8.3M parameters, frozen) → 512 × 180 × 180 BEV feature map over ±122.4 m → planner over a history of 3 BEV maps 0.5 s apart and the vector-map context (27.3M parameters) |
| Runtime | TensorRT through the `autoware_oneplanner` node: the planner in fp32, the backbone in fp16 |
| Latency | About 13 ms per planning pass (median), about 9 ms of it planner inference, on an NVIDIA RTX PRO 6000 Blackwell Max-Q; the BEV feature extraction runs per LiDAR frame, outside the pass |
| Format | ONNX (opset 18). The backbone uses the sparse-convolution plugins of `autoware_tensorrt_plugins`; the planner needs no plugin. TensorRT engines are built locally |
| License | Apache-2.0 |

## Files

| File | Description |
| --- | --- |
| `oneplanner_planner.onnx` | The planner graph, with static shapes |
| `bevfusion_lidar_feature.onnx` | The BEVFusion-L LiDAR backbone: voxels in, the BEV feature map and undecoded detection proposals out |
| `ml_package_oneplanner.param.yaml` | The network description the node reads: input convention, tensor names, voxelization geometry, BEV history, detection thresholds, horizon and precision |
| `media/` | The scenes on this page |
| `replay.mp4` | An open-loop replay of two recorded drives, shown as this page's video preview |
| `deploy_metadata.yaml` | Deployment metadata recording the artifact version of this repository |
| `SHA256SUMS` | Checksums of the files above |

> **TensorRT engines are not distributed here.** Engines are specific to the GPU architecture and TensorRT
> version they are built on. The node builds one per graph on first launch and caches it beside the ONNX file
> (`<stem>.engine`, with a `<stem>.engine.identity` file that triggers a rebuild when anything changes), so the
> model directory must be writable. `build_only:=true` builds them and exits.
## Inputs and outputs
### ROS 2 interface (as used by the node)
**Inputs**
| Topic | Message type | Description |
| --- | --- | --- |
| `~/input/pointcloud` | `sensor_msgs/msg/PointCloud2` | Concatenated point cloud in `base_link`, Autoware `PointXYZIRC` layout |
| `~/input/odometry` | `nav_msgs/msg/Odometry` | Ego pose and twist |
| `~/input/acceleration` | `geometry_msgs/msg/AccelWithCovarianceStamped` | Ego acceleration |
| `~/input/steering` | `autoware_vehicle_msgs/msg/SteeringReport` | Steering tire angle |
| `~/input/velocity` | `autoware_vehicle_msgs/msg/VelocityReport` | Vehicle speed, used where the odometry twist is exactly zero |
| `~/input/traffic_signals` | `autoware_perception_msgs/msg/TrafficLightGroupArray` | Traffic-light recognition |
| `~/input/route` | `autoware_planning_msgs/msg/LaneletRoute` | Mission route |
| `~/input/vector_map` | `autoware_map_msgs/msg/LaneletMapBin` | Lanelet2 map |

**Outputs**

| Topic | Message type | Description |
| --- | --- | --- |
| `~/output/trajectory` | `autoware_planning_msgs/msg/Trajectory` | Ego trajectory in `map`: a point at t = 0, then one per 0.1 s for 6 s |
| `~/output/trajectories` | `autoware_internal_planning_msgs/msg/CandidateTrajectories` | The trajectory and the prior trajectory as candidates |
| `~/output/turn_indicators` | `autoware_vehicle_msgs/msg/TurnIndicatorsCommand` | Turn-indicator command from the model's logits, filtered by the node |
| `~/output/detected_objects` | `autoware_perception_msgs/msg/DetectedObjects` | Objects of the backbone's detection head, one message per LiDAR frame |
| `/diagnostics` | `diagnostic_msgs/msg/DiagnosticArray` | `inference_status` |

### Graphs

The node reads the planner graph's input manifest and fills every input; all coordinates are in the ego frame
of the planning pose, x forward, y left, metres. The
[model contract](https://github.com/autowarefoundation/autoware_oneplanner/blob/main/autoware_oneplanner/docs/model_contract.md)
specifies each tensor's content.

**Planner** (`oneplanner_planner.onnx`)

| Tensor | dtype / shape | Content |
| --- | --- | --- |
| `bev_feature_history` | float32 `[1, 3, 512, 180, 180]` | The last 3 BEV maps, 0.5 s apart, current first, each warped into the planning frame |
| `ego_current_state` | float32 `[1, 10]` | Ego pose in its own frame, velocity, acceleration, steering tire angle, yaw rate |
| `ego_agent_past` | float32 `[1, 31, 4]` | Ego poses over the last 3 s at 0.1 s |
| `lanes`, `lanes_speed_limit`, `lanes_has_speed_limit`, `lanes_on_route` | `[1, 140, 20, 33]`, `[1, 140, 1]`, bool `[1, 140, 1]`, `[1, 140, 2]` | The 140 nearest lanes: centerline, bounds, traffic-light state and line types per point; speed limit; route membership |
| `route_lanes`, `route_lanes_speed_limit`, `route_lanes_has_speed_limit` | `[1, 25, 20, 33]`, `[1, 25, 1]`, bool `[1, 25, 1]` | The route's next 25 lanes |
| `polygons` | float32 `[1, 32, 40, 4]` | Intersection areas and crosswalks |
| `line_strings` | float32 `[1, 128, 20, 4]` | Stop lines and road borders |
| `goal_pose` | float32 `[1, 4]` | Route goal |
| `ego_shape` | float32 `[1, 3]` | Wheel base, length, width |
| `sensor_latency` | float32 `[1, 1]` | Time from the LiDAR stamp to the planning time, in seconds |
| `turn_indicators` | int32 `[1, 31]` | Carried for a stable signature only; the node binds a constant |

| Output | dtype / shape | Content |
| --- | --- | --- |
| `trajectory` | float32 `[1, 60, 4]` | x, y, cos(yaw), sin(yaw) at 0.1 s … 6.0 s |
| `prior_trajectory` | float32 `[1, 60, 4]` | A second ego trajectory, published as a candidate |
| `turn_indicator_logits` | float32 `[1, 3]` | Logits of DISABLE, ENABLE_LEFT, ENABLE_RIGHT |

**Backbone** (`bevfusion_lidar_feature.onnx`)

| Tensor | dtype / shape | Content |
| --- | --- | --- |
| `voxels` (in) | float32 `[M, 32, 5]` | Points per voxel: x, y, z, intensity, time lag; voxel size 0.17 × 0.17 × 0.2 m over ±122.4 m, z −3…5 m |
| `coors` (in) | int32 `[M, 3]` | Voxel coordinates z, y, x |
| `num_points_per_voxel` (in) | int32 `[M]` | Valid points per voxel; M up to 256,000 |
| `bev_feature` (out) | float32 `[1, 512, 180, 180]` | The BEV feature map |
| `bbox_pred`, `score`, `label_pred` (out) | `[10, 500]`, `[500]`, int64 `[500]` | Undecoded detection proposals for car, truck, bus, bicycle, pedestrian, traffic cone and barrier |

## Usage in Autoware

Build [`autoware_oneplanner`](https://github.com/autowarefoundation/autoware_oneplanner) in an Autoware
workspace as its README describes (ROS 2, Autoware Core, an NVIDIA GPU with CUDA and TensorRT 10, and
`autoware_tensorrt_plugins` from Autoware Universe for the backbone). Then download the model to the node's
default model directory and launch it:

```bash
hf download AutowareFoundation/oneplanner --revision v1.0 --local-dir ~/autoware_data/oneplanner
ros2 launch autoware_oneplanner oneplanner.launch.xml \
  model_path:=$HOME/autoware_data/oneplanner model_name:=oneplanner
```

Pin `--revision` to a tag rather than relying on `main`, and add `--exclude "media/*" "replay.mp4"` to skip
the demo media. The first launch builds both TensorRT engines, which takes several minutes. To see the output in
Autoware's RViz layout, remap `output_trajectory:=/planning/trajectory`; to drive with it, wire it into your
planning stack's trajectory input behind Autoware's trajectory validation.

The node feeds the inputs the model was trained with, the `oneplanner_v1` input convention: the ego state as
recorded, the map conversion, the traffic-light selection and the temporal BEV history are specified in the
model contract, and the node refuses a model package that states another convention.

## Training

The planner was trained by TIER IV on driving data recorded in Japan; the data is not public. The BEVFusion-L
backbone was trained with [AWML](https://github.com/tier4/AWML) and kept frozen while the planner was trained.
Accuracy figures are measured on internal validation data and are not published.

## Limitations

- **Research model.** It is not a certified driving system and must not control a vehicle on public roads.
  Run it in simulation, on closed courses, or with a trained safety driver and independent safeguards
  downstream of the planner; see the node's
  [DISCLAIMER](https://github.com/autowarefoundation/autoware_oneplanner/blob/main/DISCLAIMER.md).
- **Other road users through the LiDAR only.** The planner reads no tracked or predicted objects: other road
  users reach it through the BEV features of the point cloud.
- **Sensor setup and domain.** Trained on the vehicles, LiDAR configuration and roads of its training data in
  Japan (left-hand traffic, Japanese traffic lights). Performance on other platforms and operational domains is
  not characterized.
- **Evaluation.** The video preview and the scenes on this page are open-loop replays of recorded drives: the
  plan is drawn against the path a person drove, not executed.

## Provenance

| | |
| --- | --- |
| Source | Exported to ONNX by the OnePlanner training code |
| This repository | `AutowareFoundation/oneplanner`, tag `v1.0` |

The graphs are the exported files, renamed to the node's default file names. `SHA256SUMS` lists their
checksums.

## References

- `autoware_oneplanner`: <https://github.com/autowarefoundation/autoware_oneplanner>
- Model contract: <https://github.com/autowarefoundation/autoware_oneplanner/blob/main/autoware_oneplanner/docs/model_contract.md>
- AWML: <https://github.com/tier4/AWML>
- Autoware: <https://github.com/autowarefoundation/autoware>

	---
	license: apache-2.0
	pipeline_tag: robotics
	tags:
	- autoware
	- ros2
	- autonomous-driving
	- planning
	- end-to-end
	- e2e
	- trajectory-generation
	- lidar
	- bird-eye-view
	- 3d-object-detection
	- tensorrt
	- onnx
	- oneplanner
	---

	# OnePlanner for Autoware (`oneplanner`)

	End-to-end planning model for autonomous driving, run in [Autoware](https://github.com/autowarefoundation/autoware)
	by the [`autoware_oneplanner`](https://github.com/autowarefoundation/autoware_oneplanner) ROS 2 node.
	OnePlanner turns the LiDAR point cloud, the Lanelet2 vector map, the route, traffic-light states and the ego
	state into a 6 s ego trajectory, at the LiDAR's rate. A frozen BEVFusion-L backbone turns each point cloud into
	a bird's-eye-view (BEV) feature map, and the planner reads a temporal history of those maps together with the
	vector-map, route and ego context.

	<p align="center">
	<img src="media/oneplanner_scenes.png" width="100%" alt="Six scenes: crossing a large intersection on green, a right turn at a signalized intersection, a sweeping curve at 40 km/h, holding at a red light and turning left on green, stopping for a red light, and slowing behind traffic queued at a red light">
	</p>

	## Model overview

	\| \| \|
	\| --- \| --- \|
	\| Task \| Ego trajectory planning: a 6 s trajectory at 0.1 s steps, a second (prior) trajectory and turn-indicator logits, plus 3D object detection from the backbone's head \|
	\| Inputs \| Concatenated LiDAR point cloud; 3 s of ego history and the ego state; the nearest lanes, route lanes, polygons and line strings within ±100 m with their traffic-light states and speed limits; the route goal \|
	\| Architecture \| BEVFusion-L LiDAR backbone (sparse-convolution voxel encoder with a detection head, 8.3M parameters, frozen) → 512 × 180 × 180 BEV feature map over ±122.4 m → planner over a history of 3 BEV maps 0.5 s apart and the vector-map context (27.3M parameters) \|
	\| Runtime \| TensorRT through the `autoware_oneplanner` node: the planner in fp32, the backbone in fp16 \|
	\| Latency \| About 13 ms per planning pass (median), about 9 ms of it planner inference, on an NVIDIA RTX PRO 6000 Blackwell Max-Q; the BEV feature extraction runs per LiDAR frame, outside the pass \|
	\| Format \| ONNX (opset 18). The backbone uses the sparse-convolution plugins of `autoware_tensorrt_plugins`; the planner needs no plugin. TensorRT engines are built locally \|
	\| License \| Apache-2.0 \|

	## Files

	\| File \| Description \|
	\| --- \| --- \|
	\| `oneplanner_planner.onnx` \| The planner graph, with static shapes \|
	\| `bevfusion_lidar_feature.onnx` \| The BEVFusion-L LiDAR backbone: voxels in, the BEV feature map and undecoded detection proposals out \|
	\| `ml_package_oneplanner.param.yaml` \| The network description the node reads: input convention, tensor names, voxelization geometry, BEV history, detection thresholds, horizon and precision \|
	\| `media/` \| The scenes on this page \|
	\| `replay.mp4` \| An open-loop replay of two recorded drives, shown as this page's video preview \|
	\| `deploy_metadata.yaml` \| Deployment metadata recording the artifact version of this repository \|
	\| `SHA256SUMS` \| Checksums of the files above \|

	> TensorRT engines are not distributed here. Engines are specific to the GPU architecture and TensorRT
	> version they are built on. The node builds one per graph on first launch and caches it beside the ONNX file
	> (`<stem>.engine`, with a `<stem>.engine.identity` file that triggers a rebuild when anything changes), so the
	> model directory must be writable. `build_only:=true` builds them and exits.

	## Inputs and outputs

	### ROS 2 interface (as used by the node)

	Inputs

	\| Topic \| Message type \| Description \|
	\| --- \| --- \| --- \|
	\| `~/input/pointcloud` \| `sensor_msgs/msg/PointCloud2` \| Concatenated point cloud in `base_link`, Autoware `PointXYZIRC` layout \|
	\| `~/input/odometry` \| `nav_msgs/msg/Odometry` \| Ego pose and twist \|
	\| `~/input/acceleration` \| `geometry_msgs/msg/AccelWithCovarianceStamped` \| Ego acceleration \|
	\| `~/input/steering` \| `autoware_vehicle_msgs/msg/SteeringReport` \| Steering tire angle \|
	\| `~/input/velocity` \| `autoware_vehicle_msgs/msg/VelocityReport` \| Vehicle speed, used where the odometry twist is exactly zero \|
	\| `~/input/traffic_signals` \| `autoware_perception_msgs/msg/TrafficLightGroupArray` \| Traffic-light recognition \|
	\| `~/input/route` \| `autoware_planning_msgs/msg/LaneletRoute` \| Mission route \|
	\| `~/input/vector_map` \| `autoware_map_msgs/msg/LaneletMapBin` \| Lanelet2 map \|

	Outputs

	\| Topic \| Message type \| Description \|
	\| --- \| --- \| --- \|
	\| `~/output/trajectory` \| `autoware_planning_msgs/msg/Trajectory` \| Ego trajectory in `map`: a point at t = 0, then one per 0.1 s for 6 s \|
	\| `~/output/trajectories` \| `autoware_internal_planning_msgs/msg/CandidateTrajectories` \| The trajectory and the prior trajectory as candidates \|
	\| `~/output/turn_indicators` \| `autoware_vehicle_msgs/msg/TurnIndicatorsCommand` \| Turn-indicator command from the model's logits, filtered by the node \|
	\| `~/output/detected_objects` \| `autoware_perception_msgs/msg/DetectedObjects` \| Objects of the backbone's detection head, one message per LiDAR frame \|
	\| `/diagnostics` \| `diagnostic_msgs/msg/DiagnosticArray` \| `inference_status` \|

	### Graphs

	The node reads the planner graph's input manifest and fills every input; all coordinates are in the ego frame
	of the planning pose, x forward, y left, metres. The
	[model contract](https://github.com/autowarefoundation/autoware_oneplanner/blob/main/autoware_oneplanner/docs/model_contract.md)
	specifies each tensor's content.

	Planner (`oneplanner_planner.onnx`)

	\| Tensor \| dtype / shape \| Content \|
	\| --- \| --- \| --- \|
	\| `bev_feature_history` \| float32 `[1, 3, 512, 180, 180]` \| The last 3 BEV maps, 0.5 s apart, current first, each warped into the planning frame \|
	\| `ego_current_state` \| float32 `[1, 10]` \| Ego pose in its own frame, velocity, acceleration, steering tire angle, yaw rate \|
	\| `ego_agent_past` \| float32 `[1, 31, 4]` \| Ego poses over the last 3 s at 0.1 s \|
	\| `lanes`, `lanes_speed_limit`, `lanes_has_speed_limit`, `lanes_on_route` \| `[1, 140, 20, 33]`, `[1, 140, 1]`, bool `[1, 140, 1]`, `[1, 140, 2]` \| The 140 nearest lanes: centerline, bounds, traffic-light state and line types per point; speed limit; route membership \|
	\| `route_lanes`, `route_lanes_speed_limit`, `route_lanes_has_speed_limit` \| `[1, 25, 20, 33]`, `[1, 25, 1]`, bool `[1, 25, 1]` \| The route's next 25 lanes \|
	\| `polygons` \| float32 `[1, 32, 40, 4]` \| Intersection areas and crosswalks \|
	\| `line_strings` \| float32 `[1, 128, 20, 4]` \| Stop lines and road borders \|
	\| `goal_pose` \| float32 `[1, 4]` \| Route goal \|
	\| `ego_shape` \| float32 `[1, 3]` \| Wheel base, length, width \|
	\| `sensor_latency` \| float32 `[1, 1]` \| Time from the LiDAR stamp to the planning time, in seconds \|
	\| `turn_indicators` \| int32 `[1, 31]` \| Carried for a stable signature only; the node binds a constant \|

	\| Output \| dtype / shape \| Content \|
	\| --- \| --- \| --- \|
	\| `trajectory` \| float32 `[1, 60, 4]` \| x, y, cos(yaw), sin(yaw) at 0.1 s … 6.0 s \|
	\| `prior_trajectory` \| float32 `[1, 60, 4]` \| A second ego trajectory, published as a candidate \|
	\| `turn_indicator_logits` \| float32 `[1, 3]` \| Logits of DISABLE, ENABLE_LEFT, ENABLE_RIGHT \|

	Backbone (`bevfusion_lidar_feature.onnx`)

	\| Tensor \| dtype / shape \| Content \|
	\| --- \| --- \| --- \|
	\| `voxels` (in) \| float32 `[M, 32, 5]` \| Points per voxel: x, y, z, intensity, time lag; voxel size 0.17 × 0.17 × 0.2 m over ±122.4 m, z −3…5 m \|
	\| `coors` (in) \| int32 `[M, 3]` \| Voxel coordinates z, y, x \|
	\| `num_points_per_voxel` (in) \| int32 `[M]` \| Valid points per voxel; M up to 256,000 \|
	\| `bev_feature` (out) \| float32 `[1, 512, 180, 180]` \| The BEV feature map \|
	\| `bbox_pred`, `score`, `label_pred` (out) \| `[10, 500]`, `[500]`, int64 `[500]` \| Undecoded detection proposals for car, truck, bus, bicycle, pedestrian, traffic cone and barrier \|

	## Usage in Autoware

	Build [`autoware_oneplanner`](https://github.com/autowarefoundation/autoware_oneplanner) in an Autoware
	workspace as its README describes (ROS 2, Autoware Core, an NVIDIA GPU with CUDA and TensorRT 10, and
	`autoware_tensorrt_plugins` from Autoware Universe for the backbone). Then download the model to the node's
	default model directory and launch it:

	```bash
	hf download AutowareFoundation/oneplanner --revision v1.0 --local-dir ~/autoware_data/oneplanner
	ros2 launch autoware_oneplanner oneplanner.launch.xml \
	model_path:=$HOME/autoware_data/oneplanner model_name:=oneplanner
	```

	Pin `--revision` to a tag rather than relying on `main`, and add `--exclude "media/*" "replay.mp4"` to skip
	the demo media. The first launch builds both TensorRT engines, which takes several minutes. To see the output in
	Autoware's RViz layout, remap `output_trajectory:=/planning/trajectory`; to drive with it, wire it into your
	planning stack's trajectory input behind Autoware's trajectory validation.

	The node feeds the inputs the model was trained with, the `oneplanner_v1` input convention: the ego state as
	recorded, the map conversion, the traffic-light selection and the temporal BEV history are specified in the
	model contract, and the node refuses a model package that states another convention.

	## Training

	The planner was trained by TIER IV on driving data recorded in Japan; the data is not public. The BEVFusion-L
	backbone was trained with [AWML](https://github.com/tier4/AWML) and kept frozen while the planner was trained.
	Accuracy figures are measured on internal validation data and are not published.

	## Limitations

	- Research model. It is not a certified driving system and must not control a vehicle on public roads.
	Run it in simulation, on closed courses, or with a trained safety driver and independent safeguards
	downstream of the planner; see the node's
	[DISCLAIMER](https://github.com/autowarefoundation/autoware_oneplanner/blob/main/DISCLAIMER.md).
	- Other road users through the LiDAR only. The planner reads no tracked or predicted objects: other road
	users reach it through the BEV features of the point cloud.
	- Sensor setup and domain. Trained on the vehicles, LiDAR configuration and roads of its training data in
	Japan (left-hand traffic, Japanese traffic lights). Performance on other platforms and operational domains is
	not characterized.
	- Evaluation. The video preview and the scenes on this page are open-loop replays of recorded drives: the
	plan is drawn against the path a person drove, not executed.

	## Provenance

	\| \| \|
	\| --- \| --- \|
	\| Source \| Exported to ONNX by the OnePlanner training code \|
	\| This repository \| `AutowareFoundation/oneplanner`, tag `v1.0` \|

	The graphs are the exported files, renamed to the node's default file names. `SHA256SUMS` lists their
	checksums.

	## References

	- `autoware_oneplanner`: <https://github.com/autowarefoundation/autoware_oneplanner>
	- Model contract: <https://github.com/autowarefoundation/autoware_oneplanner/blob/main/autoware_oneplanner/docs/model_contract.md>
	- AWML: <https://github.com/tier4/AWML>
	- Autoware: <https://github.com/autowarefoundation/autoware>