README.md

---
license: apache-2.0
pipeline_tag: image-segmentation
tags:
  - autoware
  - ros2
  - autonomous-driving
  - camera
  - road-segmentation
  - localization
  - yabloc
  - tensorflow
---

# Road Segmentation for Autoware YabLoc (`yabloc_pose_initializer`)

Road semantic segmentation model used by the
[`yabloc_pose_initializer`](https://github.com/autowarefoundation/autoware_universe/tree/main/localization/yabloc/yabloc_pose_initializer)
package in [Autoware](https://github.com/autowarefoundation/autoware).

The `camera_pose_initializer` node estimates the vehicle's initial pose from a camera image at the request of
AD API. It segments the road surface in the undistorted camera image with this model and matches the result
against the Lanelet2 vector map to score initial pose candidates.

The model is Intel Open Model Zoo's **road-segmentation-adas-0001**, converted to a TensorFlow frozen graph via
the PINTO model zoo (entry 136). It runs on CPU through OpenCV DNN inside the node; no TensorRT or ONNX runtime
is involved.

## Model overview

| | |
| --- | --- |
| Task | Road semantic segmentation of a camera image, used for camera-based initial pose estimation |
| Origin | Intel Open Model Zoo `road-segmentation-adas-0001`, converted by the PINTO model zoo |
| Runtime | OpenCV DNN (`cv::dnn::readNet`), OpenCV backend, CPU target |
| Format | TensorFlow frozen graph, float32 (`model_float32.pb`) |
| Network input | `1 x 3 x 512 x 896` float32 blob (RGB, scale 1.0, no mean subtraction) |
| Network output | 4-channel segmentation score map |
| License | Apache-2.0 (Intel Open Model Zoo) |

For the class definitions and architecture details of the segmentation network, see the upstream
[Intel Open Model Zoo model page](https://github.com/openvinotoolkit/open_model_zoo/tree/master/models/intel/road-segmentation-adas-0001).

## Files

| File | Description |
| --- | --- |
| `saved_model/model_float32.pb` | TensorFlow frozen graph, float32; the only file the node loads |
| `deploy_metadata.yaml` | Deployment metadata recording the artifact version of this repository |

> The upstream PINTO model zoo export also contains other formats (TFLite, TF.js, OpenVINO IR, ONNX,
> pre-serialized TensorRT saved models). They are intentionally not distributed here: the node consumes only the
> TensorFlow frozen graph, and pre-built TensorRT engines are environment-specific and not portable.

The directory layout (`saved_model/model_float32.pb`) is preserved exactly as the package's launch file expects
it.

## Inputs and outputs (as used by the node)

**Subscriptions**

| Topic | Type | Description |
| --- | --- | --- |
| `~/input/camera_info` | `sensor_msgs/msg/CameraInfo` | undistorted camera info |
| `~/input/image_raw` | `sensor_msgs/msg/Image` | undistorted camera image |
| `~/input/vector_map` | `autoware_map_msgs/msg/LaneletMapBin` | vector map |

**Publications**

| Topic | Type | Description |
| --- | --- | --- |
| `~/debug/init_candidates` | `visualization_msgs/msg/MarkerArray` | initial pose candidates (the package README lists this topic as `output/candidates`, but the node publishes it under `debug/init_candidates`) |

**Services**

| Service | Type | Description |
| --- | --- | --- |
| `~/yabloc_align_srv` | `autoware_internal_localization_msgs/srv/PoseWithCovarianceStamped` | initial pose estimation request |

Pre-processing and post-processing run in the node: the image is resized to 896 x 512 and converted to a float32
RGB blob; the 4-channel output score map is resized back to the image resolution, the first (background) channel
is dropped, and the remaining three channels are thresholded into a binary mask image used for map matching.

The node's only ROS parameter besides `model_path` is `angle_resolution` (default 30, the number of divisions of
the 1 sigma angle range).

## Usage in Autoware

Autoware downloads this artifact to `~/autoware_data/ml_models/yabloc_pose_initializer/` during environment
setup (the ansible artifacts role). To fetch it manually:

```bash
hf download AutowareFoundation/yabloc_pose_initializer --revision v1.0 \
  --local-dir ~/autoware_data/ml_models/yabloc_pose_initializer
```

The package's launch file resolves the model at:

```text
$HOME/autoware_data/ml_models/yabloc_pose_initializer/saved_model/model_float32.pb
```

and passes it to the node as the `model_path` parameter (overridable via the `model_path` launch argument). The
node is started as part of the YabLoc localization stack; see the
[package README](https://github.com/autowarefoundation/autoware_universe/tree/main/localization/yabloc/yabloc_pose_initializer)
for details. If the model is missing, initialization still completes, but accuracy may be compromised.

## Training

This model was not trained by the Autoware project; it is redistributed as-is from upstream:

- Original model: Intel Open Model Zoo `road-segmentation-adas-0001`
  (<https://github.com/openvinotoolkit/open_model_zoo/tree/master/models/intel/road-segmentation-adas-0001>),
  licensed under Apache License 2.0.
- Conversion to TensorFlow: PINTO model zoo entry 136
  (<https://github.com/PINTO0309/PINTO_model_zoo/tree/main/136_road-segmentation-adas-0001>); the conversion
  scripts are released under the MIT license.

Training datasets, schedules, and metrics are not documented in the Autoware sources; refer to the Intel Open
Model Zoo model page for upstream details.

## Provenance and versioning

| | |
| --- | --- |
| Tag | `v1.0` |
| Original source | `https://autoware-files.s3.us-west-2.amazonaws.com/models/yabloc/136_road-segmentation-adas-0001/resources.tar.gz` (unversioned tarball) |
| `saved_model/model_float32.pb` sha256 | `c4e373552f4efb91592ed99f49afc6df179f8a95174ceaddd1ab35692f435b47` |

The original tarball bundled the full PINTO model zoo export; only the file the node consumes is distributed
here.

## Limitations

- Inference runs on CPU via OpenCV DNN; the node does not use a GPU for this model.
- The network operates at a fixed 896 x 512 input resolution; images are resized by the node.
- The model was trained upstream by Intel, not on Autoware-specific data; segmentation quality on cameras or
  scenes that differ from the upstream training domain is not characterized here.
- Pose initialization quality depends on the vector map and the undistorted camera input; a missing or poorly
  matching segmentation degrades the initial pose accuracy.

## References

- Intel Open Model Zoo, road-segmentation-adas-0001:
  <https://github.com/openvinotoolkit/open_model_zoo/tree/master/models/intel/road-segmentation-adas-0001>
- PINTO model zoo, entry 136: <https://github.com/PINTO0309/PINTO_model_zoo/tree/main/136_road-segmentation-adas-0001>
- Consuming package:
  <https://github.com/autowarefoundation/autoware_universe/tree/main/localization/yabloc/yabloc_pose_initializer>
- Autoware: <https://github.com/autowarefoundation/autoware>

## Acknowledgment

Special thanks to [openvinotoolkit/open_model_zoo](https://github.com/openvinotoolkit/open_model_zoo) and
[PINTO0309](https://github.com/PINTO0309) for providing and converting the original model.

## Legal Notice

_The original model is distributed by Intel Open Model Zoo under the Apache License, Version 2.0. The PINTO
model zoo conversion scripts are released under the MIT license. See the upstream repositories for the full
license terms._