Image Classification
TensorRT
ONNX
autoware
ros2
autonomous-driving
lidar
camera
calibration
sensing
Instructions to use AutowareFoundation/calibration_status_classifier with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- TensorRT
How to use AutowareFoundation/calibration_status_classifier with TensorRT:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- Notebooks
- Google Colab
- Kaggle
feat: add calibration_status_classifier v2.0 artifacts (from awf.ml.dev.web.auto/sensing/models/calibration_status_classifier/v2)
Browse files- .gitignore +5 -0
- README.md +115 -0
- calibration_status_classifier.onnx +3 -0
- deploy_metadata.yaml +1 -0
- ml_package_calibration_status_classifier.param.yaml +7 -0
.gitignore
ADDED
|
@@ -0,0 +1,5 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Auto-generated TensorRT artifacts, built locally by Autoware from the ONNX
|
| 2 |
+
# files (see autoware_tensorrt_common). They are environment-specific
|
| 3 |
+
# (GPU arch + TensorRT version) and must not be committed to this repo.
|
| 4 |
+
*.engine
|
| 5 |
+
*.json
|
README.md
ADDED
|
@@ -0,0 +1,115 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
pipeline_tag: image-classification
|
| 4 |
+
tags:
|
| 5 |
+
- autoware
|
| 6 |
+
- ros2
|
| 7 |
+
- autonomous-driving
|
| 8 |
+
- lidar
|
| 9 |
+
- camera
|
| 10 |
+
- calibration
|
| 11 |
+
- sensing
|
| 12 |
+
- tensorrt
|
| 13 |
+
- onnx
|
| 14 |
+
---
|
| 15 |
+
|
| 16 |
+
# Calibration Status Classifier for Autoware (`calibration_status_classifier`)
|
| 17 |
+
|
| 18 |
+
Deep learning model for real-time LiDAR-camera calibration validation, used by the
|
| 19 |
+
[`autoware_calibration_status_classifier`](https://github.com/autowarefoundation/autoware_universe/tree/main/sensing/autoware_calibration_status_classifier)
|
| 20 |
+
node in [Autoware](https://github.com/autowarefoundation/autoware).
|
| 21 |
+
|
| 22 |
+
The node projects the LiDAR point cloud onto the (undistorted) camera image, feeds the resulting
|
| 23 |
+
5-channel tensor to this network, and classifies whether the LiDAR-camera extrinsic calibration is
|
| 24 |
+
still valid. The model is exported as ONNX and runs with TensorRT inside Autoware; Autoware builds
|
| 25 |
+
the TensorRT engine from the ONNX file on first launch.
|
| 26 |
+
|
| 27 |
+
## Model overview
|
| 28 |
+
|
| 29 |
+
| | |
|
| 30 |
+
| --- | --- |
|
| 31 |
+
| Task | Binary classification of LiDAR-camera calibration status (calibrated vs. miscalibrated) |
|
| 32 |
+
| Input | 5-channel normalized image tensor: RGB + projected LiDAR depth + intensity |
|
| 33 |
+
| Output | Calibration status with a miscalibration confidence score |
|
| 34 |
+
| Input resolution | Dynamic; TensorRT optimization profile height `[1080, 1860, 2160]`, width `[1920, 2880, 3840]` (min, opt, max) |
|
| 35 |
+
| Runtime | TensorRT (FP16 by default, FP32 selectable) via the `autoware_calibration_status_classifier` ROS 2 node |
|
| 36 |
+
| Format | ONNX (Autoware builds the TensorRT engine locally on first launch) |
|
| 37 |
+
| License | Apache-2.0 |
|
| 38 |
+
|
| 39 |
+
Pre-processing runs CUDA-accelerated in the node, not in the ONNX graph: image undistortion,
|
| 40 |
+
projection of 3D LiDAR points onto the 2D image plane (adding depth and intensity channels), and
|
| 41 |
+
morphological dilation of the projected points.
|
| 42 |
+
|
| 43 |
+
## Files
|
| 44 |
+
|
| 45 |
+
| File | Description |
|
| 46 |
+
| --- | --- |
|
| 47 |
+
| `calibration_status_classifier.onnx` | Calibration status classification network |
|
| 48 |
+
| `ml_package_calibration_status_classifier.param.yaml` | Model parameters (max projection depth, dilation kernel size, input resolution profile) |
|
| 49 |
+
| `deploy_metadata.yaml` | Deployment metadata recording the artifact version of this repository |
|
| 50 |
+
|
| 51 |
+
> **TensorRT engines are not distributed here.** TensorRT engines are specific to the GPU
|
| 52 |
+
> architecture and TensorRT version they are built on and are not portable, so Autoware builds them
|
| 53 |
+
> locally from the ONNX file on first launch (or via `build_only:=true`).
|
| 54 |
+
|
| 55 |
+
## Inputs and outputs (as used by the node)
|
| 56 |
+
|
| 57 |
+
**Inputs**
|
| 58 |
+
|
| 59 |
+
- LiDAR point cloud topics (`sensor_msgs/msg/PointCloud2`, XYZIRC format with intensity)
|
| 60 |
+
- Camera image topics (`sensor_msgs/msg/Image`, BGR8 format)
|
| 61 |
+
- Camera info topics (`sensor_msgs/msg/CameraInfo`, intrinsics and distortion coefficients)
|
| 62 |
+
|
| 63 |
+
**Outputs**
|
| 64 |
+
|
| 65 |
+
- `/diagnostics` (`diagnostic_msgs/msg/DiagnosticArray`): calibration status per camera-LiDAR pair
|
| 66 |
+
- Preview image topics (`sensor_msgs/msg/Image`): visualization with projected points
|
| 67 |
+
|
| 68 |
+
The node supports three runtime modes: `MANUAL` (on-demand validation via a `std_srvs/srv/Trigger`
|
| 69 |
+
service), `PERIODIC` (validation at a configurable interval), and `ACTIVE` (continuous monitoring
|
| 70 |
+
with synchronized sensor data). Inference is gated by configurable prerequisite checks on vehicle
|
| 71 |
+
linear velocity, angular velocity, and the number of detected objects in the scene.
|
| 72 |
+
|
| 73 |
+
## Usage in Autoware
|
| 74 |
+
|
| 75 |
+
The node loads these artifacts from `~/autoware_data/ml_models/calibration_status_classifier/` and
|
| 76 |
+
launches with:
|
| 77 |
+
|
| 78 |
+
```bash
|
| 79 |
+
ros2 launch autoware_calibration_status_classifier calibration_status_classifier.launch.xml
|
| 80 |
+
```
|
| 81 |
+
|
| 82 |
+
Add `build_only:=true` to build the TensorRT engine from the ONNX as a one-off pre-task, and
|
| 83 |
+
`model_path:=...` to point at a non-default artifact directory. See the
|
| 84 |
+
[package README](https://github.com/autowarefoundation/autoware_universe/tree/main/sensing/autoware_calibration_status_classifier)
|
| 85 |
+
for the full parameter reference.
|
| 86 |
+
|
| 87 |
+
## Training
|
| 88 |
+
|
| 89 |
+
The model was trained with the AWML CalibrationStatusClassification project:
|
| 90 |
+
<https://github.com/tier4/AWML/tree/main/projects/CalibrationStatusClassification>.
|
| 91 |
+
It was trained on calibrated and miscalibrated LiDAR-camera data; the exact training datasets and
|
| 92 |
+
schedules are not publicly documented beyond the AWML project page.
|
| 93 |
+
|
| 94 |
+
## Provenance and versioning
|
| 95 |
+
|
| 96 |
+
| Hugging Face tag | Original source |
|
| 97 |
+
| --- | --- |
|
| 98 |
+
| `v2.0` | `https://awf.ml.dev.web.auto/sensing/models/calibration_status_classifier/v2/` |
|
| 99 |
+
|
| 100 |
+
This repository migrates the artifacts previously hosted at the URL above. Consumers should pin a
|
| 101 |
+
tag (`--revision v2.0`) rather than tracking `main`.
|
| 102 |
+
|
| 103 |
+
## Limitations
|
| 104 |
+
|
| 105 |
+
- Input images must be BGR8 (8-bit per channel); input point clouds must contain intensity
|
| 106 |
+
(XYZIRC format).
|
| 107 |
+
- The classifier judges calibration from the projected overlay, so scenes must satisfy the node's
|
| 108 |
+
prerequisite checks (motion and object-count gates) for reliable results.
|
| 109 |
+
- Sensor setups that differ substantially from the training configuration may require retraining
|
| 110 |
+
via the AWML project.
|
| 111 |
+
|
| 112 |
+
## References
|
| 113 |
+
|
| 114 |
+
- AWML CalibrationStatusClassification: <https://github.com/tier4/AWML/tree/main/projects/CalibrationStatusClassification>
|
| 115 |
+
- Consuming package: <https://github.com/autowarefoundation/autoware_universe/tree/main/sensing/autoware_calibration_status_classifier>
|
calibration_status_classifier.onnx
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:13d6119e8dc73945ab0efe76b0c9492e2536187fdc7963ba62b8625900f54474
|
| 3 |
+
size 44825085
|
deploy_metadata.yaml
ADDED
|
@@ -0,0 +1 @@
|
|
|
|
|
|
|
| 1 |
+
version: v2.0
|
ml_package_calibration_status_classifier.param.yaml
ADDED
|
@@ -0,0 +1,7 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
/**:
|
| 2 |
+
ros__parameters:
|
| 3 |
+
max_depth: 128.0
|
| 4 |
+
dilation_size: 1
|
| 5 |
+
height: [1080, 1860, 2160] # [min, opt, max]
|
| 6 |
+
width: [1920, 2880, 3840] # [min, opt, max]
|
| 7 |
+
|