| --- |
| library_name: onnxruntime |
| license: bsd-3-clause |
| tags: |
| - foundation |
| - amd |
| - rocm |
| - stereo-depth-estimation |
| pipeline_tag: depth-estimation |
| --- |
| |
|  |
|
|
| # RAFT-Stereo: Optimized for AMD ROCm |
|
|
| RAFT-Stereo ([Princeton VL](https://github.com/princeton-vl/RAFT-Stereo)) estimates dense per-pixel disparity from a rectified stereo image pair using multilevel recurrent field transforms. This repository packages evaluation/inference for stereo disparity estimation using ONNX Runtime, exported and validated for **AMD ROCm** so it runs efficiently on AMD GPUs and CPUs, as well as AMD Ryzen AI NPUs. |
|
|
| This is based on the implementation of RAFT-Stereo found [here](https://github.com/princeton-vl/RAFT-Stereo). |
| This repository contains configurations and scripts optimized for **AMD® ROCm™** platforms. You can use the [RAFT-Stereo AMD scripts](https://github.com/AMD-PAVS/physical_ai_sdk/blob/main/models/RAFT-Stereo) to reproduce results or export with custom configurations. |
|
|
| --- |
|
|
| ## Task Overview |
|
|
| **Task:** Stereo disparity estimation |
|
|
| **Dataset:** Middlebury MiddEval3 (F/H/Q splits), ETH3D two-view training set, KITTI 2012/2015 stereo, and FlyingThings3D (SceneFlow) |
|
|
| **Output metrics:** EPE (End-Point Error, px), D1-error (%) |
|
|
| > **ONNX Runtime note:** CPU runs FP32 only. GPU supports FP32/FP16/INT8 individually or all at once via the MIGraphX execution provider. NPU has no precision knob (Vitis AI compiles the graph as-is); instead it offers a **smoke** graph (few unrolled GRU iterations, compiles in minutes) vs. the **full** graph (32 iterations, Vitis AI compilation can take hours). |
|
|
| --- |
|
|
| ## AMD ROCm Optimization |
|
|
| This model export has been adapted and validated for **AMD Instinct™ / Radeon™ GPUs** running **ROCm**, as well as AMD CPUs and AMD Ryzen AI NPUs. Key points: |
|
|
| - Exported/tested with ROCm `7.x` for the MIGraphX execution provider. |
| - Validated backends: **ONNX Runtime CPU EP**, **ONNX Runtime MIGraphX EP** (ROCm GPU), and **Vitis AI EP** (Ryzen AI NPU). |
| - No code changes required versus the upstream RAFT-Stereo implementation — only environment/runtime configuration differs. |
| - CPU fallback path supported for environments without a ROCm-capable GPU. |
|
|
| | Runtime | Precision | Backend | Hardware | Notes | |
| |---|---|---|---|---| |
| | ONNX Runtime | FP32 | CPU EP | AMD CPU | Only precision supported on CPU | |
| | ONNX Runtime | FP32 / FP16 / INT8 | MIGraphX EP | AMD GPU (e.g. gfx11xx) | Precisions run individually or all at once | |
| | ONNX Runtime | N/A (compiled as-is) | Vitis AI EP | AMD Ryzen AI NPU | Smoke graph (fast compile) vs. full graph (32 iterations, long compile) | |
|
|
| --- |
|
|
| ## Getting Started |
|
|
| For setup instructions, evaluation scripts, and custom configuration options, see the [RAFT-Stereo on GitHub](https://github.com/AMD-PAVS/physical_ai_sdk/blob/main/models/RAFT-Stereo). |
|
|
| --- |
|
|
| ## Model Details |
|
|
| **Model Type:** Stereo disparity estimation (multilevel recurrent field transform network) |
|
|
| **Model Stats:** |
| - No `MODEL_SIZE` variants — single architecture |
| - Static ONNX export by default: `[1, 3, 384, 512]` input resolution (configurable via `--height`/`--width`, or `--dynamic` for variable axes) |
| - Precision tested: FP32 (CPU); FP32, FP16, INT8 (GPU); Vitis AI-compiled (NPU) |
|
|
| --- |
|
|
| ## Accuracy Pipeline |
|
|
| Accuracy evaluation is not yet implemented for this model. |
|
|
| --- |
|
|
| ## Dig Deeper |
|
|
| Want to explore the full evaluation scripts, config options, and other AMD-optimized model examples? |
|
|
| 📂 **[View the full project on GitHub](https://github.com/AMD-PAVS/physical_ai_sdk/blob/main/models/RAFT-Stereo)** |
|
|
| The GitHub repository includes: |
| - Setup and prerequisites for ROCm environments |
| - Scripts for the supported runners |
| - Additional model variants and datasets |
| - Benchmarking and reproduction instructions |
|
|
