| --- |
| license: mit |
| --- |
| |
| # DepthART (AX650 Relative Depth Estimation) |
|
|
| DepthART is a family of relative depth estimation models based on selective state |
| spaces (Selective Scan). This repository provides the ONNX model of the `small / 448` |
| variant, an `axmodel` compiled for **AX650**, and tools for host ONNX inference, board |
| axmodel inference, and model rewriting. |
|
|
| The model outputs **affine-invariant relative depth**: larger values mean closer, |
| smaller values mean farther. It does not directly produce metric depth; align it with |
| ground truth depth using a global scale/offset before use. |
|
|
| ## Single-image result |
|
|
| Left: input image; right: relative depth output by the model (INFERNO colormap, |
| stretched to the 2%–98% percentile range). |
|
|
|  |
|
|
| ## Repository contents |
|
|
| | Path | Description | |
| | --- | --- | |
| | `models/relative_s_448_default.onnx` | Original FP32 ONNX model, containing the custom op `com.depthart::SelectiveScan` | |
| | `models/relative_s_448_default_ax650.axmodel` | Compiled AX650 model | |
| | `src/infer/depthart_infer_onnx.py` | Host ONNX inference (rewrites `SelectiveScan` automatically) | |
| | `src/infer/depthart_infer_axmodel.py` | AX650 board axmodel inference (standalone script) | |
| | `src/tool/depthart_export_axonnx.py` | Rewrites `SelectiveScan` into a pure ONNX subgraph that pulsar2 can compile | |
| | `src/tool/compare_onnx_axmodel.py`, `compare_depth_outputs.py` | Numerical comparison of outputs from different backends | |
| | `src/tool/evaluate_nyu654_depth.py` | Accuracy evaluation on the NYU Depth V2 eigen test split | |
| | `host.jpg`, `host_depth.jpg` | Single-image example input and side-by-side visualization result | |
|
|
| ## Model interface |
|
|
| ### ONNX |
|
|
| | Item | Value | |
| | --- | --- | |
| | Input | `image` `[1, 3, 448, 448]` float32, NCHW, RGB | |
| | Output | `depth` `[1, 1, 448, 448]` float32, relative depth (larger = closer) | |
|
|
| ## AX650 board inference |
|
|
| The script `src/infer/depthart_infer_axmodel.py` does not depend on other scripts in |
| the repository. For deployment, just place the following files in the same directory |
| on the board: |
|
|
| ```text |
| relative_s_448_default_ax650.axmodel |
| depthart_infer_axmodel.py |
| images or image directory to test |
| ``` |
|
|
| Board runtime requires `axengine`. |
|
|
| ### Single-image validation |
|
|
| ```bash |
| python3 depthart_infer_axmodel.py \ |
| --axmodel relative_s_448_default_ax650.axmodel \ |
| --image ./host.jpg \ |
| --output ./results/ax_single |
| ``` |
|
|
| Outputs `results/ax_single/host_depth.jpg` (side-by-side visualization) and |
| `host_depth.npy`; the console prints depth statistics, NPU inference time, CPU |
| processing time, and per-stage timings. A single image is the first inference and |
| includes runtime first-execution overhead; use dataset mode for performance evaluation. |
|
|
| ### Dataset inference |
|
|
| By default, fast mode only measures `read / preprocess / npu` and skips depth |
| restoration, visualization, and file writing, for performance testing: |
|
|
| ```bash |
| python3 depthart_infer_axmodel.py \ |
| --axmodel relative_s_448_default_ax650.axmodel \ |
| --dataset ./images \ |
| --output ./results/board |
| ``` |
|
|
| Full output (depth restoration, visualization, summary NPZ), for accuracy evaluation: |
|
|
| ```bash |
| python3 depthart_infer_axmodel.py \ |
| --axmodel relative_s_448_default_ax650.axmodel \ |
| --dataset ./images \ |
| --output ./results/board_full \ |
| --full-output \ |
| --dump-npz ./results/board_full/raw_outputs.npz |
| ``` |
|
|
| Image-by-image comparison against the host ONNX baseline summary NPZ: |
|
|
| ```bash |
| python3 depthart_infer_axmodel.py \ |
| --axmodel relative_s_448_default_ax650.axmodel \ |
| --dataset ./images \ |
| --output ./results/board_check \ |
| --ref-npz ./results/onnx_full/raw_outputs.npz |
| ``` |
|
|
