File size: 3,708 Bytes
7f13ce4 efaa3d7 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 | ---
license: mit
---
# DepthART (AX650 Relative Depth Estimation)
DepthART is a family of relative depth estimation models based on selective state
spaces (Selective Scan). This repository provides the ONNX model of the `small / 448`
variant, an `axmodel` compiled for **AX650**, and tools for host ONNX inference, board
axmodel inference, and model rewriting.
The model outputs **affine-invariant relative depth**: larger values mean closer,
smaller values mean farther. It does not directly produce metric depth; align it with
ground truth depth using a global scale/offset before use.
## Single-image result
Left: input image; right: relative depth output by the model (INFERNO colormap,
stretched to the 2%–98% percentile range).
## Repository contents
| Path | Description |
| --- | --- |
| `models/relative_s_448_default.onnx` | Original FP32 ONNX model, containing the custom op `com.depthart::SelectiveScan` |
| `models/relative_s_448_default_ax650.axmodel` | Compiled AX650 model |
| `src/infer/depthart_infer_onnx.py` | Host ONNX inference (rewrites `SelectiveScan` automatically) |
| `src/infer/depthart_infer_axmodel.py` | AX650 board axmodel inference (standalone script) |
| `src/tool/depthart_export_axonnx.py` | Rewrites `SelectiveScan` into a pure ONNX subgraph that pulsar2 can compile |
| `src/tool/compare_onnx_axmodel.py`, `compare_depth_outputs.py` | Numerical comparison of outputs from different backends |
| `src/tool/evaluate_nyu654_depth.py` | Accuracy evaluation on the NYU Depth V2 eigen test split |
| `host.jpg`, `host_depth.jpg` | Single-image example input and side-by-side visualization result |
## Model interface
### ONNX
| Item | Value |
| --- | --- |
| Input | `image` `[1, 3, 448, 448]` float32, NCHW, RGB |
| Output | `depth` `[1, 1, 448, 448]` float32, relative depth (larger = closer) |
## AX650 board inference
The script `src/infer/depthart_infer_axmodel.py` does not depend on other scripts in
the repository. For deployment, just place the following files in the same directory
on the board:
```text
relative_s_448_default_ax650.axmodel
depthart_infer_axmodel.py
images or image directory to test
```
Board runtime requires `axengine`.
### Single-image validation
```bash
python3 depthart_infer_axmodel.py \
--axmodel relative_s_448_default_ax650.axmodel \
--image ./host.jpg \
--output ./results/ax_single
```
Outputs `results/ax_single/host_depth.jpg` (side-by-side visualization) and
`host_depth.npy`; the console prints depth statistics, NPU inference time, CPU
processing time, and per-stage timings. A single image is the first inference and
includes runtime first-execution overhead; use dataset mode for performance evaluation.
### Dataset inference
By default, fast mode only measures `read / preprocess / npu` and skips depth
restoration, visualization, and file writing, for performance testing:
```bash
python3 depthart_infer_axmodel.py \
--axmodel relative_s_448_default_ax650.axmodel \
--dataset ./images \
--output ./results/board
```
Full output (depth restoration, visualization, summary NPZ), for accuracy evaluation:
```bash
python3 depthart_infer_axmodel.py \
--axmodel relative_s_448_default_ax650.axmodel \
--dataset ./images \
--output ./results/board_full \
--full-output \
--dump-npz ./results/board_full/raw_outputs.npz
```
Image-by-image comparison against the host ONNX baseline summary NPZ:
```bash
python3 depthart_infer_axmodel.py \
--axmodel relative_s_448_default_ax650.axmodel \
--dataset ./images \
--output ./results/board_check \
--ref-npz ./results/onnx_full/raw_outputs.npz
```
|