README.md

---
license: other
license_name: dinov3-license
license_link: https://ai.meta.com/resources/models-and-libraries/dinov3-license/
tags:
  - depth-estimation
  - semantic-segmentation
  - onnx
  - robotics
  - ros2
  - jetson
  - dinov3
library_name: onnx
pipeline_tag: depth-estimation
---

# M2H-MX ONNX weights — monocular depth + semantics

Exported ONNX weights for the two **M2H-MX** networks used by
[`vio_stack_jetson`](https://github.com/BavanthaU/vio_stack_jetson), a monocular
VIO stack running on a Jetson Orin with a ZED X One GS camera.

Both networks take a single RGB frame and produce **metric depth** and
**per-pixel semantic labels** in one pass. Both publish the same two ROS 2
topics, so they are swappable at runtime:

```
/m2h/depth/image             32FC1 metric depth
/m2h/semantic/labels_argmax  mono8 class IDs
```

**Built with DINOv3.**

## Files

The layout mirrors the ROS packages that consume these files, so a manifest line
maps to a path with no translation:

| File | Size | Notes |
|---|---|---|
| `m2h_mx_base_onnx_ros/models/scannet/m2h_mx_b_scannet_240x320.onnx` | 421 MB | base network, default |
| `m2h_mx_base_onnx_ros/models/scannet/m2h_mx_b_scannet_480x640.onnx` | 422 MB | base network, full res |
| `m2h_mx_base_onnx_ros/models/nyudv2/m2h_mx_b_nyudv2_240x320.onnx` | 494 MB | NYUDv2-trained variant |
| `m2h_mx_base_onnx_ros/models/nyudv2/m2h_mx_b_nyudv2_480x640.onnx` | 496 MB | NYUDv2-trained variant |
| `m2h_mx_large_onnx/onnx_models/scannet_depth_sem_192x256_trt_clean.onnx` | 1.26 GB | large network, default |
| `m2h_mx_large_onnx/onnx_models/scannet_depth_sem_320x416.onnx` | 1.26 GB | large network, higher res |

`MANIFEST.sha256` in each package directory lists the expected SHA256 of every
file. Verify after downloading — a network running on the wrong weights produces
plausible-looking depth rather than an error, which is much harder to notice than
a refusal to start.

## Usage

```bash
git clone https://github.com/BavanthaU/vio_stack_jetson
cd vio_stack_jetson
./tools/fetch_weights.sh          # pulls these files and verifies every sum
```

Or directly:

```python
from huggingface_hub import snapshot_download
snapshot_download(repo_id="Bavantha11/vio-stack-jetson-weights", repo_type="model")
```

## Measured on the rig

Jetson Orin, JetPack r36.5, ONNX Runtime with the TensorRT execution provider:

| Network | Resolution | Latency | Rate |
|---|---|---|---|
| base | 240x320 | ~130 ms/frame | ~6.9 Hz |
| large | 192x256 | ~155 ms/frame | — |

The first run at any resolution builds a TensorRT engine — minutes, once, cached
per machine. The engines are **not** portable across machines or driver versions
and are deliberately not published here.

Depth is not fed back into the VIO filter by default; it runs alongside it.

## Architecture

- **Backbone:** DINOv3 ViT-B/16 (`dinov3-vitb16-pretrain-lvd1689m`)
- **Heads:** two — semantic segmentation (40 classes) and metric depth
- **Opset:** 17, static shapes
- **Training data:** ScanNet, NYUDv2

## Licensing — read before use

These weights combine inputs with different terms. The most restrictive one
governs the result.

**DINOv3 backbone** — [DINOv3 License](https://ai.meta.com/resources/models-and-libraries/dinov3-license/),
included here as `DINOv3_LICENSE.md`. It permits commercial use, modification and
redistribution of derivative works, and asks in return that redistributions carry
a copy of the licence and display "Built with DINOv3". Both are satisfied above.

**ScanNet** — [Terms of Use](http://kaldir.vc.in.tum.de/scannet/ScanNet_TOS.pdf).
**Non-commercial research and educational use only**, and those terms extend to
derivative works.

**NYUDv2** — research use.

**Therefore these weights are for non-commercial research and educational use
only.** ScanNet is the binding constraint; DINOv3 permitting commercial use does
not lift it. Anyone needing a commercial deployment must retrain on commercially
licensed data — the ROS packages and the export pipeline are MIT and unaffected.

The MIT licence on the `m2h_mx_*` ROS packages covers the **wrapper code only**,
not these weights.

## Citation

If you use these in published work, please acknowledge DINOv3 and cite the
ScanNet and NYUDv2 datasets as their terms require.