Add model card
Browse files
README.md
ADDED
|
@@ -0,0 +1,63 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
base_model: depth-anything/DA3MONO-LARGE
|
| 4 |
+
pipeline_tag: depth-estimation
|
| 5 |
+
library_name: coreml
|
| 6 |
+
tags:
|
| 7 |
+
- coreml
|
| 8 |
+
- depth-estimation
|
| 9 |
+
- monocular-depth
|
| 10 |
+
- depth-anything
|
| 11 |
+
- apple-silicon
|
| 12 |
+
- stereo
|
| 13 |
+
---
|
| 14 |
+
|
| 15 |
+
# DepthAnythingV3Mono-CoreML
|
| 16 |
+
|
| 17 |
+
A **CoreML conversion** of [`depth-anything/DA3MONO-LARGE`](https://huggingface.co/depth-anything/DA3MONO-LARGE)
|
| 18 |
+
— the monocular-depth variant of Depth Anything 3 (DINOv2 ViT-L backbone + DPT head, ~0.35B params) —
|
| 19 |
+
packaged for on-device inference on Apple Silicon (macOS 14+).
|
| 20 |
+
|
| 21 |
+
This is a derivative work of the original model, which is licensed **Apache-2.0**; this conversion is
|
| 22 |
+
released under the same license. All credit for the model itself goes to ByteDance / the Depth Anything 3
|
| 23 |
+
authors. See the [original repo](https://github.com/bytedance-seed/depth-anything-3).
|
| 24 |
+
|
| 25 |
+
## What's in here
|
| 26 |
+
|
| 27 |
+
- `DepthAnythingV3Mono.mlpackage` — an ML Program, **FP16** weights, minimum deployment target **macOS 14**.
|
| 28 |
+
|
| 29 |
+
## Interface
|
| 30 |
+
|
| 31 |
+
- **Input** `image`: an RGB image, **504×504** (a multiple of the DINOv2 patch size, 14).
|
| 32 |
+
ImageNet normalization is **baked into the graph**; the CoreML `ImageType` only rescales 0–255 → 0–1,
|
| 33 |
+
so you can hand it a `CVPixelBuffer` built straight from a `CGImage` with no manual preprocessing.
|
| 34 |
+
- **Output** `depth`: a single-channel `MLMultiArray` of shape `(1, 504, 504)` holding **relative** depth
|
| 35 |
+
(model-relative units). Consumers typically min-max normalize to `0…1`.
|
| 36 |
+
|
| 37 |
+
## Conversion notes
|
| 38 |
+
|
| 39 |
+
Converted with `coremltools` from a `torch.jit.trace` of `backbone → head → depth`. The full
|
| 40 |
+
Depth Anything 3 `forward()` also runs camera-pose, sky and Gaussian-splat post-processing; those are
|
| 41 |
+
either inert for the mono model or not traceable (the sky refinement is a data-dependent `torch.quantile`),
|
| 42 |
+
so only the raw relative-depth path is converted. DINOv2's bicubic positional-embedding interpolation is
|
| 43 |
+
substituted with **bilinear** (coremltools has no `upsample_bicubic2d`); this is a sub-pixel approximation.
|
| 44 |
+
|
| 45 |
+
**Fidelity:** on a structured test image, the CoreML output matches the FP32 PyTorch reference with a
|
| 46 |
+
Pearson correlation of **0.99996** (normalized MAE 0.15%).
|
| 47 |
+
|
| 48 |
+
## Usage (Swift / CoreML)
|
| 49 |
+
|
| 50 |
+
```swift
|
| 51 |
+
import CoreML
|
| 52 |
+
import CoreImage
|
| 53 |
+
|
| 54 |
+
let model = try MLModel(contentsOf: compiledURL) // compile the .mlpackage first
|
| 55 |
+
// Provide `image` as a 504×504 CVPixelBuffer (32BGRA); read `depth` as an MLMultiArray (1×504×504).
|
| 56 |
+
```
|
| 57 |
+
|
| 58 |
+
It is used as the default depth model in the SBS 3D image viewer (replacing Depth Anything V2-Large),
|
| 59 |
+
chosen specifically because DA3MONO-LARGE is Apache-2.0 and therefore safe for commercial distribution.
|
| 60 |
+
|
| 61 |
+
## License & attribution
|
| 62 |
+
|
| 63 |
+
Apache-2.0, inherited from the upstream model. If you use this, please cite the original Depth Anything 3 work.
|