sdkv2 commited on
Commit
bc5ba3a
·
verified ·
1 Parent(s): 9e8debd

Add model card

Browse files
Files changed (1) hide show
  1. README.md +63 -0
README.md ADDED
@@ -0,0 +1,63 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ base_model: depth-anything/DA3MONO-LARGE
4
+ pipeline_tag: depth-estimation
5
+ library_name: coreml
6
+ tags:
7
+ - coreml
8
+ - depth-estimation
9
+ - monocular-depth
10
+ - depth-anything
11
+ - apple-silicon
12
+ - stereo
13
+ ---
14
+
15
+ # DepthAnythingV3Mono-CoreML
16
+
17
+ A **CoreML conversion** of [`depth-anything/DA3MONO-LARGE`](https://huggingface.co/depth-anything/DA3MONO-LARGE)
18
+ — the monocular-depth variant of Depth Anything 3 (DINOv2 ViT-L backbone + DPT head, ~0.35B params) —
19
+ packaged for on-device inference on Apple Silicon (macOS 14+).
20
+
21
+ This is a derivative work of the original model, which is licensed **Apache-2.0**; this conversion is
22
+ released under the same license. All credit for the model itself goes to ByteDance / the Depth Anything 3
23
+ authors. See the [original repo](https://github.com/bytedance-seed/depth-anything-3).
24
+
25
+ ## What's in here
26
+
27
+ - `DepthAnythingV3Mono.mlpackage` — an ML Program, **FP16** weights, minimum deployment target **macOS 14**.
28
+
29
+ ## Interface
30
+
31
+ - **Input** `image`: an RGB image, **504×504** (a multiple of the DINOv2 patch size, 14).
32
+ ImageNet normalization is **baked into the graph**; the CoreML `ImageType` only rescales 0–255 → 0–1,
33
+ so you can hand it a `CVPixelBuffer` built straight from a `CGImage` with no manual preprocessing.
34
+ - **Output** `depth`: a single-channel `MLMultiArray` of shape `(1, 504, 504)` holding **relative** depth
35
+ (model-relative units). Consumers typically min-max normalize to `0…1`.
36
+
37
+ ## Conversion notes
38
+
39
+ Converted with `coremltools` from a `torch.jit.trace` of `backbone → head → depth`. The full
40
+ Depth Anything 3 `forward()` also runs camera-pose, sky and Gaussian-splat post-processing; those are
41
+ either inert for the mono model or not traceable (the sky refinement is a data-dependent `torch.quantile`),
42
+ so only the raw relative-depth path is converted. DINOv2's bicubic positional-embedding interpolation is
43
+ substituted with **bilinear** (coremltools has no `upsample_bicubic2d`); this is a sub-pixel approximation.
44
+
45
+ **Fidelity:** on a structured test image, the CoreML output matches the FP32 PyTorch reference with a
46
+ Pearson correlation of **0.99996** (normalized MAE 0.15%).
47
+
48
+ ## Usage (Swift / CoreML)
49
+
50
+ ```swift
51
+ import CoreML
52
+ import CoreImage
53
+
54
+ let model = try MLModel(contentsOf: compiledURL) // compile the .mlpackage first
55
+ // Provide `image` as a 504×504 CVPixelBuffer (32BGRA); read `depth` as an MLMultiArray (1×504×504).
56
+ ```
57
+
58
+ It is used as the default depth model in the SBS 3D image viewer (replacing Depth Anything V2-Large),
59
+ chosen specifically because DA3MONO-LARGE is Apache-2.0 and therefore safe for commercial distribution.
60
+
61
+ ## License & attribution
62
+
63
+ Apache-2.0, inherited from the upstream model. If you use this, please cite the original Depth Anything 3 work.