File size: 3,438 Bytes
ee6b5cf | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 | ---
license: apache-2.0
tags:
- depth-estimation
- monocular-depth
- core-ai
- coreai
- apple
- on-device
- depth-anything
pipeline_tag: depth-estimation
base_model:
- depth-anything/DA3-SMALL
- depth-anything/DA3-BASE
library_name: coreai
---
# Depth Anything 3 — Core AI
**The [coreai-model-zoo](https://github.com/john-rocky/coreai-model-zoo)'s first depth model.**
Monocular (single-image) **relative depth** estimation running fully on-device on Apple's Core AI
runtime, as a single static `.aimodel`. A conversion of ByteDance's
[Depth Anything 3](https://github.com/ByteDance-Seed/depth-anything-3)
([`depth-anything/DA3-SMALL`](https://huggingface.co/depth-anything/DA3-SMALL) /
[`DA3-BASE`](https://huggingface.co/depth-anything/DA3-BASE), Apache-2.0): a DINOv2 ViT backbone +
DPT-style head. Drop in an RGB image, get a depth map (and a confidence map). No NMS, no sampling —
host post-processing is just a colormap.
## Bundles
| dir | variant | params | dtype | size | M4 Max GPU |
|---|---|---|---|---|---|
| `small/da3-small_float16.aimodel` | ViT-S | 34.3M | fp16 | **54 MB** | **65.7 FPS** |
| `small/da3-small_float32.aimodel` | ViT-S | 34.3M | fp32 | 105 MB | 56.5 FPS |
| `base/da3-base_float16.aimodel` | ViT-B | 135.4M | fp16 | 202 MB | 26.5 FPS |
| `base/da3-base_float32.aimodel` | ViT-B | 135.4M | fp32 | 402 MB | 23.0 FPS |
`small · fp16` is the on-device hero — 54 MB, 65 FPS at 504² on an M4 Max, comfortably real-time on
iPhone-class GPUs. Each `.aimodel` is a directory bundle (`main.mlirb` + `metadata.json`).
## I/O contract
```
input : image [1, 3, 504, 504] RGB, raw [0, 1] (ImageNet normalization is folded into the graph)
output: depth [1, 504, 504] relative depth (exp-activated; larger = nearer)
depth_conf [1, 504, 504] confidence
```
Host: resize the RGB image to 504 × 504 (e.g. cv2 `INTER_AREA`), feed raw [0, 1], run, then resize
the depth map back to the original H × W. For display, the DA3 convention is inverse-depth →
percentile 2–98 normalize → `Spectral` colormap.
## Fidelity
- **Bit-exact conversion:** the Core AI engine matches the PyTorch reference at **cos 1.000000** (≤
~1e-5 / ~1e-2 per-pixel for fp32 / fp16) on both CPU and GPU, at any fixed input shape.
- **vs the official DA3 viewer:** **mean Pearson r ≈ 0.98** across diverse aspect ratios (square
inputs r = 1.000) — within DA3's own resolution sensitivity (its 504-vs-518 outputs differ by
r ≈ 0.975–0.984).
## Usage (CoreAIKit / coreai.runtime)
```python
import coreai.runtime as rt, numpy as np
from PIL import Image
m = await rt.AIModel.load("small/da3-small_float16.aimodel",
rt.SpecializationOptions.from_preferred_compute_unit_kind(rt.ComputeUnitKind.gpu()))
fn = m.load_function("main")
img = np.asarray(Image.open("photo.jpg").convert("RGB").resize((504, 504)))
x = (img.astype(np.float16) / 255.0).transpose(2, 0, 1)[None] # raw [0,1], NCHW
depth = (await fn({"image": rt.NDArray(x)}))["depth"].numpy().reshape(504, 504)
```
## Links
- Conversion script + model card: [coreai-model-zoo `zoo/depth-anything-3.md`](https://github.com/john-rocky/coreai-model-zoo/blob/main/zoo/depth-anything-3.md)
- Source: [Depth Anything 3](https://github.com/ByteDance-Seed/depth-anything-3) · Apache-2.0
---
*On-device ML / Core ML / Core AI model porting — get in touch: open an issue on the
[zoo](https://github.com/john-rocky/coreai-model-zoo).*
|