Bartosz Hanc commited on
Commit ·
9e726b6
1
Parent(s): 70834e2
Add Depth Anything V2 Small ExecuTorch models
Browse files- .gitattributes +1 -0
- README.md +74 -0
- config.json +3 -0
- small/coreml/config.json +43 -0
- small/coreml/depth_anything_v2_small_coreml_fp16.pte +3 -0
- small/vulkan/config.json +43 -0
- small/vulkan/depth_anything_v2_small_vulkan_fp16.pte +3 -0
- small/xnnpack/config.json +74 -0
- small/xnnpack/depth_anything_v2_small_xnnpack_fp32.pte +3 -0
- small/xnnpack/depth_anything_v2_small_xnnpack_int8.pte +3 -0
.gitattributes
CHANGED
|
@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 36 |
+
*.pte filter=lfs diff=lfs merge=lfs -text
|
README.md
CHANGED
|
@@ -1,3 +1,77 @@
|
|
| 1 |
---
|
| 2 |
license: apache-2.0
|
|
|
|
|
|
|
|
|
|
| 3 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
license: apache-2.0
|
| 3 |
+
pipeline_tag: depth-estimation
|
| 4 |
+
library_name: executorch
|
| 5 |
+
base_model: depth-anything/Depth-Anything-V2-Small-hf
|
| 6 |
---
|
| 7 |
+
|
| 8 |
+
# depth-anything-v2
|
| 9 |
+
|
| 10 |
+
This repository hosts **Depth Anything V2 Small** (ViT-S/14) exported for
|
| 11 |
+
[React Native ExecuTorch](https://www.npmjs.com/package/react-native-executorch)
|
| 12 |
+
as ExecuTorch `.pte` programs for on-device relative depth estimation.
|
| 13 |
+
|
| 14 |
+
Upstream model: [depth-anything/Depth-Anything-V2-Small-hf](https://huggingface.co/depth-anything/Depth-Anything-V2-Small-hf),
|
| 15 |
+
revision `5426e4f0f36572d16453bbda7a8389317b1bef99`.
|
| 16 |
+
|
| 17 |
+
## Variants
|
| 18 |
+
|
| 19 |
+
| Path | Backend | Precision |
|
| 20 |
+
| --- | --- | --- |
|
| 21 |
+
| `small/xnnpack/depth_anything_v2_small_xnnpack_fp32.pte` | xnnpack | fp32 |
|
| 22 |
+
| `small/xnnpack/depth_anything_v2_small_xnnpack_int8.pte` | xnnpack | int8 |
|
| 23 |
+
| `small/coreml/depth_anything_v2_small_coreml_fp16.pte` | coreml | fp16 |
|
| 24 |
+
| `small/vulkan/depth_anything_v2_small_vulkan_fp16.pte` | vulkan | fp16 |
|
| 25 |
+
|
| 26 |
+
## Input and output
|
| 27 |
+
|
| 28 |
+
The `forward` method takes a float32 RGB tensor `[1, 3, 518, 518]` in NCHW
|
| 29 |
+
layout. Resize the image to 518 × 518 using bicubic interpolation, scale pixel
|
| 30 |
+
values to `[0, 1]`, then normalize channels with mean `[0.485, 0.456, 0.406]`
|
| 31 |
+
and standard deviation `[0.229, 0.224, 0.225]`.
|
| 32 |
+
|
| 33 |
+
The output is a float32 tensor `[1, 518, 518]` containing raw relative inverse
|
| 34 |
+
depth: larger values indicate closer regions. Values are not distances in
|
| 35 |
+
meters and are not normalized to `[0, 1]`. Resize the depth map to the source
|
| 36 |
+
image dimensions as needed. All variants retain float32 public inputs and outputs.
|
| 37 |
+
|
| 38 |
+
## XNNPACK int8
|
| 39 |
+
|
| 40 |
+
The INT8 variant uses per-channel INT8 weights and dynamic INT8 activation
|
| 41 |
+
quantization for linear layers; convolutions remain floating point. It is a
|
| 42 |
+
mixed-precision model. FP32 remains the default variant in the XNNPACK config.
|
| 43 |
+
|
| 44 |
+
INT8 passed a provisional check of range-normalized RMSE below 2% and
|
| 45 |
+
correlation above 0.99 on four held-out upstream example images. A synthetic
|
| 46 |
+
random input exceeded that RMSE threshold (3.32%). This is limited numerical
|
| 47 |
+
validation, not a depth benchmark or a task accuracy guarantee.
|
| 48 |
+
|
| 49 |
+
## Backend validation
|
| 50 |
+
|
| 51 |
+
- XNNPACK FP32 and INT8 were executed on macOS and checked against the float model.
|
| 52 |
+
- CoreML FP16 was executed on macOS and numerically checked against the float model.
|
| 53 |
+
- Vulkan FP16 was exported and its serialized contract and delegation inspected;
|
| 54 |
+
execution on a Vulkan device has not been validated.
|
| 55 |
+
|
| 56 |
+
## Repository structure
|
| 57 |
+
|
| 58 |
+
```
|
| 59 |
+
config.json 39 B
|
| 60 |
+
small/coreml/config.json 928 B
|
| 61 |
+
small/coreml/depth_anything_v2_small_coreml_fp16.pte 50.2 MB
|
| 62 |
+
small/vulkan/config.json 928 B
|
| 63 |
+
small/vulkan/depth_anything_v2_small_vulkan_fp16.pte 49.6 MB
|
| 64 |
+
small/xnnpack/config.json 1.6 kB
|
| 65 |
+
small/xnnpack/depth_anything_v2_small_xnnpack_fp32.pte 99.0 MB
|
| 66 |
+
small/xnnpack/depth_anything_v2_small_xnnpack_int8.pte 35.5 MB
|
| 67 |
+
```
|
| 68 |
+
|
| 69 |
+
## Compatibility
|
| 70 |
+
|
| 71 |
+
These files are published for the **ExecuTorch v1.4.1** runtime. ExecuTorch gives
|
| 72 |
+
no forward compatibility guarantee, so an older runtime may fail to load them.
|
| 73 |
+
|
| 74 |
+
For React Native ExecuTorch model loading, see the
|
| 75 |
+
[documentation](https://docs.swmansion.com/react-native-executorch/docs/fundamentals/downloading-models).
|
| 76 |
+
To load these files in your own runtime, read the
|
| 77 |
+
[compatibility note](https://github.com/pytorch/executorch/blob/main/runtime/COMPATIBILITY.md).
|
config.json
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"modelName": "depth-anything-v2"
|
| 3 |
+
}
|
small/coreml/config.json
ADDED
|
@@ -0,0 +1,43 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"$schema": "https://huggingface.co/software-mansion/react-native-executorch-spec/resolve/main/config.schema.json",
|
| 3 |
+
"model": "depth_anything_v2_small",
|
| 4 |
+
"family": "depth-anything-v2",
|
| 5 |
+
"capabilities": [
|
| 6 |
+
"depth-estimation"
|
| 7 |
+
],
|
| 8 |
+
"backend": "coreml",
|
| 9 |
+
"license": "apache-2.0",
|
| 10 |
+
"variants": [
|
| 11 |
+
{
|
| 12 |
+
"file": "depth_anything_v2_small_coreml_fp16.pte",
|
| 13 |
+
"precision": "fp16",
|
| 14 |
+
"quantized": false,
|
| 15 |
+
"default": true,
|
| 16 |
+
"methods": {
|
| 17 |
+
"forward": {
|
| 18 |
+
"inputs": [
|
| 19 |
+
{
|
| 20 |
+
"shape": [
|
| 21 |
+
1,
|
| 22 |
+
3,
|
| 23 |
+
518,
|
| 24 |
+
518
|
| 25 |
+
],
|
| 26 |
+
"dtype": "float32"
|
| 27 |
+
}
|
| 28 |
+
],
|
| 29 |
+
"outputs": [
|
| 30 |
+
{
|
| 31 |
+
"shape": [
|
| 32 |
+
1,
|
| 33 |
+
518,
|
| 34 |
+
518
|
| 35 |
+
],
|
| 36 |
+
"dtype": "float32"
|
| 37 |
+
}
|
| 38 |
+
]
|
| 39 |
+
}
|
| 40 |
+
}
|
| 41 |
+
}
|
| 42 |
+
]
|
| 43 |
+
}
|
small/coreml/depth_anything_v2_small_coreml_fp16.pte
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:5e6e06eee9067b2b884b8f79f9444313db596704e2baf9f02d2eda3dde628071
|
| 3 |
+
size 50181365
|
small/vulkan/config.json
ADDED
|
@@ -0,0 +1,43 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"$schema": "https://huggingface.co/software-mansion/react-native-executorch-spec/resolve/main/config.schema.json",
|
| 3 |
+
"model": "depth_anything_v2_small",
|
| 4 |
+
"family": "depth-anything-v2",
|
| 5 |
+
"capabilities": [
|
| 6 |
+
"depth-estimation"
|
| 7 |
+
],
|
| 8 |
+
"backend": "vulkan",
|
| 9 |
+
"license": "apache-2.0",
|
| 10 |
+
"variants": [
|
| 11 |
+
{
|
| 12 |
+
"file": "depth_anything_v2_small_vulkan_fp16.pte",
|
| 13 |
+
"precision": "fp16",
|
| 14 |
+
"quantized": false,
|
| 15 |
+
"default": true,
|
| 16 |
+
"methods": {
|
| 17 |
+
"forward": {
|
| 18 |
+
"inputs": [
|
| 19 |
+
{
|
| 20 |
+
"shape": [
|
| 21 |
+
1,
|
| 22 |
+
3,
|
| 23 |
+
518,
|
| 24 |
+
518
|
| 25 |
+
],
|
| 26 |
+
"dtype": "float32"
|
| 27 |
+
}
|
| 28 |
+
],
|
| 29 |
+
"outputs": [
|
| 30 |
+
{
|
| 31 |
+
"shape": [
|
| 32 |
+
1,
|
| 33 |
+
518,
|
| 34 |
+
518
|
| 35 |
+
],
|
| 36 |
+
"dtype": "float32"
|
| 37 |
+
}
|
| 38 |
+
]
|
| 39 |
+
}
|
| 40 |
+
}
|
| 41 |
+
}
|
| 42 |
+
]
|
| 43 |
+
}
|
small/vulkan/depth_anything_v2_small_vulkan_fp16.pte
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:aed86d706e97a37db1bb54b385e350dde806ce0507b9465779b4bcd532998790
|
| 3 |
+
size 49595524
|
small/xnnpack/config.json
ADDED
|
@@ -0,0 +1,74 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"$schema": "https://huggingface.co/software-mansion/react-native-executorch-spec/resolve/main/config.schema.json",
|
| 3 |
+
"model": "depth_anything_v2_small",
|
| 4 |
+
"family": "depth-anything-v2",
|
| 5 |
+
"capabilities": [
|
| 6 |
+
"depth-estimation"
|
| 7 |
+
],
|
| 8 |
+
"backend": "xnnpack",
|
| 9 |
+
"license": "apache-2.0",
|
| 10 |
+
"variants": [
|
| 11 |
+
{
|
| 12 |
+
"file": "depth_anything_v2_small_xnnpack_fp32.pte",
|
| 13 |
+
"precision": "fp32",
|
| 14 |
+
"quantized": false,
|
| 15 |
+
"default": true,
|
| 16 |
+
"methods": {
|
| 17 |
+
"forward": {
|
| 18 |
+
"inputs": [
|
| 19 |
+
{
|
| 20 |
+
"shape": [
|
| 21 |
+
1,
|
| 22 |
+
3,
|
| 23 |
+
518,
|
| 24 |
+
518
|
| 25 |
+
],
|
| 26 |
+
"dtype": "float32"
|
| 27 |
+
}
|
| 28 |
+
],
|
| 29 |
+
"outputs": [
|
| 30 |
+
{
|
| 31 |
+
"shape": [
|
| 32 |
+
1,
|
| 33 |
+
518,
|
| 34 |
+
518
|
| 35 |
+
],
|
| 36 |
+
"dtype": "float32"
|
| 37 |
+
}
|
| 38 |
+
]
|
| 39 |
+
}
|
| 40 |
+
}
|
| 41 |
+
},
|
| 42 |
+
{
|
| 43 |
+
"file": "depth_anything_v2_small_xnnpack_int8.pte",
|
| 44 |
+
"precision": "int8",
|
| 45 |
+
"quantized": true,
|
| 46 |
+
"default": false,
|
| 47 |
+
"methods": {
|
| 48 |
+
"forward": {
|
| 49 |
+
"inputs": [
|
| 50 |
+
{
|
| 51 |
+
"shape": [
|
| 52 |
+
1,
|
| 53 |
+
3,
|
| 54 |
+
518,
|
| 55 |
+
518
|
| 56 |
+
],
|
| 57 |
+
"dtype": "float32"
|
| 58 |
+
}
|
| 59 |
+
],
|
| 60 |
+
"outputs": [
|
| 61 |
+
{
|
| 62 |
+
"shape": [
|
| 63 |
+
1,
|
| 64 |
+
518,
|
| 65 |
+
518
|
| 66 |
+
],
|
| 67 |
+
"dtype": "float32"
|
| 68 |
+
}
|
| 69 |
+
]
|
| 70 |
+
}
|
| 71 |
+
}
|
| 72 |
+
}
|
| 73 |
+
]
|
| 74 |
+
}
|
small/xnnpack/depth_anything_v2_small_xnnpack_fp32.pte
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:f83737200f4421a898c5b975695dacf4c693b0546d718adf96e9e6645084718f
|
| 3 |
+
size 99023364
|
small/xnnpack/depth_anything_v2_small_xnnpack_int8.pte
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:a3ccc836262d16529fd91b734c5df35da55169a905fcf7958992fc71b0c74ecb
|
| 3 |
+
size 35481732
|