psiddh commited on
Commit
aeba0bb
·
verified ·
1 Parent(s): c20e0ae

Add ExecuTorch WebGPU program for depth-anything-v2-small

Browse files
Files changed (2) hide show
  1. NOTICE +1 -1
  2. README.md +51 -8
NOTICE CHANGED
@@ -1,6 +1,6 @@
1
  This repository contains a modified version of depth-anything/Depth-Anything-V2-Small-hf.
2
 
3
  Modifications by Meta Platforms, Inc.: conversion to the ExecuTorch program format (.pte) for the
4
- ExecuTorch WebGPU backend, and quantization as described in README.md.
5
 
6
  The original work and these modifications are distributed under the license in LICENSE.
 
1
  This repository contains a modified version of depth-anything/Depth-Anything-V2-Small-hf.
2
 
3
  Modifications by Meta Platforms, Inc.: conversion to the ExecuTorch program format (.pte) for the
4
+ ExecuTorch WebGPU backend.
5
 
6
  The original work and these modifications are distributed under the license in LICENSE.
README.md CHANGED
@@ -2,9 +2,9 @@
2
  license: apache-2.0
3
  base_model:
4
  - depth-anything/Depth-Anything-V2-Small-hf
5
- base_model_relation: quantized
6
  library_name: executorch
7
  pipeline_tag: depth-estimation
 
8
  tags:
9
  - executorch
10
  - webgpu
@@ -15,8 +15,7 @@ tags:
15
 
16
  This repository contains an [ExecuTorch](https://github.com/pytorch/executorch) conversion of
17
  [depth-anything/Depth-Anything-V2-Small-hf](https://huggingface.co/depth-anything/Depth-Anything-V2-Small-hf) for the ExecuTorch **WebGPU** backend, so the model runs entirely in a web browser on
18
- the visitor's GPU. It powers the corresponding demo in the
19
- ExecuTorch WebGPU Playground (launching soon on [executorch.ai](https://executorch.ai)).
20
 
21
  - **Base model:** [depth-anything/Depth-Anything-V2-Small-hf](https://huggingface.co/depth-anything/Depth-Anything-V2-Small-hf)
22
  - **Format:** ExecuTorch `.pte` programs delegated to the WebGPU backend
@@ -25,6 +24,32 @@ ExecuTorch WebGPU Playground (launching soon on [executorch.ai](https://executor
25
 
26
  > **Note:** This is the Small variant, which is licensed Apache-2.0 (the larger Depth Anything V2 variants use a different license).
27
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
28
  ## Files
29
 
30
  | File | Size | SHA-256 |
@@ -35,10 +60,13 @@ Full checksums are in `manifest.json`. The playground verifies them after downlo
35
 
36
  ## Running it
37
 
38
- These programs need the ExecuTorch WebGPU runtime compiled to WebAssembly
39
- ([backends/webgpu](https://github.com/pytorch/executorch/tree/main/backends/webgpu)). The easiest way to
40
- try the model is the ExecuTorch WebGPU Playground (launching soon on [executorch.ai](https://executorch.ai)),
41
- which downloads these files once, caches them in the browser, and runs everything locally.
 
 
 
42
 
43
  ## Provenance
44
 
@@ -47,8 +75,23 @@ which downloads these files once, caches them in the browser, and runs everythin
47
  - **Reproduce:** `python recipes/depth-anything-v2-small.py --out out/depth-anything-v2-small` (recipe in `export/`)
48
  - Re-running the recipe produces byte-identical files.
49
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
50
  ## License
51
 
52
  This is a derivative of the upstream model and is distributed under the same terms
53
  (`apache-2.0`); see `LICENSE`.
54
- Modifications (conversion to the ExecuTorch format and quantization) by Meta Platforms, Inc.
 
2
  license: apache-2.0
3
  base_model:
4
  - depth-anything/Depth-Anything-V2-Small-hf
 
5
  library_name: executorch
6
  pipeline_tag: depth-estimation
7
+ inference: false
8
  tags:
9
  - executorch
10
  - webgpu
 
15
 
16
  This repository contains an [ExecuTorch](https://github.com/pytorch/executorch) conversion of
17
  [depth-anything/Depth-Anything-V2-Small-hf](https://huggingface.co/depth-anything/Depth-Anything-V2-Small-hf) for the ExecuTorch **WebGPU** backend, so the model runs entirely in a web browser on
18
+ the visitor's GPU. It powers the corresponding demo in the ExecuTorch WebGPU Playground (launching soon on [executorch.ai](https://executorch.ai)).
 
19
 
20
  - **Base model:** [depth-anything/Depth-Anything-V2-Small-hf](https://huggingface.co/depth-anything/Depth-Anything-V2-Small-hf)
21
  - **Format:** ExecuTorch `.pte` programs delegated to the WebGPU backend
 
24
 
25
  > **Note:** This is the Small variant, which is licensed Apache-2.0 (the larger Depth Anything V2 variants use a different license).
26
 
27
+ ## About the original model
28
+
29
+ The Small variant of Depth Anything V2, a monocular depth estimation model trained on about 595K synthetic labeled images and 62M+ real unlabeled images.
30
+
31
+ - **Developed by:** Lihe Yang et al. (HKU and TikTok)
32
+ - **Paper:** [Depth Anything V2](https://arxiv.org/abs/2406.09414)
33
+ - 24.8M parameters
34
+ - Predicts relative depth from a single image
35
+
36
+ For training data, evaluation results and full details, see the original model card: [depth-anything/Depth-Anything-V2-Small-hf](https://huggingface.co/depth-anything/Depth-Anything-V2-Small-hf).
37
+
38
+ ## Intended use and limitations
39
+
40
+ Relative depth maps for photos and camera frames.
41
+
42
+ Outputs relative (not metric) depth. Accuracy drops on reflective or transparent surfaces and unusual scenes.
43
+
44
+ ## What changed in this conversion
45
+
46
+ - Fixed 224x224 input (upstream processes larger images)
47
+
48
+ ## Inputs and outputs
49
+
50
+ - Input `[1, 3, 224, 224]` float32 RGB, ImageNet-normalized: `(x / 255 - mean) / std`, mean 0.485/0.456/0.406, std 0.229/0.224/0.225
51
+ - Output `[1, 224, 224]` float32 relative inverse depth (larger means closer)
52
+
53
  ## Files
54
 
55
  | File | Size | SHA-256 |
 
60
 
61
  ## Running it
62
 
63
+ These programs were exported with the ExecuTorch WebGPU backend
64
+ ([backends/webgpu](https://github.com/pytorch/executorch/tree/main/backends/webgpu)) and validated with the
65
+ WebAssembly runtime build used by the ExecuTorch WebGPU Playground (launching soon on [executorch.ai](https://executorch.ai)). That runtime build is not published yet; until it is,
66
+ the playground is the supported way to run them. It downloads these files once, caches them in the browser,
67
+ and runs everything locally.
68
+
69
+ Requirements: a browser with WebGPU enabled.
70
 
71
  ## Provenance
72
 
 
75
  - **Reproduce:** `python recipes/depth-anything-v2-small.py --out out/depth-anything-v2-small` (recipe in `export/`)
76
  - Re-running the recipe produces byte-identical files.
77
 
78
+ ## Citation
79
+
80
+ If you use this model, please cite the original work:
81
+
82
+ ```bibtex
83
+ @misc{yang2024depth,
84
+ title={Depth Anything V2},
85
+ author={Lihe Yang and Bingyi Kang and Zilong Huang and Zhen Zhao and Xiaogang Xu and Jiashi Feng and Hengshuang Zhao},
86
+ year={2024},
87
+ eprint={2406.09414},
88
+ archivePrefix={arXiv},
89
+ primaryClass={cs.CV}
90
+ }
91
+ ```
92
+
93
  ## License
94
 
95
  This is a derivative of the upstream model and is distributed under the same terms
96
  (`apache-2.0`); see `LICENSE`.
97
+ Modifications (conversion to the ExecuTorch format) by Meta Platforms, Inc.