Commit ·
25c2928
0
Parent(s):
init
Browse filesCo-authored-by: amael-apple <amael-apple@users.noreply.huggingface.co>
Co-authored-by: patricejaton <patricejaton@users.noreply.huggingface.co>
- .gitattributes +44 -0
- LICENSE +88 -0
- README.md +134 -0
- assets/hero-poster.jpg +3 -0
- assets/hero.mp4 +3 -0
- assets/method-dark.png +3 -0
- assets/method-light.png +3 -0
- assets/teaser.jpg +3 -0
- assets/tradeoff-dark.png +3 -0
- assets/tradeoff-light.png +3 -0
.gitattributes
ADDED
|
@@ -0,0 +1,44 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
*.7z filter=lfs diff=lfs merge=lfs -text
|
| 2 |
+
*.arrow filter=lfs diff=lfs merge=lfs -text
|
| 3 |
+
*.bin filter=lfs diff=lfs merge=lfs -text
|
| 4 |
+
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
| 5 |
+
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
| 6 |
+
*.ftz filter=lfs diff=lfs merge=lfs -text
|
| 7 |
+
*.gz filter=lfs diff=lfs merge=lfs -text
|
| 8 |
+
*.h5 filter=lfs diff=lfs merge=lfs -text
|
| 9 |
+
*.joblib filter=lfs diff=lfs merge=lfs -text
|
| 10 |
+
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
| 11 |
+
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
| 12 |
+
*.model filter=lfs diff=lfs merge=lfs -text
|
| 13 |
+
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
| 14 |
+
*.npy filter=lfs diff=lfs merge=lfs -text
|
| 15 |
+
*.npz filter=lfs diff=lfs merge=lfs -text
|
| 16 |
+
*.onnx filter=lfs diff=lfs merge=lfs -text
|
| 17 |
+
*.ot filter=lfs diff=lfs merge=lfs -text
|
| 18 |
+
*.parquet filter=lfs diff=lfs merge=lfs -text
|
| 19 |
+
*.pb filter=lfs diff=lfs merge=lfs -text
|
| 20 |
+
*.pickle filter=lfs diff=lfs merge=lfs -text
|
| 21 |
+
*.pkl filter=lfs diff=lfs merge=lfs -text
|
| 22 |
+
*.pt filter=lfs diff=lfs merge=lfs -text
|
| 23 |
+
*.pth filter=lfs diff=lfs merge=lfs -text
|
| 24 |
+
*.rar filter=lfs diff=lfs merge=lfs -text
|
| 25 |
+
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
| 26 |
+
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
| 27 |
+
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
| 28 |
+
*.tar filter=lfs diff=lfs merge=lfs -text
|
| 29 |
+
*.tflite filter=lfs diff=lfs merge=lfs -text
|
| 30 |
+
*.tgz filter=lfs diff=lfs merge=lfs -text
|
| 31 |
+
*.wasm filter=lfs diff=lfs merge=lfs -text
|
| 32 |
+
*.xz filter=lfs diff=lfs merge=lfs -text
|
| 33 |
+
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
+
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
+
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 36 |
+
assets/teaser.jpg filter=lfs diff=lfs merge=lfs -text
|
| 37 |
+
assets/hero.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 38 |
+
assets/hero-poster.jpg filter=lfs diff=lfs merge=lfs -text
|
| 39 |
+
assets/tradeoff.png filter=lfs diff=lfs merge=lfs -text
|
| 40 |
+
assets/method.png filter=lfs diff=lfs merge=lfs -text
|
| 41 |
+
assets/tradeoff-light.png filter=lfs diff=lfs merge=lfs -text
|
| 42 |
+
assets/tradeoff-dark.png filter=lfs diff=lfs merge=lfs -text
|
| 43 |
+
assets/method-light.png filter=lfs diff=lfs merge=lfs -text
|
| 44 |
+
assets/method-dark.png filter=lfs diff=lfs merge=lfs -text
|
LICENSE
ADDED
|
@@ -0,0 +1,88 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
Disclaimer: IMPORTANT: This Apple Machine Learning Research Model is
|
| 2 |
+
specifically developed and released by Apple Inc. ("Apple") for the sole purpose
|
| 3 |
+
of scientific research of artificial intelligence and machine-learning
|
| 4 |
+
technology. “Apple Machine Learning Research Model” means the model, including
|
| 5 |
+
but not limited to algorithms, formulas, trained model weights, parameters,
|
| 6 |
+
configurations, checkpoints, and any related materials (including
|
| 7 |
+
documentation).
|
| 8 |
+
|
| 9 |
+
This Apple Machine Learning Research Model is provided to You by
|
| 10 |
+
Apple in consideration of your agreement to the following terms, and your use,
|
| 11 |
+
modification, creation of Model Derivatives, and or redistribution of the Apple
|
| 12 |
+
Machine Learning Research Model constitutes acceptance of this Agreement. If You
|
| 13 |
+
do not agree with these terms, please do not use, modify, create Model
|
| 14 |
+
Derivatives of, or distribute this Apple Machine Learning Research Model or
|
| 15 |
+
Model Derivatives.
|
| 16 |
+
|
| 17 |
+
* License Scope: In consideration of your agreement to abide by the following
|
| 18 |
+
terms, and subject to these terms, Apple hereby grants you a personal,
|
| 19 |
+
non-exclusive, worldwide, non-transferable, royalty-free, revocable, and
|
| 20 |
+
limited license, to use, copy, modify, distribute, and create Model
|
| 21 |
+
Derivatives (defined below) of the Apple Machine Learning Research Model
|
| 22 |
+
exclusively for Research Purposes. You agree that any Model Derivatives You
|
| 23 |
+
may create or that may be created for You will be limited to Research Purposes
|
| 24 |
+
as well. “Research Purposes” means non-commercial scientific research and
|
| 25 |
+
academic development activities, such as experimentation, analysis, testing
|
| 26 |
+
conducted by You with the sole intent to advance scientific knowledge and
|
| 27 |
+
research. “Research Purposes” does not include any commercial exploitation,
|
| 28 |
+
product development or use in any commercial product or service.
|
| 29 |
+
|
| 30 |
+
* Distribution of Apple Machine Learning Research Model and Model Derivatives:
|
| 31 |
+
If you choose to redistribute Apple Machine Learning Research Model or its
|
| 32 |
+
Model Derivatives, you must provide a copy of this Agreement to such third
|
| 33 |
+
party, and ensure that the following attribution notice be provided: “Apple
|
| 34 |
+
Machine Learning Research Model is licensed under the Apple Machine Learning
|
| 35 |
+
Research Model License Agreement.” Additionally, all Model Derivatives must
|
| 36 |
+
clearly be identified as such, including disclosure of modifications and
|
| 37 |
+
changes made to the Apple Machine Learning Research Model. The name,
|
| 38 |
+
trademarks, service marks or logos of Apple may not be used to endorse or
|
| 39 |
+
promote Model Derivatives or the relationship between You and Apple. “Model
|
| 40 |
+
Derivatives” means any models or any other artifacts created by modifications,
|
| 41 |
+
improvements, adaptations, alterations to the architecture, algorithm or
|
| 42 |
+
training processes of the Apple Machine Learning Research Model, or by any
|
| 43 |
+
retraining, fine-tuning of the Apple Machine Learning Research Model.
|
| 44 |
+
|
| 45 |
+
* No Other License: Except as expressly stated in this notice, no other rights
|
| 46 |
+
or licenses, express or implied, are granted by Apple herein, including but
|
| 47 |
+
not limited to any patent, trademark, and similar intellectual property rights
|
| 48 |
+
worldwide that may be infringed by the Apple Machine Learning Research Model,
|
| 49 |
+
the Model Derivatives or by other works in which the Apple Machine Learning
|
| 50 |
+
Research Model may be incorporated.
|
| 51 |
+
|
| 52 |
+
* Compliance with Laws: Your use of Apple Machine Learning Research Model must
|
| 53 |
+
be in compliance with all applicable laws and regulations.
|
| 54 |
+
|
| 55 |
+
* Term and Termination: The term of this Agreement will begin upon your
|
| 56 |
+
acceptance of this Agreement or use of the Apple Machine Learning Research
|
| 57 |
+
Model and will continue until terminated in accordance with the following
|
| 58 |
+
terms. Apple may terminate this Agreement at any time if You are in breach of
|
| 59 |
+
any term or condition of this Agreement. Upon termination of this Agreement,
|
| 60 |
+
You must cease to use all Apple Machine Learning Research Models and Model
|
| 61 |
+
Derivatives and permanently delete any copy thereof. Sections 3, 6 and 7 will
|
| 62 |
+
survive termination.
|
| 63 |
+
|
| 64 |
+
* Disclaimer and Limitation of Liability: This Apple Machine Learning Research
|
| 65 |
+
Model and any outputs generated by the Apple Machine Learning Research Model
|
| 66 |
+
are provided on an “AS IS” basis. APPLE MAKES NO WARRANTIES, EXPRESS OR
|
| 67 |
+
IMPLIED, INCLUDING WITHOUT LIMITATION THE IMPLIED WARRANTIES OF
|
| 68 |
+
NON-INFRINGEMENT, MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE,
|
| 69 |
+
REGARDING THE APPLE MACHINE LEARNING RESEARCH MODEL OR OUTPUTS GENERATED BY
|
| 70 |
+
THE APPLE MACHINE LEARNING RESEARCH MODEL. You are solely responsible for
|
| 71 |
+
determining the appropriateness of using or redistributing the Apple Machine
|
| 72 |
+
Learning Research Model and any outputs of the Apple Machine Learning Research
|
| 73 |
+
Model and assume any risks associated with Your use of the Apple Machine
|
| 74 |
+
Learning Research Model and any output and results. IN NO EVENT SHALL APPLE BE
|
| 75 |
+
LIABLE FOR ANY SPECIAL, INDIRECT, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING
|
| 76 |
+
IN ANY WAY OUT OF THE USE, REPRODUCTION, MODIFICATION AND/OR DISTRIBUTION OF
|
| 77 |
+
THE APPLE MACHINE LEARNING RESEARCH MODEL AND ANY OUTPUTS OF THE APPLE MACHINE
|
| 78 |
+
LEARNING RESEARCH MODEL, HOWEVER CAUSED AND WHETHER UNDER THEORY OF CONTRACT,
|
| 79 |
+
TORT (INCLUDING NEGLIGENCE), STRICT LIABILITY OR OTHERWISE, EVEN IF APPLE HAS
|
| 80 |
+
BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
| 81 |
+
|
| 82 |
+
* Governing Law: This Agreement will be governed by and construed under the laws
|
| 83 |
+
of the State of California without regard to its choice of law principles. The
|
| 84 |
+
Convention on Contracts for the International Sale of Goods shall not apply to
|
| 85 |
+
the Agreement except that the arbitration clause and any arbitration hereunder
|
| 86 |
+
shall be governed by the Federal Arbitration Act, Chapters 1 and 2.
|
| 87 |
+
|
| 88 |
+
Copyright (C) 2026 Apple Inc. All Rights Reserved.
|
README.md
ADDED
|
@@ -0,0 +1,134 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: other
|
| 3 |
+
license_name: apple-amlr
|
| 4 |
+
license_link: https://github.com/apple-aiml-research/ml-depth-pro-2/blob/main/LICENSE_MODEL
|
| 5 |
+
pipeline_tag: depth-estimation
|
| 6 |
+
library_name: depth-pro-2
|
| 7 |
+
tags:
|
| 8 |
+
- depth-estimation
|
| 9 |
+
- metric-depth
|
| 10 |
+
- monocular-depth
|
| 11 |
+
- video-depth
|
| 12 |
+
---
|
| 13 |
+
|
| 14 |
+
<h1 align="center">Depth Pro 2: Sharp and Temporally Consistent Metric Monocular Depth</h1>
|
| 15 |
+
|
| 16 |
+
<p align="center">
|
| 17 |
+
<a href="https://github.com/apple-aiml-research/ml-depth-pro-2"><img src="https://img.shields.io/badge/GitHub-181717?style=flat-square&logo=github&logoColor=white" alt="GitHub"/></a>
|
| 18 |
+
<a href="https://arxiv.org/abs/TODO"><img src="https://img.shields.io/badge/Technical%20Report-B31B1B?style=flat-square&logo=arxiv&logoColor=white" alt="Technical Report"/></a>
|
| 19 |
+
<a href="https://apple-aiml-research.github.io/ml-depth-pro-2/"><img src="https://img.shields.io/badge/Project%20Page-666666?style=flat-square&logo=data%3Aimage%2Fsvg%2Bxml%3Bbase64%2CPHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHZpZXdCb3g9IjAgMCAyNCAyNCIgZmlsbD0ibm9uZSIgc3Ryb2tlPSJ3aGl0ZSIgc3Ryb2tlLXdpZHRoPSIyIj48Y2lyY2xlIGN4PSIxMiIgY3k9IjEyIiByPSIxMCIvPjxwYXRoIGQ9Ik0yIDEyaDIwTTEyIDJhMTUgMTUgMCAwIDEgMCAyME0xMiAyYTE1IDE1IDAgMCAwIDAgMjAiLz48L3N2Zz4%3D" alt="Project Page"/></a>
|
| 20 |
+
</p>
|
| 21 |
+
<!-- Hidden until the Space is public:
|
| 22 |
+
<h3 align="center">
|
| 23 |
+
🚀 <a href="https://huggingface.co/spaces/apple/DepthPro2">Try the Interactive Demo →</a>
|
| 24 |
+
</h3>
|
| 25 |
+
-->
|
| 26 |
+
|
| 27 |
+
Depth Pro 2 estimates metric depth for every pixel from an ordinary camera: a single photo, a
|
| 28 |
+
whole video, or a live feed, at the input's full resolution. Its depth is sharp at object edges
|
| 29 |
+
and steady from frame to frame.
|
| 30 |
+
|
| 31 |
+
<video src="https://huggingface.co/apple/DepthPro2/resolve/main/assets/hero.mp4" poster="https://huggingface.co/apple/DepthPro2/resolve/main/assets/hero-poster.jpg" autoplay muted loop playsinline width="100%"></video>
|
| 32 |
+
|
| 33 |
+
## Models
|
| 34 |
+
|
| 35 |
+
Each size is its own repository with its weights. Larger sizes give sharper, steadier depth;
|
| 36 |
+
smaller ones run faster.
|
| 37 |
+
|
| 38 |
+
| Model | Repository | Parameters | Download size |
|
| 39 |
+
|:---:|:---:|:---:|:---:|
|
| 40 |
+
| S | [apple/DepthPro2-S](https://huggingface.co/apple/DepthPro2-S) | 24M | 97 MB |
|
| 41 |
+
| B | [apple/DepthPro2-B](https://huggingface.co/apple/DepthPro2-B) | 97M | 386 MB |
|
| 42 |
+
| L | [apple/DepthPro2-L](https://huggingface.co/apple/DepthPro2-L) | 334M | 1.3 GB |
|
| 43 |
+
| H+ | [apple/DepthPro2-H](https://huggingface.co/apple/DepthPro2-H) | 872M | 3.5 GB |
|
| 44 |
+
|
| 45 |
+
All sizes share a focal length estimator,
|
| 46 |
+
[apple/DepthPro2-focal](https://huggingface.co/apple/DepthPro2-focal) (23.5M parameters, 94 MB),
|
| 47 |
+
which loads with each size and predicts the focal length when it is unknown.
|
| 48 |
+
|
| 49 |
+
## Quick start
|
| 50 |
+
|
| 51 |
+
```bash
|
| 52 |
+
pip install "git+https://github.com/apple-aiml-research/ml-depth-pro-2"
|
| 53 |
+
```
|
| 54 |
+
|
| 55 |
+
```python
|
| 56 |
+
from functools import partial
|
| 57 |
+
|
| 58 |
+
import torch
|
| 59 |
+
from depth_pro_2 import DepthPro2, load_rgb, upright
|
| 60 |
+
from depth_pro_2.model import to_normalized_tensor
|
| 61 |
+
|
| 62 |
+
# apple/DepthPro2-S, -B, -L or -H; bfloat16 is how the models run on a GPU.
|
| 63 |
+
model = DepthPro2.from_pretrained("apple/DepthPro2-H", precision=torch.bfloat16)
|
| 64 |
+
model = model.to("cuda")
|
| 65 |
+
transform = partial(to_normalized_tensor, device=torch.device("cuda"))
|
| 66 |
+
```
|
| 67 |
+
|
| 68 |
+
### Image
|
| 69 |
+
|
| 70 |
+
```python
|
| 71 |
+
# f_px is the focal length recorded in the photo, or None.
|
| 72 |
+
image, f_px = load_rgb("photo.jpg")
|
| 73 |
+
depth = model.infer(transform(image), f_px=f_px)["depth"] # meters, (H, W)
|
| 74 |
+
```
|
| 75 |
+
|
| 76 |
+
### Video
|
| 77 |
+
|
| 78 |
+
```python
|
| 79 |
+
import av
|
| 80 |
+
|
| 81 |
+
# 16 frames at a time: one prediction per frame, in order.
|
| 82 |
+
video = av.open("video.mp4")
|
| 83 |
+
frames = (transform(upright(f)) for f in video.decode(video=0))
|
| 84 |
+
for prediction in model.infer_video(frames):
|
| 85 |
+
depth = prediction["depth"] # meters, (H, W)
|
| 86 |
+
```
|
| 87 |
+
|
| 88 |
+
Depth is infinite where the scene is infinitely far, such as the sky. The
|
| 89 |
+
[code](https://github.com/apple-aiml-research/ml-depth-pro-2) has the command line and a live-feed
|
| 90 |
+
example.
|
| 91 |
+
|
| 92 |
+
## Results
|
| 93 |
+
|
| 94 |
+
### Sharpness and temporal consistency
|
| 95 |
+
|
| 96 |
+
<img class="block dark:hidden" src="https://huggingface.co/apple/DepthPro2/resolve/main/assets/tradeoff-light.png" alt="Boundary sharpness against temporal consistency for Depth Pro 2 and prior single-frame and multi-frame methods"/>
|
| 97 |
+
<img class="hidden dark:block" src="https://huggingface.co/apple/DepthPro2/resolve/main/assets/tradeoff-dark.png" alt="Boundary sharpness against temporal consistency for Depth Pro 2 and prior single-frame and multi-frame methods"/>
|
| 98 |
+
|
| 99 |
+
Depth Pro 2 outperforms prior work operating on a single frame or multiple frames on sharpness
|
| 100 |
+
(Boundary F1 score) and temporal consistency (Jitter). Higher F1 and lower jitter are better, so the
|
| 101 |
+
top right is best. Results averaged over Sintel and Spring datasets.
|
| 102 |
+
|
| 103 |
+
### Photos
|
| 104 |
+
|
| 105 |
+

|
| 106 |
+
|
| 107 |
+
Top row: the input photos. Bottom row: Depth Pro 2's depth, red near and blue far.
|
| 108 |
+
|
| 109 |
+
## Method
|
| 110 |
+
|
| 111 |
+
<a href="https://huggingface.co/apple/DepthPro2/blob/main/assets/method-light.png"><img class="block dark:hidden" src="https://huggingface.co/apple/DepthPro2/resolve/main/assets/method-light.png" alt="Depth Pro 2's architecture: a multi-view transformer over the input frames, then a DPT decoder per frame"/></a>
|
| 112 |
+
<a href="https://huggingface.co/apple/DepthPro2/blob/main/assets/method-dark.png"><img class="hidden dark:block" src="https://huggingface.co/apple/DepthPro2/resolve/main/assets/method-dark.png" alt="Depth Pro 2's architecture: a multi-view transformer over the input frames, then a DPT decoder per frame"/></a>
|
| 113 |
+
|
| 114 |
+
A multi-view transformer processes one or more frames jointly, and a DPT decoder predicts an inverse
|
| 115 |
+
depth map for each frame. ① After the first third of its blocks, the transformer alternates
|
| 116 |
+
attention within each frame and attention across frames. ② The decoder assembles tokens from
|
| 117 |
+
several blocks into feature maps at 1/32 to 1/4 of the input resolution.
|
| 118 |
+
|
| 119 |
+
## License
|
| 120 |
+
|
| 121 |
+
The weights are released under the [Apple Machine Learning Research Model license](https://github.com/apple-aiml-research/ml-depth-pro-2/blob/main/LICENSE_MODEL), for
|
| 122 |
+
research use only: non-commercial scientific research and academic development.
|
| 123 |
+
|
| 124 |
+
## Citation
|
| 125 |
+
|
| 126 |
+
```bibtex
|
| 127 |
+
@article{DepthPro2:2026:arxiv,
|
| 128 |
+
title = {Depth Pro 2: Sharp and Temporally Consistent Metric Monocular Depth},
|
| 129 |
+
author = {Peiyun Hu and Marko Mihajlovic and Lyndon Duong and Lahav Lipson and Wei Dong and Ama\"{e}l Delaunoy and Stephan R. Richter and Vladlen Koltun},
|
| 130 |
+
journal = {arXiv preprint arXiv:TODO},
|
| 131 |
+
year = {2026},
|
| 132 |
+
url = {https://arxiv.org/abs/TODO},
|
| 133 |
+
}
|
| 134 |
+
```
|
assets/hero-poster.jpg
ADDED
|
Git LFS Details
|
assets/hero.mp4
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:de9b022998c774145f15d9c1a307f092ea89178e274241144dbbd6b06cf38003
|
| 3 |
+
size 19092201
|
assets/method-dark.png
ADDED
|
Git LFS Details
|
assets/method-light.png
ADDED
|
Git LFS Details
|
assets/teaser.jpg
ADDED
|
Git LFS Details
|
assets/tradeoff-dark.png
ADDED
|
Git LFS Details
|
assets/tradeoff-light.png
ADDED
|
Git LFS Details
|