Upload folder using huggingface_hub
Browse files
DepthProMetric1024.mlpackage/Data/com.apple.CoreML/model.mlmodel
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:c035cbaf691056c1546a17e2e9ddaa024e31a310195e800d434f0fafdbc88255
|
| 3 |
+
size 1118793
|
DepthProMetric1024.mlpackage/Data/com.apple.CoreML/weights/weight.bin
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:aaf4d71dc47865a6cd19f22084d193f4d46b752330a748b8854dab5e45a8bf4d
|
| 3 |
+
size 714474304
|
DepthProMetric1024.mlpackage/Manifest.json
ADDED
|
@@ -0,0 +1,18 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"fileFormatVersion": "1.0.0",
|
| 3 |
+
"itemInfoEntries": {
|
| 4 |
+
"31A01AA5-7FBD-45DB-AA21-B51D03998597": {
|
| 5 |
+
"author": "com.apple.CoreML",
|
| 6 |
+
"description": "CoreML Model Weights",
|
| 7 |
+
"name": "weights",
|
| 8 |
+
"path": "com.apple.CoreML/weights"
|
| 9 |
+
},
|
| 10 |
+
"DF8A79AB-286A-4734-A4EA-B88031155FBF": {
|
| 11 |
+
"author": "com.apple.CoreML",
|
| 12 |
+
"description": "CoreML Model Specification",
|
| 13 |
+
"name": "model.mlmodel",
|
| 14 |
+
"path": "com.apple.CoreML/model.mlmodel"
|
| 15 |
+
}
|
| 16 |
+
},
|
| 17 |
+
"rootModelIdentifier": "DF8A79AB-286A-4734-A4EA-B88031155FBF"
|
| 18 |
+
}
|
LICENSE
ADDED
|
@@ -0,0 +1,47 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
Copyright (C) 2024 Apple Inc. All Rights Reserved.
|
| 2 |
+
|
| 3 |
+
Disclaimer: IMPORTANT: This Apple software is supplied to you by Apple
|
| 4 |
+
Inc. ("Apple") in consideration of your agreement to the following
|
| 5 |
+
terms, and your use, installation, modification or redistribution of
|
| 6 |
+
this Apple software constitutes acceptance of these terms. If you do
|
| 7 |
+
not agree with these terms, please do not use, install, modify or
|
| 8 |
+
redistribute this Apple software.
|
| 9 |
+
|
| 10 |
+
In consideration of your agreement to abide by the following terms, and
|
| 11 |
+
subject to these terms, Apple grants you a personal, non-exclusive
|
| 12 |
+
license, under Apple's copyrights in this original Apple software (the
|
| 13 |
+
"Apple Software"), to use, reproduce, modify and redistribute the Apple
|
| 14 |
+
Software, with or without modifications, in source and/or binary forms;
|
| 15 |
+
provided that if you redistribute the Apple Software in its entirety and
|
| 16 |
+
without modifications, you must retain this notice and the following
|
| 17 |
+
text and disclaimers in all such redistributions of the Apple Software.
|
| 18 |
+
Neither the name, trademarks, service marks or logos of Apple Inc. may
|
| 19 |
+
be used to endorse or promote products derived from the Apple Software
|
| 20 |
+
without specific prior written permission from Apple. Except as
|
| 21 |
+
expressly stated in this notice, no other rights or licenses, express or
|
| 22 |
+
implied, are granted by Apple herein, including but not limited to any
|
| 23 |
+
patent rights that may be infringed by your derivative works or by other
|
| 24 |
+
works in which the Apple Software may be incorporated.
|
| 25 |
+
|
| 26 |
+
The Apple Software is provided by Apple on an "AS IS" basis. APPLE
|
| 27 |
+
MAKES NO WARRANTIES, EXPRESS OR IMPLIED, INCLUDING WITHOUT LIMITATION
|
| 28 |
+
THE IMPLIED WARRANTIES OF NON-INFRINGEMENT, MERCHANTABILITY AND FITNESS
|
| 29 |
+
FOR A PARTICULAR PURPOSE, REGARDING THE APPLE SOFTWARE OR ITS USE AND
|
| 30 |
+
OPERATION ALONE OR IN COMBINATION WITH YOUR PRODUCTS.
|
| 31 |
+
|
| 32 |
+
IN NO EVENT SHALL APPLE BE LIABLE FOR ANY SPECIAL, INDIRECT, INCIDENTAL
|
| 33 |
+
OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
|
| 34 |
+
SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
|
| 35 |
+
INTERRUPTION) ARISING IN ANY WAY OUT OF THE USE, REPRODUCTION,
|
| 36 |
+
MODIFICATION AND/OR DISTRIBUTION OF THE APPLE SOFTWARE, HOWEVER CAUSED
|
| 37 |
+
AND WHETHER UNDER THEORY OF CONTRACT, TORT (INCLUDING NEGLIGENCE),
|
| 38 |
+
STRICT LIABILITY OR OTHERWISE, EVEN IF APPLE HAS BEEN ADVISED OF THE
|
| 39 |
+
POSSIBILITY OF SUCH DAMAGE.
|
| 40 |
+
|
| 41 |
+
|
| 42 |
+
-------------------------------------------------------------------------------
|
| 43 |
+
SOFTWARE DISTRIBUTED IN THIS REPOSITORY:
|
| 44 |
+
|
| 45 |
+
This software includes a number of subcomponents with separate
|
| 46 |
+
copyright notices and license terms - please see the file ACKNOWLEDGEMENTS.
|
| 47 |
+
-------------------------------------------------------------------------------
|
README.md
ADDED
|
@@ -0,0 +1,73 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: other
|
| 3 |
+
license_name: apple-sample-code-license
|
| 4 |
+
license_link: LICENSE
|
| 5 |
+
pipeline_tag: depth-estimation
|
| 6 |
+
tags:
|
| 7 |
+
- coreml
|
| 8 |
+
- depth
|
| 9 |
+
- metric-depth
|
| 10 |
+
- apple-silicon
|
| 11 |
+
- touchdesigner
|
| 12 |
+
---
|
| 13 |
+
|
| 14 |
+
# Depth Pro — Core ML (metric depth, 1024, 6-bit palettized)
|
| 15 |
+
|
| 16 |
+
A Core ML conversion of [Apple's Depth Pro](https://github.com/apple/ml-depth-pro)
|
| 17 |
+
— **metric** monocular depth: absolute metres from a single image, with no
|
| 18 |
+
camera intrinsics, plus an estimated focal length.
|
| 19 |
+
|
| 20 |
+
Converted for [AML (TD Apple ML)](https://github.com/mickeyvanolst), where it
|
| 21 |
+
runs in the CoreML TOP inside TouchDesigner, but there is nothing
|
| 22 |
+
TouchDesigner-specific about the package.
|
| 23 |
+
|
| 24 |
+
## What it gives you
|
| 25 |
+
|
| 26 |
+
| output | shape | meaning |
|
| 27 |
+
|---|---|---|
|
| 28 |
+
| `depth_m` | `[1, 1, 1024, 1024]` | depth in **metres** |
|
| 29 |
+
| `focallength_px` | `[1]` | estimated focal length in pixels |
|
| 30 |
+
|
| 31 |
+
Input: `image`, 1536×1536 RGB (the network's own size — the depth map is
|
| 32 |
+
resized to 1024 inside the graph, while the values are still float).
|
| 33 |
+
|
| 34 |
+
## How it was converted
|
| 35 |
+
|
| 36 |
+
1. `depth_pro.pt` from Apple's CDN (the repo's own `get_pretrained_models.sh`).
|
| 37 |
+
2. Traced at 1536×1536 and converted with coremltools 9 (fp16 compute).
|
| 38 |
+
`deg2rad` has no Core ML conversion and was replaced with a multiply.
|
| 39 |
+
3. The metric arithmetic from `DepthPro.infer` is baked into the graph, so
|
| 40 |
+
the model returns metres rather than canonical inverse depth:
|
| 41 |
+
`f_px = 0.5·W / tan(0.5·fov)`, `depth = 1 / (canonical · W / f_px)`.
|
| 42 |
+
4. The depth map is resized to 1024² in-graph.
|
| 43 |
+
5. Weights palettized to 6 bits (kmeans, per-tensor).
|
| 44 |
+
|
| 45 |
+
## Accuracy and cost
|
| 46 |
+
|
| 47 |
+
Measured against the PyTorch reference on the same photograph, Apple silicon
|
| 48 |
+
(fanless M-series):
|
| 49 |
+
|
| 50 |
+
| | size | per frame | vs PyTorch |
|
| 51 |
+
|---|---|---|---|
|
| 52 |
+
| fp16 | 1.8 GB | 4.8 s | median 0.15 % |
|
| 53 |
+
| **6-bit (this package)** | **682 MB** | **3.6 s** | median 1.17 %, p95 4.5 % |
|
| 54 |
+
|
| 55 |
+
First load compiles for the Neural Engine and takes a couple of minutes;
|
| 56 |
+
afterwards it is cached by the OS.
|
| 57 |
+
|
| 58 |
+
## Licence
|
| 59 |
+
|
| 60 |
+
Apple's licence for Depth Pro, redistributed verbatim as `LICENSE`. The
|
| 61 |
+
weights come from the repository's own download script; this package is a
|
| 62 |
+
format conversion of them.
|
| 63 |
+
|
| 64 |
+
## Citation
|
| 65 |
+
|
| 66 |
+
@inproceedings{Bochkovskii2025:depthpro,
|
| 67 |
+
title = {Depth Pro: Sharp Monocular Metric Depth in Less Than a Second},
|
| 68 |
+
author = {Aleksei Bochkovskii and Ama\"{e}l Delaunoy and Hugo Germain and
|
| 69 |
+
Marcel Santos and Yichao Zhou and Stephan R. Richter and
|
| 70 |
+
Vladlen Koltun},
|
| 71 |
+
booktitle = {International Conference on Learning Representations},
|
| 72 |
+
year = {2025},
|
| 73 |
+
}
|