Spaces:
Runtime error
Runtime error
Upload README.md
Browse files
README.md
CHANGED
|
@@ -1,137 +1,11 @@
|
|
| 1 |
-
|
| 2 |
-
|
| 3 |
-
|
| 4 |
-
:
|
| 5 |
-
|
| 6 |
-
|
| 7 |
-
|
| 8 |
-
|
| 9 |
-
|
| 10 |
-
|
| 11 |
-
|
| 12 |
-
<a href="https://scholar.google.com/citations?user=6CIDtZQAAAAJ&hl=zh-CN">Jian Yang✉</a><sup>1</sup> <!--‡-->
|
| 13 |
-
</p>
|
| 14 |
-
|
| 15 |
-
<p align="center">
|
| 16 |
-
<sup>*</sup>Equal contribution
|
| 17 |
-
<sup>✉</sup>Corresponding author <br>
|
| 18 |
-
<sup>1</sup>Nanjing University of Science and Technology
|
| 19 |
-
<br>
|
| 20 |
-
<sup>2</sup>Nanjing University of Posts and Telecommunications
|
| 21 |
-
<sup>3</sup>Nanjing University
|
| 22 |
-
</p>
|
| 23 |
-
|
| 24 |
-
<p align="center">
|
| 25 |
-
<img src="Figs/Pipeline.png", width="800"/>
|
| 26 |
-
</p>
|
| 27 |
-
|
| 28 |
-
|
| 29 |
-
Overview of DORNet. Given $\boldsymbol D_{up}$ as input, the degradation learning first encodes it to produce degradation representations $\boldsymbol {\tilde{D}}$ and $\boldsymbol D $. Then, $\boldsymbol {\tilde{D}}$, $\boldsymbol D $, $\boldsymbol D_{lr} $, and $\boldsymbol I_{r}$ are fed into multiple degradation-oriented feature transformation (DOFT) modules, generating the HR depth $\boldsymbol D_{hr}$. Finally, $\boldsymbol D$ and $\boldsymbol D_{hr}$ are sent to the degradation regularization to obtain $\boldsymbol D_{d}$, which is used as input for the degradation loss $\mathcal L_{deg}$ and the contrastive loss $\mathcal L_{cont}$. The degradation regularization only applies during training and adds no extra overhead in testing.
|
| 30 |
-
|
| 31 |
-
## Dependencies
|
| 32 |
-
|
| 33 |
-
```bash
|
| 34 |
-
Python==3.11.5
|
| 35 |
-
PyTorch==2.1.0
|
| 36 |
-
numpy==1.23.5
|
| 37 |
-
torchvision==0.16.0
|
| 38 |
-
scipy==1.11.3
|
| 39 |
-
Pillow==10.0.1
|
| 40 |
-
tqdm==4.65.0
|
| 41 |
-
scikit-image==0.21.0
|
| 42 |
-
mmcv-full==1.7.2
|
| 43 |
-
```
|
| 44 |
-
|
| 45 |
-
## Datasets
|
| 46 |
-
|
| 47 |
-
[RGB-D-D](https://github.com/lingzhi96/RGB-D-D-Dataset)
|
| 48 |
-
|
| 49 |
-
[TOFDSR](https://yanzq95.github.io/projectpage/TOFDC/index.html)
|
| 50 |
-
|
| 51 |
-
[NYU-v2](https://cs.nyu.edu/~fergus/datasets/nyu_depth_v2.html)
|
| 52 |
-
|
| 53 |
-
## Models
|
| 54 |
-
|
| 55 |
-
Pretrained models can be found in <a href="https://github.com/yanzq95/DORNet/tree/main/checkpoints">checkpoints</a>.
|
| 56 |
-
|
| 57 |
-
|
| 58 |
-
## Training
|
| 59 |
-
|
| 60 |
-
For the RGB-D-D and NYU-v2 datasets, we use a single GPU to train our DORNet. For the larger TOFDC dataset, we employ multiple GPUs to accelerate training.
|
| 61 |
-
|
| 62 |
-
### DORNet
|
| 63 |
-
```
|
| 64 |
-
Train on real-world RGB-D-D
|
| 65 |
-
> python train_nyu_rgbdd.py
|
| 66 |
-
Train on real-world TOFDSR
|
| 67 |
-
> python -m torch.distributed.launch --nproc_per_node 2 train_tofdsr.py --result_root 'experiment/TOFDSR'
|
| 68 |
-
Train on synthetic NYU-v2
|
| 69 |
-
> python train_nyu_rgbdd.py
|
| 70 |
-
```
|
| 71 |
-
|
| 72 |
-
### DORNet-T
|
| 73 |
-
```
|
| 74 |
-
Train on real-world RGB-D-D
|
| 75 |
-
> python train_nyu_rgbdd.py --tiny_model
|
| 76 |
-
Train on real-world TOFDSR
|
| 77 |
-
> python -m torch.distributed.launch --nproc_per_node 2 train_tofdsr.py --result_root 'experiment/TOFDSR_T' --tiny_model
|
| 78 |
-
Train on synthetic NYU-v2
|
| 79 |
-
> python train_nyu_rgbdd.py --tiny_model
|
| 80 |
-
```
|
| 81 |
-
|
| 82 |
-
## Testing
|
| 83 |
-
|
| 84 |
-
### DORNet
|
| 85 |
-
```
|
| 86 |
-
Test on real-world RGB-D-D
|
| 87 |
-
> python test_nyu_rgbdd.py
|
| 88 |
-
Test on real-world TOFDSR
|
| 89 |
-
> python test_tofdsr.py
|
| 90 |
-
Test on synthetic NYU-v2
|
| 91 |
-
> python test_nyu_rgbdd.py
|
| 92 |
-
```
|
| 93 |
-
|
| 94 |
-
### DORNet-T
|
| 95 |
-
```
|
| 96 |
-
Test on real-world RGB-D-D
|
| 97 |
-
> python test_nyu_rgbdd.py --tiny_model
|
| 98 |
-
Test on real-world TOFDSR
|
| 99 |
-
> python test_tofdsr.py --tiny_model
|
| 100 |
-
Test on synthetic NYU-v2
|
| 101 |
-
> python test_nyu_rgbdd.py --tiny_model
|
| 102 |
-
```
|
| 103 |
-
|
| 104 |
-
## Experiments
|
| 105 |
-
|
| 106 |
-
### Quantitative comparison
|
| 107 |
-
|
| 108 |
-
<p align="center">
|
| 109 |
-
<img src="Figs/Params_Time.png", width="500"/>
|
| 110 |
-
<br>
|
| 111 |
-
Complexity on RGB-D-D (w/o Noisy) tested by a 4090 GPU. A larger circle diameter indicates a higher inference time.
|
| 112 |
-
</p>
|
| 113 |
-
|
| 114 |
-
|
| 115 |
-
|
| 116 |
-
### Visual comparison
|
| 117 |
-
|
| 118 |
-
<p align="center">
|
| 119 |
-
<img src="Figs/RGBDD.png", width="1000"/>
|
| 120 |
-
<br>
|
| 121 |
-
Visual results on the real-world RGB-D-D dataset (w/o Noise).
|
| 122 |
-
</p>
|
| 123 |
-
|
| 124 |
-
|
| 125 |
-
## Citation
|
| 126 |
-
|
| 127 |
-
If our method proves to be of any assistance, please consider citing:
|
| 128 |
-
```
|
| 129 |
-
@inproceedings{wang2025dornet,
|
| 130 |
-
title={DORNet: A Degradation Oriented and Regularized Network for Blind Depth Super-Resolution},
|
| 131 |
-
author={Wang, Zhengxue and Yan, Zhiqiang and Pan, Jinshan and Gao, Guangwei and Zhang, Kai and Yang, Jian},
|
| 132 |
-
booktitle={Proceedings of the Computer Vision and Pattern Recognition Conference},
|
| 133 |
-
pages={15813--15822},
|
| 134 |
-
year={2025}
|
| 135 |
-
}
|
| 136 |
-
```
|
| 137 |
-
|
|
|
|
| 1 |
+
---
|
| 2 |
+
title: DORNet
|
| 3 |
+
emoji: 🌍
|
| 4 |
+
colorFrom: yellow
|
| 5 |
+
colorTo: indigo
|
| 6 |
+
sdk: gradio
|
| 7 |
+
sdk_version: 3.50.2
|
| 8 |
+
app_file: app.py
|
| 9 |
+
pinned: false
|
| 10 |
+
---
|
| 11 |
+
[DORNet: A Degradation Oriented and Regularized Network for Blind Depth Super-Resolution](https://arxiv.org/pdf/2410.11666)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|