RaynWu2002 commited on
Commit
304f228
·
verified ·
1 Parent(s): e5854fc

Upload README.md

Browse files
Files changed (1) hide show
  1. README.md +11 -137
README.md CHANGED
@@ -1,137 +1,11 @@
1
- <p align="center">
2
- <h3 align="center"> DORNet: A Degradation Oriented and Regularized Network for <br> Blind Depth Super-Resolution
3
- <br>
4
- :star2: CVPR 2025 (Oral Presentation) :star2:
5
- </h3>
6
-
7
- <p align="center"><a href="https://scholar.google.com/citations?user=VogTuQkAAAAJ&hl=zh-CN">Zhengxue Wang</a><sup>1*</sup>,
8
- <a href="https://yanzq95.github.io/">Zhiqiang Yan✉</a><sup>1*</sup>,
9
- <a href="https://jspan.github.io/">Jinshan Pan</a><sup>1</sup>,
10
- <a href="https://guangweigao.github.io/">Guangwei Gao</a><sup>2</sup>,
11
- <a href="https://cszn.github.io/">Kai Zhang</a><sup>3</sup>,
12
- <a href="https://scholar.google.com/citations?user=6CIDtZQAAAAJ&hl=zh-CN">Jian Yang✉</a><sup>1</sup> <!--&Dagger;-->
13
- </p>
14
-
15
- <p align="center">
16
- <sup>*</sup>Equal contribution&nbsp;&nbsp;&nbsp;
17
- <sup>✉</sup>Corresponding author&nbsp;&nbsp;&nbsp;<br>
18
- <sup>1</sup>Nanjing University of Science and Technology&nbsp;&nbsp;&nbsp;
19
- <br>
20
- <sup>2</sup>Nanjing University of Posts and Telecommunications&nbsp;&nbsp;&nbsp;
21
- <sup>3</sup>Nanjing University&nbsp;&nbsp;&nbsp;
22
- </p>
23
-
24
- <p align="center">
25
- <img src="Figs/Pipeline.png", width="800"/>
26
- </p>
27
-
28
-
29
- Overview of DORNet. Given $\boldsymbol D_{up}$ as input, the degradation learning first encodes it to produce degradation representations $\boldsymbol {\tilde{D}}$ and $\boldsymbol D $. Then, $\boldsymbol {\tilde{D}}$, $\boldsymbol D $, $\boldsymbol D_{lr} $, and $\boldsymbol I_{r}$ are fed into multiple degradation-oriented feature transformation (DOFT) modules, generating the HR depth $\boldsymbol D_{hr}$. Finally, $\boldsymbol D$ and $\boldsymbol D_{hr}$ are sent to the degradation regularization to obtain $\boldsymbol D_{d}$, which is used as input for the degradation loss $\mathcal L_{deg}$ and the contrastive loss $\mathcal L_{cont}$. The degradation regularization only applies during training and adds no extra overhead in testing.
30
-
31
- ## Dependencies
32
-
33
- ```bash
34
- Python==3.11.5
35
- PyTorch==2.1.0
36
- numpy==1.23.5
37
- torchvision==0.16.0
38
- scipy==1.11.3
39
- Pillow==10.0.1
40
- tqdm==4.65.0
41
- scikit-image==0.21.0
42
- mmcv-full==1.7.2
43
- ```
44
-
45
- ## Datasets
46
-
47
- [RGB-D-D](https://github.com/lingzhi96/RGB-D-D-Dataset)
48
-
49
- [TOFDSR](https://yanzq95.github.io/projectpage/TOFDC/index.html)
50
-
51
- [NYU-v2](https://cs.nyu.edu/~fergus/datasets/nyu_depth_v2.html)
52
-
53
- ## Models
54
-
55
- Pretrained models can be found in <a href="https://github.com/yanzq95/DORNet/tree/main/checkpoints">checkpoints</a>.
56
-
57
-
58
- ## Training
59
-
60
- For the RGB-D-D and NYU-v2 datasets, we use a single GPU to train our DORNet. For the larger TOFDC dataset, we employ multiple GPUs to accelerate training.
61
-
62
- ### DORNet
63
- ```
64
- Train on real-world RGB-D-D
65
- > python train_nyu_rgbdd.py
66
- Train on real-world TOFDSR
67
- > python -m torch.distributed.launch --nproc_per_node 2 train_tofdsr.py --result_root 'experiment/TOFDSR'
68
- Train on synthetic NYU-v2
69
- > python train_nyu_rgbdd.py
70
- ```
71
-
72
- ### DORNet-T
73
- ```
74
- Train on real-world RGB-D-D
75
- > python train_nyu_rgbdd.py --tiny_model
76
- Train on real-world TOFDSR
77
- > python -m torch.distributed.launch --nproc_per_node 2 train_tofdsr.py --result_root 'experiment/TOFDSR_T' --tiny_model
78
- Train on synthetic NYU-v2
79
- > python train_nyu_rgbdd.py --tiny_model
80
- ```
81
-
82
- ## Testing
83
-
84
- ### DORNet
85
- ```
86
- Test on real-world RGB-D-D
87
- > python test_nyu_rgbdd.py
88
- Test on real-world TOFDSR
89
- > python test_tofdsr.py
90
- Test on synthetic NYU-v2
91
- > python test_nyu_rgbdd.py
92
- ```
93
-
94
- ### DORNet-T
95
- ```
96
- Test on real-world RGB-D-D
97
- > python test_nyu_rgbdd.py --tiny_model
98
- Test on real-world TOFDSR
99
- > python test_tofdsr.py --tiny_model
100
- Test on synthetic NYU-v2
101
- > python test_nyu_rgbdd.py --tiny_model
102
- ```
103
-
104
- ## Experiments
105
-
106
- ### Quantitative comparison
107
-
108
- <p align="center">
109
- <img src="Figs/Params_Time.png", width="500"/>
110
- <br>
111
- Complexity on RGB-D-D (w/o Noisy) tested by a 4090 GPU. A larger circle diameter indicates a higher inference time.
112
- </p>
113
-
114
-
115
-
116
- ### Visual comparison
117
-
118
- <p align="center">
119
- <img src="Figs/RGBDD.png", width="1000"/>
120
- <br>
121
- Visual results on the real-world RGB-D-D dataset (w/o Noise).
122
- </p>
123
-
124
-
125
- ## Citation
126
-
127
- If our method proves to be of any assistance, please consider citing:
128
- ```
129
- @inproceedings{wang2025dornet,
130
- title={DORNet: A Degradation Oriented and Regularized Network for Blind Depth Super-Resolution},
131
- author={Wang, Zhengxue and Yan, Zhiqiang and Pan, Jinshan and Gao, Guangwei and Zhang, Kai and Yang, Jian},
132
- booktitle={Proceedings of the Computer Vision and Pattern Recognition Conference},
133
- pages={15813--15822},
134
- year={2025}
135
- }
136
- ```
137
-
 
1
+ ---
2
+ title: DORNet
3
+ emoji: 🌍
4
+ colorFrom: yellow
5
+ colorTo: indigo
6
+ sdk: gradio
7
+ sdk_version: 3.50.2
8
+ app_file: app.py
9
+ pinned: false
10
+ ---
11
+ [DORNet: A Degradation Oriented and Regularized Network for Blind Depth Super-Resolution](https://arxiv.org/pdf/2410.11666)