Hugging Face's logo

GWu
/
TUR

Model card Files
xet
 Community
GWu commited on
Commit
0caa4ef
·
verified ·
1 Parent(s): 8bf6733

Upload 6 files

Browse files
Files changed (6) hide show
  1. AirNet_TUR_3task.pth +3 -0
  2. README.md +131 -0
  3. Transweather_TUR_5task.pth +3 -0
  4. Transweather_TUR_AllWeather.pth +3 -0
  5. Transweather_TUR_real.pth +3 -0
  6. Uformer_TUR_7task.pth +3 -0
AirNet_TUR_3task.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4f5bc463358053ebd5b6d6e5279e4b2ce0a812177ac46e975db7f30977b079f3
3
+ size 39422181
README.md CHANGED
@@ -1,3 +1,134 @@
1
  ---
2
  license: cc-by-4.0
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: cc-by-4.0
3
  ---
4
+
5
+
6
+
7
+ <div align="center">
8
+
9
+ # [AAAI 2025] Debiased All-in-one Image Restoration with Task Uncertainty Regularization
10
+
11
+ [Gang Wu (吴刚)](https://scholar.google.com/citations?user=JSqb7QIAAAAJ), [Junjun Jiang (江俊君)](http://homepage.hit.edu.cn/jiangjunjun), [Yijun Wang (王奕钧)](), [Kui Jiang (江奎)](https://github.com/kuijiang94), and [Xianming Liu (刘贤明)](http://homepage.hit.edu.cn/xmliu)
12
+
13
+ [AIIA Lab](https://aiialabhit.github.io/team/), Faculty of Computing, Harbin Institute of Technology, Harbin 150001, China.
14
+
15
+ [![Paper](http://img.shields.io/badge/Paper-OpenReview-FF6B6B.svg)](https://openreview.net/forum?id=kx7eyKgEGz&referrer=%5BAuthor%20Console%5D(%2Fgroup%3Fid%3DAAAI.org%2F2025%2FConference%2FAuthors%23your-submissions))
16
+ [![arXiv](https://img.shields.io/badge/AAAI-2025-red.svg)]()
17
+ [![Models](https://img.shields.io/badge/BaiduPan-Models-blue.svg)](https://pan.baidu.com/s/1YN3P-CmnisXVIdLHTWB9Fw?pwd=AAAI)
18
+ <a href="https://hits.seeyoufarm.com"><img src="https://hits.seeyoufarm.com/api/count/incr/badge.svg?url=https%3A%2F%2Fgithub.com%2FAitical%2FTUR%2F&count_bg=%2379C83D&title_bg=%23555555&icon=&icon_color=%23E7E7E7&title=hits&edge_flat=false"/></a>
19
+ </div>
20
+
21
+ </div>
22
+
23
+ >All-in-one image restoration is a fundamental low-level vision task with significant real-world applications. The primary challenge lies in addressing diverse degradations within a single model.
24
+ While current methods primarily exploit task prior information to guide the restoration models, they typically employ uniform multi-task learning, overlooking the heterogeneity in model optimization across different degradation tasks.
25
+ To eliminate the bias,
26
+ we propose a task-aware optimization strategy, that introduces adaptive task-specific regularization for multi-task image restoration learning.
27
+ Specifically, our method dynamically weights and balances losses for different restoration tasks during training, encouraging the implementation of the most reasonable optimization route. In this way, we can achieve more robust and effective model training.
28
+ Notably, our approach can serve as a plug-and-play strategy to enhance existing models without requiring modifications during inference.
29
+ Extensive experiments across diverse all-in-one restoration settings demonstrate the superiority and generalization of our approach. For instance, AirNet retrained with TUR achieves average improvements of **1.16 dB** on three distinct tasks and **1.81 dB** on five distinct all-in-one tasks. These results underscore TUR's effectiveness in advancing the SOTAs in all-in-one image restoration, paving the way for more robust and versatile image restoration.
30
+
31
+ ## Overview
32
+ <a href="https://www.imagehub.cc/image/WX20241217-222854%402x.CUG4nh"><img src="https://s1.imagehub.cc/images/2024/12/17/5ccbe23fd247fb3142f13945fa8a44dc.png" alt="WX20241217 222854@2x" border="0"></a>
33
+
34
+ ## Results
35
+
36
+ Table: Comparison on 7 distinct degradation tasks introduced in [1].
37
+
38
+ | Methods | SR | Blur | Noise | JPEG | Rain | Haze | Dark | Avg. |
39
+ |---------|-----|------|-------|------|------|------|------|------|
40
+ | SRResNet | 25.52 | 30.01 | 30.49 | 32.46 | 32.38 | 25.57 | 30.20 | 29.52 |
41
+ | SRResNet-S [1] | 25.72 | 30.49 | 30.67 | 32.73 | 32.81 | 25.78 | 30.45 | 29.84 |
42
+ | **SRResNet (Ours)** | **25.55** | **30.65** | **30.65** | **32.92** | **35.20** | **26.16** | **32.04** | **30.45** |
43
+ | Uformer | 25.80 | 30.53 | 30.84 | 33.13 | 33.39 | 27.93 | 33.27 | 30.70 |
44
+ | Uformer-S [1] | 26.07 | 31.11 | 30.96 | 33.27 | 35.96 | 28.29 | 32.80 | 31.21 |
45
+ | **Uformer (Ours)** | **26.11** | **31.51** | **31.20** | **33.46** | **38.13** | **30.91** | **38.24** | **32.79** |
46
+
47
+
48
+ Table: Comparisons under the three-degradation all-in-one setting: a unified model is trained on a combined set of images obtained from all degradation types and levels.
49
+ | Method | Dehazing on SOTS | Deraining on Rain100L | Denoising on BSD68 $\sigma=15$ | Denoising on BSD68 $\sigma=25$ | Denoising on BSD68 $\sigma=50$ | Average |
50
+ |--------|-------------------|------------------------|--------------------------------|--------------------------------|--------------------------------|---------|
51
+ | BRDNet [1] | 23.23/0.895 | 27.42/0.895 | 32.26/0.898 | 29.76/0.836 | 26.34/0.693 | 27.80/0.843 |
52
+ | LPNet [2] | 20.84/0.828 | 24.88/0.784 | 26.47/0.778 | 24.77/0.748 | 21.26/0.552 | 23.64/0.738 |
53
+ | FDGAN [3] | 24.71/0.929 | 29.89/0.933 | 30.25/0.910 | 28.81/0.868 | 26.43/0.776 | 28.02/0.883 |
54
+ | MPRNet [4] | 25.28/0.955 | 33.57/0.954 | 33.54/0.927 | 30.89/0.880 | 27.56/0.779 | 30.17/0.899 |
55
+ | DL [5] | 26.92/0.931 | 32.62/0.931 | 33.05/0.914 | 30.41/0.861 | 26.90/0.740 | 29.98/0.876 |
56
+ | AirNet [6] | 27.94/0.962 | 34.90/0.968 | 33.92/0.933 | 31.26/0.888 | 28.00/0.797 | 31.20/0.910 |
57
+ | **AirNet (Ours)** | **30.41/0.976** | **38.04/0.983** | **33.97/0.931** | **31.32/0.886** | **28.05/0.795** | **32.50/0.914** |
58
+ | PromptIR [7] | 30.58/0.974 | 36.37/0.972 | 33.98/0.933 | 31.31/0.888 | 28.06/0.799 | 32.06/0.913 |
59
+ | **PromptIR (Ours)** | **31.17/0.978** | **38.57/0.984** | **34.06/0.932** | **31.40/0.887** | **28.13/0.797** | **32.67/0.916** |
60
+
61
+
62
+ Table: Comparative results on five distinct tasks in all-in-one image restoration.
63
+
64
+ | Method | Dehazing on SOTS | Deraining on Rain100L | Denoising on BSD68 | Deblurring on GoPro | Low-Light on LOL | Average |
65
+ |--------|-------------------|------------------------|---------------------|----------------------|-------------------|---------|
66
+ | NAFNet [1] | 25.23/0.939 | 35.56/0.967 | 31.02/0.883 | 26.53/0.808 | 20.49/0.809 | 27.76/0.881 |
67
+ | MPRNet [2] | 24.27/0.937 | 38.16/0.981 | 31.35/0.889 | 26.87/0.823 | 20.84/0.824 | 28.27/0.890 |
68
+ | SwinIR [3] | 21.50/0.891 | 30.78/0.923 | 30.59/0.868 | 24.52/0.773 | 17.81/0.723 | 25.04/0.835 |
69
+ | DL [4] | 20.54/0.826 | 21.96/0.762 | 23.09/0.745 | 19.86/0.672 | 19.83/0.712 | 21.05/0.743 |
70
+ | TAPE [5] | 22.16/0.861 | 29.67/0.904 | 30.18/0.855 | 24.47/0.763 | 18.97/0.621 | 25.09/0.801 |
71
+ | IDR [6] | 25.24/0.943 | 35.63/0.965 | 31.60/0.887 | 27.87/0.846 | 21.34/0.826 | 28.34/0.893 |
72
+ | Transweather [7] | 21.32/0.885 | 29.43/0.905 | 29.00/0.841 | 25.12/0.757 | 21.21/0.792 | 25.22/0.836 |
73
+ | **Transweather (Ours)** | **29.68/0.966** | **33.09/0.952** | **30.40/0.869** | **26.63/0.815** | **23.02/0.838** | **28.56/0.888** |
74
+ | AirNet [8] | 21.04/0.884 | 32.98/0.951 | 30.91/0.882 | 24.35/0.781 | 18.18/0.735 | 25.49/0.846 |
75
+ | **AirNet (Ours)** | **27.59/0.954** | **33.95/0.962** | **30.93/0.875** | **26.13/0.801** | **17.88/0.772** | **27.30/0.873** |
76
+
77
+ Table: Comparison on deweathering tasks with Allweather dataset [3].
78
+
79
+ | Datasets | Method | PSNR ↑ | SSIM ↑ |
80
+ |----------|--------|--------|--------|
81
+ | Outdoor-Rain | All-in-One [1] | 24.71 | 0.8980 |
82
+ | | WeatherDiff₁₂₈ [2] | 29.72 | 0.9216 |
83
+ | | TransWeather [3] | 28.83 | 0.9000 |
84
+ | | **TransWeather (Ours)** | **29.75** | **0.9073** |
85
+ | Snow100K | DDMSNet [4] | 28.85 | 0.8772 |
86
+ | | All-in-One [1] | 28.33 | 0.8820 |
87
+ | | WeatherDiff₁₂₈ [2] | 29.58 | 0.8941 |
88
+ | | TransWeather [3] | 29.31 | 0.8879 |
89
+ | | **TransWeather (Ours)** | **30.62** | **0.9086** |
90
+ | RainDrop | All-in-One [1] | 31.12 | 0.9268 |
91
+ | | WeatherDiff₁₂₈ [2] | 29.66 | 0.9225 |
92
+ | | TransWeather [3] | 30.17 | 0.9157 |
93
+ | | **TransWeather (Ours)** | **31.61** | **0.9330** |
94
+ | Average | All-in-One [1] | 27.12 | 0.8933 |
95
+ | | WeatherDiff₁₂₈ [2] | 29.65 | 0.9127 |
96
+ | | TransWeather [3] | 29.44 | 0.9012 |
97
+ | | **TransWeather (Ours)** | **30.66** | **0.9163** |
98
+
99
+
100
+ Table: Comparison on de-weathering tasks on real-world datasets following [2]
101
+ | Datasets | Method | PSNR ↑ | SSIM ↑ |
102
+ |----------|--------|--------|--------|
103
+ | SPA+ | Chen et al. [1] | 37.32 | 0.97 |
104
+ | | WGWSNet [2] | 38.94 | 0.98 |
105
+ | | TransWeather [3] | 33.64 | 0.93 |
106
+ | | **TransWeather (Ours)** | **39.78** | **0.98** |
107
+ | RealSnow | Chen et al. [1] | 29.37 | 0.88 |
108
+ | | WGWSNet [2] | 29.46 | 0.85 |
109
+ | | TransWeather [3] | 29.16 | 0.82 |
110
+ | | **TransWeather (Ours)** | **29.72** | **0.91** |
111
+ | REVIDE | Chen et al. [1] | 20.10 | 0.85 |
112
+ | | WGWSNet [2] | 20.44 | 0.87 |
113
+ | | TransWeather [3] | 17.33 | 0.82 |
114
+ | | **TransWeather (Ours)** | **20.38** | **0.88** |
115
+ | Average | Chen et al. [1] | 28.93 | 0.90 |
116
+ | | WGWSNet [2] | 29.61 | 0.90 |
117
+ | | TransWeather [3] | 26.71 | 0.86 |
118
+ | | **TransWeather (Ours)** | **29.96** | **0.92** |
119
+
120
+ ## Citation
121
+ If our project helps your research or work, please cite our paper or star this repo. Thank you!
122
+ ```
123
+ @inproceedings{wu2025debiased,
124
+ title={Debiased All-in-one Image Restoration with Task Uncertainty Regularization},
125
+ author={Gang Wu, Junjun Jiang, Yijun Wang, Kui Jiang, and Xianming Liu},
126
+ booktitle={Proceedings of the AAAI Conference on Artificial Intelligence},
127
+ year={2025}
128
+ }
129
+ ```
130
+
131
+ ## Acknowledgement
132
+
133
+ This project is based on [MioIR](https://github.com/Xiangtaokong/MiOIR/tree/main/basicsr), thanks for their nice sharing.
134
+
Transweather_TUR_5task.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4e0ab0cc0ca45a48154e3a312cbdbdcb13e0c617af8299a8701e1af7493617d5
3
+ size 151154573
Transweather_TUR_AllWeather.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c0de73351367080037c540d2a62878482b4279313a20766428470ddf1b5341d0
3
+ size 151154924
Transweather_TUR_real.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3b26af928cf99c352572b6b7370b760283159023573b9b9f4a0271b0566f596c
3
+ size 151152754
Uformer_TUR_7task.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e6c0be5e7fb046a02fbe476cb6cd7b6d25b8f24cf8e7ca47c5ecc3fc9aa70ff0
3
+ size 205386882