File size: 11,014 Bytes
86285ed
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
dec55d5
86285ed
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
# CD-DCU-Net

PyTorch code and checkpoints for the CD-DCU-Net experiments. The repository contains two parts:

- CD-DCU-Net with Concrete Dropout for uncertainty estimation;
- CD-DCU-Net and four backbone comparison networks without Concrete Dropout.

## Contents

The public repository is arranged as follows:

```text
.
|-- CD-DCU-Net(Incorporate CD-module)/
|   |-- CD-DCU-Net.py
|   |-- dataloader.py
|   |-- train_with_α.py
|   |-- train_without_α.py
|   |-- predict.py
|   |-- porposed.pth
|   `-- proposed(without_α).pth
|
|-- dataset_upload/
|   |-- input/
|   |   |-- 1/
|   |   |-- 2/
|   |   `-- ...
|   `-- output/
|       |-- 1/
|       |-- 2/
|       `-- ...
|
`-- Different networks compare(without CD-model)/
    |-- dataloader.py
    |-- train.py
    |-- predict.py
    |-- CD-DCU-Net/
    |   |-- CD-DCU-Net.py
    |   |-- proposed_initial.pth
    |   `-- kaiming_initial.pth
    |-- DFPP-Net/
    |   |-- DFPP-Net.py
    |   `-- DFPP-Net.pth
    |-- DR-U-Net/
    |   |-- DR-U-Net.py
    |   `-- DR-U-Net.pth
    |-- ISP-Net/
    |   |-- ISP-Net.py
    |   `-- ISP-Net.pth
    `-- TS-U-Net/
        |-- TS-U-Net.py
        `-- TS-U-Net.pth
```

## Concrete Dropout Experiments

`CD-DCU-Net(Incorporate CD-module)/CD-DCU-Net.py` adds Concrete Dropout after each encoder and decoder stage and at the bottleneck. Its output is

```python
{
    "M_mean": M_mean,
    "M_var": M_var,
    "D_mean": D_mean,
    "D_var": D_var,
}
```

The two training scripts differ in whether the temperature constant is applied to the log-variance term of the NLL loss:

```text
train_with_α.py
    10e-4 * log(var + eps) + (target - mean)^2 / (var + eps)

train_without_α.py
    log(var + eps) + (target - mean)^2 / (var + eps)
```

The temperature constant is `10e-4` and is applied to `log(var + eps)`. It is not the `BNNLoss(alpha=...)` argument, which is present in both scripts and scales the full prediction loss.

`10e-4` is the literal used by the code and evaluates to `1e-3` in Python. It is left unchanged in this repository.

The corresponding checkpoints are:

| Training setting | Checkpoint |
|---|---|
| With the temperature constant in the log-variance term | `porposed.pth` |
| Without the temperature constant | `proposed(without_α).pth` |

The spelling of `porposed.pth` matches the provided file.

## Backbone Comparisons

The comparison directory uses one shared data loader, training script, and prediction script. The available models are:

| Model | Model file | Checkpoint |
|---|---|---|
| CD-DCU-Net, proposed initialization | `CD-DCU-Net/CD-DCU-Net.py` | `CD-DCU-Net/proposed_initial.pth` |
| CD-DCU-Net, Kaiming initialization | `CD-DCU-Net/CD-DCU-Net.py` | `CD-DCU-Net/kaiming_initial.pth` |
| DFPP-Net | `DFPP-Net/DFPP-Net.py` | `DFPP-Net/DFPP-Net.pth` |
| DR-U-Net | `DR-U-Net/DR-U-Net.py` | `DR-U-Net/DR-U-Net.pth` |
| ISP-Net | `ISP-Net/ISP-Net.py` | `ISP-Net/ISP-Net.pth` |
| TS-U-Net | `TS-U-Net/TS-U-Net.py` | `TS-U-Net/TS-U-Net.pth` |

CD-DCU-Net is the proposed backbone. `kaiming_initial.pth` is the Kaiming-initialized result. Checkpoints without an explicit Kaiming label use the proposed initialization setting.

All comparison models return the same interface:

```python
{"M_mean": M_mean, "D_mean": D_mean}
```

The shared `train.py` and `predict.py` are currently configured for DFPP-Net. See [Selecting a model](#selecting-a-model) before running another backbone.

## Environment

Required packages:

```text
torch
torchvision
numpy
scipy
pillow
matplotlib
tqdm
opencv-python
torchsummary
```

Install PyTorch for the required CUDA version, then install the remaining packages:

```bash
pip install numpy scipy pillow matplotlib tqdm opencv-python torchsummary
```

Multi-GPU training uses the NCCL backend.

## Data Format

In the public repository layout, both experiment directories use one shared `dataset/` folder at the repository root. The dataset is not duplicated inside either experiment directory. `input/` contains the three input images and `output/` contains the corresponding labels. Sample folder names are numeric.

```text
dataset/
|-- input/
|   |-- 1/
|   |   |-- image1.png
|   |   |-- image2.png
|   |   `-- image3.png
|   |-- 2/
|   |   |-- image1.png
|   |   |-- image2.png
|   |   `-- image3.png
|   `-- ...
`-- output/
    |-- 1/
    |   `-- label.mat
    |-- 2/
    |   `-- label.mat
    `-- ...
```

Each `label.mat` must contain two arrays named `M` and `D`. For every sample:

- the three PNG files must be single-channel images of the same size;
- `M` and `D` must match the image height and width;
- the numeric folder name must be the same under `input/` and `output/`.

The data loader divides input images and labels by `1000.0`. It applies synchronized geometric augmentation only to the training split.

### Train/validation split

The training and validation sets are generated from the sample folders rather than stored in separate directories. All current training scripts use an 80/20 split:

```python
train_ratio = 0.8
seed = 42
```

The split is produced in the following order:

1. Read and sort the folder names under `dataset/input/` with Python `sorted()`.
2. Create one index for each sorted folder name.
3. Shuffle the indices with `np.random.RandomState(seed)`.
4. Use the first `int(0.8 * number_of_samples)` indices for training.
5. Use all remaining indices, approximately 20% of the samples, for validation.

The following script prints the exact sample folders assigned to each set:

```python
import os
import numpy as np

data_dir = "dataset"
train_ratio = 0.8
seed = 42

samples = sorted(os.listdir(os.path.join(data_dir, "input")))
indices = list(range(len(samples)))

rng = np.random.RandomState(seed)
rng.shuffle(indices)

train_size = int(train_ratio * len(samples))
train_samples = [samples[i] for i in indices[:train_size]]
val_samples = [samples[i] for i in indices[train_size:]]

print("Training samples:", train_samples)
print("Validation samples:", val_samples)
```

Run this script from the repository root. It reproduces the membership generated by `create_data_loaders()`. The distributed sampler may change the order in which training samples are read during each epoch, but it does not change which samples belong to the training or validation set.

Folder names are sorted as strings because the data loader uses `sorted(os.listdir(...))`. For example, non-padded names may be ordered as `1`, `10`, `11`, `2`. The script above deliberately uses the same rule so that its result matches the training code exactly.

## Selecting a Model

The architecture filenames contain hyphens, which cannot be used in a normal Python import. Copy or rename the selected file to an import-safe name before running the scripts.

### Concrete Dropout model

Inside `CD-DCU-Net(Incorporate CD-module)/`, copy or rename `CD-DCU-Net.py` to `cd_dcu_net.py`. Then use the same import in both training scripts and in `predict.py`:

```python
from cd_dcu_net import UNet
```

The constructor is:

```python
UNet(n_channels=3, bilinear=False)
```

This replaces `from mcunet1 import UNet` in the training scripts and `from proposed_uncertainty import UNet` in the prediction script.

### Comparison models

Copy the selected model file into `Different networks compare(without CD-model)/`, give it the module name shown below, and update both `train.py` and `predict.py`.

| Model | Module and import | Constructor |
|---|---|---|
| CD-DCU-Net | `from cd_dcu_net import UNet` | `UNet(n_channels=3, bilinear=False)` |
| DFPP-Net | `from dfppnet import DFPPNet3FrameMD` | `DFPPNet3FrameMD(n_feat=4, stage0_align=True, stage1_align=False, stage2_align=False, return_aux=False)` |
| DR-U-Net | `from dr_unet import UNet` | `UNet(n_channels=3, out_channels=2, bilinear=False)` |
| ISP-Net | `from isp_net import UNet` | `UNet(n_channels=3, bilinear=False)` |
| TS-U-Net | `from ts_unet import UNet` | `UNet(n_channels=3, bilinear=False)` |

Change the constructor in two places:

- the main block of `train.py`;
- `Predictor._load_model()` in `predict.py`.

The architecture and constructor must match the checkpoint.

## Training

Edit the configuration block at the bottom of the selected training script:

```python
data_dir = "../dataset"
batch_size = 4
train_ratio = 0.8
crop_size = (1360, 672)
num_workers = 16
num_epochs = 300
learning_rate = 0.0001
pretrained_path = "path/to/initialization_checkpoint.pth"
```

Also set `save_dir` in the `Trainer(...)` call. `crop_size` is ordered as `(height, width)` and must not exceed the image size.

Training is initialized from a checkpoint:

```python
checkpoint = torch.load(pretrained_path, map_location="cpu")
model.load_state_dict(checkpoint["model_state_dict"], strict=True)
```

The checkpoint must contain `model_state_dict` and must match the selected architecture.

Run one of the Concrete Dropout experiments from its directory:

```bash
python "train_with_α.py"
python "train_without_α.py"
```

Run a backbone comparison after updating the shared model import and constructor:

```bash
cd "Different networks compare(without CD-model)"
python train.py
```

For multi-GPU training:

```bash
torchrun --standalone --nproc_per_node=2 "train_with_α.py"
```

The scripts save epoch checkpoints, `best_model.pth`, training curves, and the five most recent rolling checkpoints. Concrete Dropout training also saves the learned dropout-rate curves.

## Prediction

Set the checkpoint, crop size, device, and three image paths at the bottom of `predict.py`:

```python
predictor = Predictor(
    model_path="path/to/model_checkpoint.pth",
    crop_size=(1360, 672),
    device="cuda",
)

img1_path = "../dataset/input/1/image1.png"
img2_path = "../dataset/input/1/image2.png"
img3_path = "../dataset/input/1/image3.png"
```

Use the same crop size as training. Set `device="cpu"` for CPU inference.

Run:

```bash
python predict.py
```

### Single prediction

```python
result = predictor.predict_single(img1_path, img2_path, img3_path)
predictor.save_prediction(result, "predictions/single_result.mat", mode="single")
```

Single prediction returns `M` and `D`.

### Uncertainty prediction

Monte Carlo prediction is used for the Concrete Dropout model:

```python
results = predictor.predict_with_uncertainty(
    img1_path,
    img2_path,
    img3_path,
    n_samples=50,
)
predictor.save_prediction(results, "predictions/mc_result.mat", mode="mc")
```

During Monte Carlo prediction, `predict.py` enables Concrete Dropout, runs `n_samples` stochastic forward passes, and restores evaluation mode afterward. The output contains mean predictions and model, data, and total uncertainty maps for both `M` and `D`.

The comparison backbones return only `M_mean` and `D_mean`; use `predict_single()` for those checkpoints.

## Hugging Face Upload

The checkpoint files are large. Track them with Git LFS before pushing the repository:

```bash
git lfs install
git lfs track "*.pth"
git add .gitattributes
git add .
git commit -m "Add code and checkpoints"
```