CD-DCU-Net / README.md
Li-zhangsheng's picture
Update README.md
dec55d5 verified
|
Raw History Blame Contribute Delete
11 kB
# CD-DCU-Net
PyTorch code and checkpoints for the CD-DCU-Net experiments. The repository contains two parts:
- CD-DCU-Net with Concrete Dropout for uncertainty estimation;
- CD-DCU-Net and four backbone comparison networks without Concrete Dropout.
## Contents
The public repository is arranged as follows:
```text
.
|-- CD-DCU-Net(Incorporate CD-module)/
| |-- CD-DCU-Net.py
| |-- dataloader.py
| |-- train_with_α.py
| |-- train_without_α.py
| |-- predict.py
| |-- porposed.pth
| `-- proposed(without_α).pth
|
|-- dataset_upload/
| |-- input/
| | |-- 1/
| | |-- 2/
| | `-- ...
| `-- output/
| |-- 1/
| |-- 2/
| `-- ...
|
`-- Different networks compare(without CD-model)/
|-- dataloader.py
|-- train.py
|-- predict.py
|-- CD-DCU-Net/
| |-- CD-DCU-Net.py
| |-- proposed_initial.pth
| `-- kaiming_initial.pth
|-- DFPP-Net/
| |-- DFPP-Net.py
| `-- DFPP-Net.pth
|-- DR-U-Net/
| |-- DR-U-Net.py
| `-- DR-U-Net.pth
|-- ISP-Net/
| |-- ISP-Net.py
| `-- ISP-Net.pth
`-- TS-U-Net/
|-- TS-U-Net.py
`-- TS-U-Net.pth
```
## Concrete Dropout Experiments
`CD-DCU-Net(Incorporate CD-module)/CD-DCU-Net.py` adds Concrete Dropout after each encoder and decoder stage and at the bottleneck. Its output is
```python
{
"M_mean": M_mean,
"M_var": M_var,
"D_mean": D_mean,
"D_var": D_var,
}
```
The two training scripts differ in whether the temperature constant is applied to the log-variance term of the NLL loss:
```text
train_with_α.py
10e-4 * log(var + eps) + (target - mean)^2 / (var + eps)
train_without_α.py
log(var + eps) + (target - mean)^2 / (var + eps)
```
The temperature constant is `10e-4` and is applied to `log(var + eps)`. It is not the `BNNLoss(alpha=...)` argument, which is present in both scripts and scales the full prediction loss.
`10e-4` is the literal used by the code and evaluates to `1e-3` in Python. It is left unchanged in this repository.
The corresponding checkpoints are:
| Training setting | Checkpoint |
|---|---|
| With the temperature constant in the log-variance term | `porposed.pth` |
| Without the temperature constant | `proposed(without_α).pth` |
The spelling of `porposed.pth` matches the provided file.
## Backbone Comparisons
The comparison directory uses one shared data loader, training script, and prediction script. The available models are:
| Model | Model file | Checkpoint |
|---|---|---|
| CD-DCU-Net, proposed initialization | `CD-DCU-Net/CD-DCU-Net.py` | `CD-DCU-Net/proposed_initial.pth` |
| CD-DCU-Net, Kaiming initialization | `CD-DCU-Net/CD-DCU-Net.py` | `CD-DCU-Net/kaiming_initial.pth` |
| DFPP-Net | `DFPP-Net/DFPP-Net.py` | `DFPP-Net/DFPP-Net.pth` |
| DR-U-Net | `DR-U-Net/DR-U-Net.py` | `DR-U-Net/DR-U-Net.pth` |
| ISP-Net | `ISP-Net/ISP-Net.py` | `ISP-Net/ISP-Net.pth` |
| TS-U-Net | `TS-U-Net/TS-U-Net.py` | `TS-U-Net/TS-U-Net.pth` |
CD-DCU-Net is the proposed backbone. `kaiming_initial.pth` is the Kaiming-initialized result. Checkpoints without an explicit Kaiming label use the proposed initialization setting.
All comparison models return the same interface:
```python
{"M_mean": M_mean, "D_mean": D_mean}
```
The shared `train.py` and `predict.py` are currently configured for DFPP-Net. See [Selecting a model](#selecting-a-model) before running another backbone.
## Environment
Required packages:
```text
torch
torchvision
numpy
scipy
pillow
matplotlib
tqdm
opencv-python
torchsummary
```
Install PyTorch for the required CUDA version, then install the remaining packages:
```bash
pip install numpy scipy pillow matplotlib tqdm opencv-python torchsummary
```
Multi-GPU training uses the NCCL backend.
## Data Format
In the public repository layout, both experiment directories use one shared `dataset/` folder at the repository root. The dataset is not duplicated inside either experiment directory. `input/` contains the three input images and `output/` contains the corresponding labels. Sample folder names are numeric.
```text
dataset/
|-- input/
| |-- 1/
| | |-- image1.png
| | |-- image2.png
| | `-- image3.png
| |-- 2/
| | |-- image1.png
| | |-- image2.png
| | `-- image3.png
| `-- ...
`-- output/
|-- 1/
| `-- label.mat
|-- 2/
| `-- label.mat
`-- ...
```
Each `label.mat` must contain two arrays named `M` and `D`. For every sample:
- the three PNG files must be single-channel images of the same size;
- `M` and `D` must match the image height and width;
- the numeric folder name must be the same under `input/` and `output/`.
The data loader divides input images and labels by `1000.0`. It applies synchronized geometric augmentation only to the training split.
### Train/validation split
The training and validation sets are generated from the sample folders rather than stored in separate directories. All current training scripts use an 80/20 split:
```python
train_ratio = 0.8
seed = 42
```
The split is produced in the following order:
1. Read and sort the folder names under `dataset/input/` with Python `sorted()`.
2. Create one index for each sorted folder name.
3. Shuffle the indices with `np.random.RandomState(seed)`.
4. Use the first `int(0.8 * number_of_samples)` indices for training.
5. Use all remaining indices, approximately 20% of the samples, for validation.
The following script prints the exact sample folders assigned to each set:
```python
import os
import numpy as np
data_dir = "dataset"
train_ratio = 0.8
seed = 42
samples = sorted(os.listdir(os.path.join(data_dir, "input")))
indices = list(range(len(samples)))
rng = np.random.RandomState(seed)
rng.shuffle(indices)
train_size = int(train_ratio * len(samples))
train_samples = [samples[i] for i in indices[:train_size]]
val_samples = [samples[i] for i in indices[train_size:]]
print("Training samples:", train_samples)
print("Validation samples:", val_samples)
```
Run this script from the repository root. It reproduces the membership generated by `create_data_loaders()`. The distributed sampler may change the order in which training samples are read during each epoch, but it does not change which samples belong to the training or validation set.
Folder names are sorted as strings because the data loader uses `sorted(os.listdir(...))`. For example, non-padded names may be ordered as `1`, `10`, `11`, `2`. The script above deliberately uses the same rule so that its result matches the training code exactly.
## Selecting a Model
The architecture filenames contain hyphens, which cannot be used in a normal Python import. Copy or rename the selected file to an import-safe name before running the scripts.
### Concrete Dropout model
Inside `CD-DCU-Net(Incorporate CD-module)/`, copy or rename `CD-DCU-Net.py` to `cd_dcu_net.py`. Then use the same import in both training scripts and in `predict.py`:
```python
from cd_dcu_net import UNet
```
The constructor is:
```python
UNet(n_channels=3, bilinear=False)
```
This replaces `from mcunet1 import UNet` in the training scripts and `from proposed_uncertainty import UNet` in the prediction script.
### Comparison models
Copy the selected model file into `Different networks compare(without CD-model)/`, give it the module name shown below, and update both `train.py` and `predict.py`.
| Model | Module and import | Constructor |
|---|---|---|
| CD-DCU-Net | `from cd_dcu_net import UNet` | `UNet(n_channels=3, bilinear=False)` |
| DFPP-Net | `from dfppnet import DFPPNet3FrameMD` | `DFPPNet3FrameMD(n_feat=4, stage0_align=True, stage1_align=False, stage2_align=False, return_aux=False)` |
| DR-U-Net | `from dr_unet import UNet` | `UNet(n_channels=3, out_channels=2, bilinear=False)` |
| ISP-Net | `from isp_net import UNet` | `UNet(n_channels=3, bilinear=False)` |
| TS-U-Net | `from ts_unet import UNet` | `UNet(n_channels=3, bilinear=False)` |
Change the constructor in two places:
- the main block of `train.py`;
- `Predictor._load_model()` in `predict.py`.
The architecture and constructor must match the checkpoint.
## Training
Edit the configuration block at the bottom of the selected training script:
```python
data_dir = "../dataset"
batch_size = 4
train_ratio = 0.8
crop_size = (1360, 672)
num_workers = 16
num_epochs = 300
learning_rate = 0.0001
pretrained_path = "path/to/initialization_checkpoint.pth"
```
Also set `save_dir` in the `Trainer(...)` call. `crop_size` is ordered as `(height, width)` and must not exceed the image size.
Training is initialized from a checkpoint:
```python
checkpoint = torch.load(pretrained_path, map_location="cpu")
model.load_state_dict(checkpoint["model_state_dict"], strict=True)
```
The checkpoint must contain `model_state_dict` and must match the selected architecture.
Run one of the Concrete Dropout experiments from its directory:
```bash
python "train_with_α.py"
python "train_without_α.py"
```
Run a backbone comparison after updating the shared model import and constructor:
```bash
cd "Different networks compare(without CD-model)"
python train.py
```
For multi-GPU training:
```bash
torchrun --standalone --nproc_per_node=2 "train_with_α.py"
```
The scripts save epoch checkpoints, `best_model.pth`, training curves, and the five most recent rolling checkpoints. Concrete Dropout training also saves the learned dropout-rate curves.
## Prediction
Set the checkpoint, crop size, device, and three image paths at the bottom of `predict.py`:
```python
predictor = Predictor(
model_path="path/to/model_checkpoint.pth",
crop_size=(1360, 672),
device="cuda",
)
img1_path = "../dataset/input/1/image1.png"
img2_path = "../dataset/input/1/image2.png"
img3_path = "../dataset/input/1/image3.png"
```
Use the same crop size as training. Set `device="cpu"` for CPU inference.
Run:
```bash
python predict.py
```
### Single prediction
```python
result = predictor.predict_single(img1_path, img2_path, img3_path)
predictor.save_prediction(result, "predictions/single_result.mat", mode="single")
```
Single prediction returns `M` and `D`.
### Uncertainty prediction
Monte Carlo prediction is used for the Concrete Dropout model:
```python
results = predictor.predict_with_uncertainty(
img1_path,
img2_path,
img3_path,
n_samples=50,
)
predictor.save_prediction(results, "predictions/mc_result.mat", mode="mc")
```
During Monte Carlo prediction, `predict.py` enables Concrete Dropout, runs `n_samples` stochastic forward passes, and restores evaluation mode afterward. The output contains mean predictions and model, data, and total uncertainty maps for both `M` and `D`.
The comparison backbones return only `M_mean` and `D_mean`; use `predict_single()` for those checkpoints.
## Hugging Face Upload
The checkpoint files are large. Track them with Git LFS before pushing the repository:
```bash
git lfs install
git lfs track "*.pth"
git add .gitattributes
git add .
git commit -m "Add code and checkpoints"
```