Replace llm Space with DIPAug project hub
Browse filesThis view is limited to 50 files because it contains too many changes. See raw diff
- README.md +48 -41
- app.py +44 -77
- configs/baselines/paper1_baselines.yaml +15 -0
- configs/baselines/paper2_baselines.yaml +13 -0
- configs/phase1/e1_baseline.yaml +19 -0
- configs/phase1/e2_geometric.yaml +20 -0
- configs/phase1/e3_dipaug_fixed.yaml +20 -0
- configs/phase1/e4_dipaug_aas.yaml +20 -0
- configs/phase1/e5_dual_branch.yaml +20 -0
- configs/phase1/e6_full.yaml +23 -0
- configs/phase2/s1_baseline.yaml +19 -0
- configs/phase2/s2_segmentation.yaml +20 -0
- configs/phase2/s3_dgsm.yaml +20 -0
- configs/phase2/s4_simclr.yaml +20 -0
- configs/phase2/s5_full.yaml +21 -0
- dipauglib/__init__.py +1 -0
- dipauglib/sampling/__init__.py +5 -0
- dipauglib/sampling/class_imbalance.py +30 -0
- dipauglib/schedulers/__init__.py +5 -0
- dipauglib/schedulers/adaptive.py +27 -0
- dipauglib/transforms/__init__.py +23 -0
- dipauglib/transforms/physics.py +239 -0
- dipauglib/transforms/pipeline.py +38 -0
- dipauglib/utils/__init__.py +1 -0
- dipauglib/utils/dataset.py +53 -0
- dipauglib/utils/io.py +14 -0
- dipauglib/utils/repro.py +19 -0
- dipaugnet/__init__.py +1 -0
- dipaugnet/evaluation/__init__.py +1 -0
- dipaugnet/evaluation/metrics.py +27 -0
- dipaugnet/models/__init__.py +5 -0
- dipaugnet/models/dipaugnet.py +84 -0
- dipaugnet/training/__init__.py +1 -0
- dipaugnet/training/engine.py +65 -0
- dipaugnet/training/losses.py +33 -0
- dipaugsevernet/__init__.py +1 -0
- dipaugsevernet/evaluation/__init__.py +1 -0
- dipaugsevernet/evaluation/metrics.py +33 -0
- dipaugsevernet/models/__init__.py +6 -0
- dipaugsevernet/models/dgsm.py +46 -0
- dipaugsevernet/models/dipaugsevernet.py +96 -0
- dipaugsevernet/training/__init__.py +1 -0
- dipaugsevernet/training/engine.py +15 -0
- dipaugsevernet/training/losses.py +39 -0
- figures/README.md +9 -0
- notebooks/README.md +8 -0
- requirements.txt +22 -2
- results/README.md +8 -0
- scripts/evaluate_phase1.py +5 -0
- scripts/evaluate_phase2.py +5 -0
README.md
CHANGED
|
@@ -1,63 +1,70 @@
|
|
| 1 |
---
|
| 2 |
-
title:
|
| 3 |
-
|
| 4 |
-
|
| 5 |
-
colorTo: green
|
| 6 |
sdk: gradio
|
| 7 |
-
sdk_version: 5.23.0
|
| 8 |
app_file: app.py
|
| 9 |
pinned: false
|
|
|
|
| 10 |
---
|
| 11 |
|
| 12 |
-
#
|
| 13 |
|
| 14 |
-
|
| 15 |
|
| 16 |
-
-
|
| 17 |
-
-
|
| 18 |
-
- Pure PyTorch transformer
|
| 19 |
-
- Gradio UI for Hugging Face Spaces
|
| 20 |
|
| 21 |
-
|
| 22 |
|
| 23 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 24 |
|
| 25 |
-
|
| 26 |
|
| 27 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 28 |
|
| 29 |
-
|
| 30 |
-
|
| 31 |
-
|
| 32 |
-
|
| 33 |
-
|
| 34 |
-
|
| 35 |
-
|
| 36 |
-
|
| 37 |
-
|
| 38 |
-
|
| 39 |
-
|
| 40 |
-
|
| 41 |
-
|
| 42 |
-
└── trainer.py
|
| 43 |
-
```
|
| 44 |
|
| 45 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 46 |
|
| 47 |
```bash
|
| 48 |
pip install -r requirements.txt
|
|
|
|
|
|
|
|
|
|
| 49 |
python app.py
|
| 50 |
```
|
| 51 |
|
| 52 |
-
## Deploy on Hugging Face Spaces
|
| 53 |
-
|
| 54 |
-
1. Create a new Space.
|
| 55 |
-
2. Choose `Gradio`.
|
| 56 |
-
3. Upload these files.
|
| 57 |
-
4. Space will install `requirements.txt`.
|
| 58 |
-
5. The app will train a small model checkpoint on first use.
|
| 59 |
-
|
| 60 |
## Notes
|
| 61 |
|
| 62 |
-
-
|
| 63 |
-
-
|
|
|
|
|
|
| 1 |
---
|
| 2 |
+
title: DIPAug Project Hub
|
| 3 |
+
colorFrom: green
|
| 4 |
+
colorTo: blue
|
|
|
|
| 5 |
sdk: gradio
|
|
|
|
| 6 |
app_file: app.py
|
| 7 |
pinned: false
|
| 8 |
+
license: mit
|
| 9 |
---
|
| 10 |
|
| 11 |
+
# Realistic Digital Image Processing-Driven Data Augmentation for Robust Wheat Leaf Disease Classification and Severity Scoring in Field Conditions
|
| 12 |
|
| 13 |
+
Short titles:
|
| 14 |
|
| 15 |
+
- `DIPAug-Net` for wheat leaf disease classification
|
| 16 |
+
- `DIPAug-SeverNet` for lesion segmentation and severity scoring
|
|
|
|
|
|
|
| 17 |
|
| 18 |
+
This repository is a Hugging Face-ready project scaffold built from the provided specification. It focuses on:
|
| 19 |
|
| 20 |
+
- shared physics-aware DIP augmentation modules
|
| 21 |
+
- phase 1 classification model and evaluation skeletons
|
| 22 |
+
- phase 2 saliency, segmentation, and severity model skeletons
|
| 23 |
+
- config-driven experiments for `E1-E6` and `S1-S5`
|
| 24 |
+
- pytest coverage for the augmentation stack and shared utilities
|
| 25 |
|
| 26 |
+
## Repository Layout
|
| 27 |
|
| 28 |
+
- `dipauglib/`: shared augmentation, scheduling, sampling, and utility code
|
| 29 |
+
- `dipaugnet/`: phase 1 model, loss, training, and evaluation skeleton
|
| 30 |
+
- `dipaugsevernet/`: phase 2 saliency, multi-task model, loss, and training skeleton
|
| 31 |
+
- `configs/`: YAML experiment configurations
|
| 32 |
+
- `tests/`: unit tests for transforms and shared logic
|
| 33 |
+
- `scripts/`: entrypoints for dataset prep, training, and evaluation
|
| 34 |
+
- `figures/`, `results/`, `notebooks/`: output and analysis directories
|
| 35 |
|
| 36 |
+
## What Is Implemented In This Scaffold
|
| 37 |
+
|
| 38 |
+
- stratified split utilities with fixed seed support
|
| 39 |
+
- ImageNet preprocessing configuration
|
| 40 |
+
- all 8 DIP augmentation classes as Albumentations `DualTransform` implementations
|
| 41 |
+
- adaptive augmentation scheduler
|
| 42 |
+
- class-imbalance-aware sampling helpers
|
| 43 |
+
- DIPAug-Net architecture skeleton with CNN + Transformer fusion
|
| 44 |
+
- DGSM and DIPAug-SeverNet architecture skeleton
|
| 45 |
+
- YAML configurations for the requested ablations
|
| 46 |
+
- a lightweight Gradio project dashboard for Hugging Face Spaces
|
| 47 |
+
|
| 48 |
+
## What Still Needs Real Training Resources
|
|
|
|
|
|
|
| 49 |
|
| 50 |
+
- dataset download and checksum validation
|
| 51 |
+
- baseline reproduction runs
|
| 52 |
+
- full training of `E1-E6` and `S1-S5`
|
| 53 |
+
- CEDB, Grad-CAM++, SAM pseudo-mask generation, and full results package
|
| 54 |
+
- publication-quality result figures and final report population
|
| 55 |
+
|
| 56 |
+
## Quick Start
|
| 57 |
|
| 58 |
```bash
|
| 59 |
pip install -r requirements.txt
|
| 60 |
+
python scripts/prepare_dataset.py --help
|
| 61 |
+
python scripts/train_phase1.py --config configs/phase1/e6_full.yaml
|
| 62 |
+
python scripts/train_phase2.py --config configs/phase2/s5_full.yaml
|
| 63 |
python app.py
|
| 64 |
```
|
| 65 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 66 |
## Notes
|
| 67 |
|
| 68 |
+
- This scaffold is intentionally config-driven and dataset-path agnostic.
|
| 69 |
+
- The training code is structured for CUDA-enabled machines, but this repository itself is safe to inspect or host on Hugging Face without a GPU.
|
| 70 |
+
- The Space UI is a project dashboard, not a full training runner.
|
app.py
CHANGED
|
@@ -1,100 +1,67 @@
|
|
| 1 |
from pathlib import Path
|
| 2 |
|
| 3 |
import gradio as gr
|
|
|
|
| 4 |
|
| 5 |
-
from mini_llm.service import LocalLLMService
|
| 6 |
|
|
|
|
| 7 |
|
| 8 |
-
BASE_DIR = Path(__file__).resolve().parent
|
| 9 |
-
service = LocalLLMService(base_dir=BASE_DIR)
|
| 10 |
|
| 11 |
-
|
| 12 |
-
|
| 13 |
-
text, status = service.generate(
|
| 14 |
-
prompt=prompt,
|
| 15 |
-
max_new_tokens=max_new_tokens,
|
| 16 |
-
temperature=temperature,
|
| 17 |
-
top_k=top_k,
|
| 18 |
-
)
|
| 19 |
-
return text, status
|
| 20 |
|
| 21 |
|
| 22 |
-
def
|
| 23 |
-
|
| 24 |
-
|
|
|
|
|
|
|
|
|
|
| 25 |
|
| 26 |
|
| 27 |
-
with gr.Blocks(title="
|
| 28 |
gr.Markdown(
|
| 29 |
"""
|
| 30 |
-
#
|
| 31 |
-
|
|
|
|
|
|
|
|
|
|
| 32 |
|
| 33 |
-
|
| 34 |
-
|
| 35 |
-
-
|
|
|
|
|
|
|
|
|
|
| 36 |
"""
|
| 37 |
)
|
| 38 |
|
| 39 |
with gr.Row():
|
| 40 |
-
with gr.Column():
|
| 41 |
-
|
| 42 |
-
|
| 43 |
-
|
| 44 |
-
|
| 45 |
-
|
| 46 |
-
|
| 47 |
-
|
| 48 |
-
|
| 49 |
-
|
| 50 |
-
|
| 51 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 52 |
)
|
| 53 |
-
temperature = gr.Slider(
|
| 54 |
-
label="Temperature",
|
| 55 |
-
minimum=0.1,
|
| 56 |
-
maximum=1.5,
|
| 57 |
-
value=0.9,
|
| 58 |
-
step=0.1,
|
| 59 |
-
)
|
| 60 |
-
top_k = gr.Slider(
|
| 61 |
-
label="Top-k",
|
| 62 |
-
minimum=1,
|
| 63 |
-
maximum=50,
|
| 64 |
-
value=20,
|
| 65 |
-
step=1,
|
| 66 |
-
)
|
| 67 |
-
generate_button = gr.Button("Generate Text", variant="primary")
|
| 68 |
|
| 69 |
-
|
| 70 |
-
output = gr.Textbox(label="Generated Text", lines=16)
|
| 71 |
-
status = gr.Textbox(label="Status", value=service.describe_model())
|
| 72 |
-
|
| 73 |
-
gr.Markdown("## Train or Refresh Model")
|
| 74 |
-
with gr.Row():
|
| 75 |
-
epochs = gr.Slider(label="Epochs", minimum=5, maximum=80, value=25, step=5)
|
| 76 |
-
learning_rate = gr.Slider(
|
| 77 |
-
label="Learning rate",
|
| 78 |
-
minimum=0.0001,
|
| 79 |
-
maximum=0.01,
|
| 80 |
-
value=0.003,
|
| 81 |
-
step=0.0001,
|
| 82 |
-
)
|
| 83 |
-
train_button = gr.Button("Train Model")
|
| 84 |
-
train_message = gr.Textbox(label="Training Result")
|
| 85 |
-
model_info = gr.Textbox(label="Model Info", value=service.describe_model())
|
| 86 |
-
|
| 87 |
-
generate_button.click(
|
| 88 |
-
fn=run_generation,
|
| 89 |
-
inputs=[prompt, max_new_tokens, temperature, top_k],
|
| 90 |
-
outputs=[output, status],
|
| 91 |
-
)
|
| 92 |
-
|
| 93 |
-
train_button.click(
|
| 94 |
-
fn=run_training,
|
| 95 |
-
inputs=[epochs, learning_rate],
|
| 96 |
-
outputs=[train_message, model_info],
|
| 97 |
-
)
|
| 98 |
|
| 99 |
|
| 100 |
if __name__ == "__main__":
|
|
|
|
| 1 |
from pathlib import Path
|
| 2 |
|
| 3 |
import gradio as gr
|
| 4 |
+
import yaml
|
| 5 |
|
|
|
|
| 6 |
|
| 7 |
+
PROJECT_ROOT = Path(__file__).resolve().parent
|
| 8 |
|
|
|
|
|
|
|
| 9 |
|
| 10 |
+
def list_configs() -> list[str]:
|
| 11 |
+
return sorted(str(path.relative_to(PROJECT_ROOT)).replace("\\", "/") for path in PROJECT_ROOT.glob("configs/**/*.yaml"))
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 12 |
|
| 13 |
|
| 14 |
+
def show_config(config_name: str) -> str:
|
| 15 |
+
if not config_name:
|
| 16 |
+
return "Select a config."
|
| 17 |
+
path = PROJECT_ROOT / config_name
|
| 18 |
+
data = yaml.safe_load(path.read_text(encoding="utf-8"))
|
| 19 |
+
return yaml.safe_dump(data, sort_keys=False)
|
| 20 |
|
| 21 |
|
| 22 |
+
with gr.Blocks(title="DIPAug Project Hub") as demo:
|
| 23 |
gr.Markdown(
|
| 24 |
"""
|
| 25 |
+
# DIPAug Project Hub
|
| 26 |
+
|
| 27 |
+
**Project Title**
|
| 28 |
+
|
| 29 |
+
Realistic Digital Image Processing-Driven Data Augmentation for Robust Wheat Leaf Disease Classification and Severity Scoring in Field Conditions
|
| 30 |
|
| 31 |
+
**Short Titles**
|
| 32 |
+
|
| 33 |
+
- `DIPAug-Net`
|
| 34 |
+
- `DIPAug-SeverNet`
|
| 35 |
+
|
| 36 |
+
This Hugging Face app is a lightweight dashboard for the project scaffold. It helps inspect the experiment configs and repository structure before training on a proper GPU machine.
|
| 37 |
"""
|
| 38 |
)
|
| 39 |
|
| 40 |
with gr.Row():
|
| 41 |
+
with gr.Column(scale=2):
|
| 42 |
+
config_input = gr.Dropdown(label="Experiment Config", choices=list_configs(), value="configs/phase1/e6_full.yaml")
|
| 43 |
+
config_output = gr.Code(label="YAML", language="yaml", value=show_config("configs/phase1/e6_full.yaml"))
|
| 44 |
+
with gr.Column(scale=2):
|
| 45 |
+
gr.Markdown(
|
| 46 |
+
"""
|
| 47 |
+
## Included Modules
|
| 48 |
+
|
| 49 |
+
- `dipauglib.transforms`: 8 physics-aware augmentations
|
| 50 |
+
- `dipauglib.schedulers`: adaptive augmentation scheduler
|
| 51 |
+
- `dipauglib.sampling`: class-imbalance-aware sampling
|
| 52 |
+
- `dipaugnet`: phase 1 classification pipeline
|
| 53 |
+
- `dipaugsevernet`: phase 2 segmentation and severity scaffold
|
| 54 |
+
|
| 55 |
+
## Status
|
| 56 |
+
|
| 57 |
+
- repository scaffold: ready
|
| 58 |
+
- configs: ready
|
| 59 |
+
- tests: included
|
| 60 |
+
- full training runs: not executed in this dashboard
|
| 61 |
+
"""
|
| 62 |
)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 63 |
|
| 64 |
+
config_input.change(fn=show_config, inputs=[config_input], outputs=[config_output])
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 65 |
|
| 66 |
|
| 67 |
if __name__ == "__main__":
|
configs/baselines/paper1_baselines.yaml
ADDED
|
@@ -0,0 +1,15 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
baselines:
|
| 2 |
+
- name: ResNet-50 + Generic Aug
|
| 3 |
+
family: resnet50
|
| 4 |
+
- name: EfficientNet-B3 (no aug)
|
| 5 |
+
family: efficientnet_b3
|
| 6 |
+
- name: EfficientNet-B3 + Generic Aug
|
| 7 |
+
family: efficientnet_b3
|
| 8 |
+
- name: SC-ConvNeXt
|
| 9 |
+
family: convnext
|
| 10 |
+
- name: GLNet
|
| 11 |
+
family: glnet
|
| 12 |
+
- name: ViT Multi-level Contrast
|
| 13 |
+
family: vit
|
| 14 |
+
- name: CropNet
|
| 15 |
+
family: shallow_cnn
|
configs/baselines/paper2_baselines.yaml
ADDED
|
@@ -0,0 +1,13 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
baselines:
|
| 2 |
+
- name: U-Net
|
| 3 |
+
family: unet
|
| 4 |
+
- name: Attention U-Net
|
| 5 |
+
family: attention_unet
|
| 6 |
+
- name: DeepLabV3+
|
| 7 |
+
family: deeplabv3plus
|
| 8 |
+
- name: PDSNets
|
| 9 |
+
family: linknet_resnet18
|
| 10 |
+
- name: DIPAug-Net + Ordinal Head
|
| 11 |
+
family: dipaugnet_ordinal
|
| 12 |
+
- name: SegLearner
|
| 13 |
+
family: severity_baseline
|
configs/phase1/e1_baseline.yaml
ADDED
|
@@ -0,0 +1,19 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
project:
|
| 2 |
+
name: DIPAug-Net
|
| 3 |
+
phase: phase1
|
| 4 |
+
experiment_id: E1
|
| 5 |
+
dataset:
|
| 6 |
+
num_classes: 11
|
| 7 |
+
image_size: 384
|
| 8 |
+
split_seed: 42
|
| 9 |
+
augmentation:
|
| 10 |
+
use_dipaug: false
|
| 11 |
+
use_aas: false
|
| 12 |
+
use_ciaa: false
|
| 13 |
+
model:
|
| 14 |
+
use_dual_branch: false
|
| 15 |
+
training:
|
| 16 |
+
batch_size: 32
|
| 17 |
+
epochs: 100
|
| 18 |
+
optimizer: adamw
|
| 19 |
+
learning_rate: 1.0e-4
|
configs/phase1/e2_geometric.yaml
ADDED
|
@@ -0,0 +1,20 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
project:
|
| 2 |
+
name: DIPAug-Net
|
| 3 |
+
phase: phase1
|
| 4 |
+
experiment_id: E2
|
| 5 |
+
dataset:
|
| 6 |
+
num_classes: 11
|
| 7 |
+
image_size: 384
|
| 8 |
+
split_seed: 42
|
| 9 |
+
augmentation:
|
| 10 |
+
use_dipaug: false
|
| 11 |
+
geometric_only: true
|
| 12 |
+
use_aas: false
|
| 13 |
+
use_ciaa: false
|
| 14 |
+
model:
|
| 15 |
+
use_dual_branch: false
|
| 16 |
+
training:
|
| 17 |
+
batch_size: 32
|
| 18 |
+
epochs: 100
|
| 19 |
+
optimizer: adamw
|
| 20 |
+
learning_rate: 1.0e-4
|
configs/phase1/e3_dipaug_fixed.yaml
ADDED
|
@@ -0,0 +1,20 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
project:
|
| 2 |
+
name: DIPAug-Net
|
| 3 |
+
phase: phase1
|
| 4 |
+
experiment_id: E3
|
| 5 |
+
dataset:
|
| 6 |
+
num_classes: 11
|
| 7 |
+
image_size: 384
|
| 8 |
+
split_seed: 42
|
| 9 |
+
augmentation:
|
| 10 |
+
use_dipaug: true
|
| 11 |
+
intensity: 0.7
|
| 12 |
+
use_aas: false
|
| 13 |
+
use_ciaa: false
|
| 14 |
+
model:
|
| 15 |
+
use_dual_branch: false
|
| 16 |
+
training:
|
| 17 |
+
batch_size: 32
|
| 18 |
+
epochs: 100
|
| 19 |
+
optimizer: adamw
|
| 20 |
+
learning_rate: 1.0e-4
|
configs/phase1/e4_dipaug_aas.yaml
ADDED
|
@@ -0,0 +1,20 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
project:
|
| 2 |
+
name: DIPAug-Net
|
| 3 |
+
phase: phase1
|
| 4 |
+
experiment_id: E4
|
| 5 |
+
dataset:
|
| 6 |
+
num_classes: 11
|
| 7 |
+
image_size: 384
|
| 8 |
+
split_seed: 42
|
| 9 |
+
augmentation:
|
| 10 |
+
use_dipaug: true
|
| 11 |
+
intensity: 0.7
|
| 12 |
+
use_aas: true
|
| 13 |
+
use_ciaa: false
|
| 14 |
+
model:
|
| 15 |
+
use_dual_branch: false
|
| 16 |
+
training:
|
| 17 |
+
batch_size: 32
|
| 18 |
+
epochs: 100
|
| 19 |
+
optimizer: adamw
|
| 20 |
+
learning_rate: 1.0e-4
|
configs/phase1/e5_dual_branch.yaml
ADDED
|
@@ -0,0 +1,20 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
project:
|
| 2 |
+
name: DIPAug-Net
|
| 3 |
+
phase: phase1
|
| 4 |
+
experiment_id: E5
|
| 5 |
+
dataset:
|
| 6 |
+
num_classes: 11
|
| 7 |
+
image_size: 384
|
| 8 |
+
split_seed: 42
|
| 9 |
+
augmentation:
|
| 10 |
+
use_dipaug: true
|
| 11 |
+
intensity: 0.7
|
| 12 |
+
use_aas: false
|
| 13 |
+
use_ciaa: false
|
| 14 |
+
model:
|
| 15 |
+
use_dual_branch: true
|
| 16 |
+
training:
|
| 17 |
+
batch_size: 32
|
| 18 |
+
epochs: 100
|
| 19 |
+
optimizer: adamw
|
| 20 |
+
learning_rate: 1.0e-4
|
configs/phase1/e6_full.yaml
ADDED
|
@@ -0,0 +1,23 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
project:
|
| 2 |
+
name: DIPAug-Net
|
| 3 |
+
phase: phase1
|
| 4 |
+
experiment_id: E6
|
| 5 |
+
dataset:
|
| 6 |
+
num_classes: 11
|
| 7 |
+
image_size: 384
|
| 8 |
+
split_seed: 42
|
| 9 |
+
augmentation:
|
| 10 |
+
use_dipaug: true
|
| 11 |
+
intensity: 0.7
|
| 12 |
+
use_aas: true
|
| 13 |
+
use_ciaa: true
|
| 14 |
+
model:
|
| 15 |
+
use_dual_branch: true
|
| 16 |
+
loss:
|
| 17 |
+
type: focal_plus_weighted_ce
|
| 18 |
+
training:
|
| 19 |
+
batch_size: 32
|
| 20 |
+
epochs: 100
|
| 21 |
+
optimizer: adamw
|
| 22 |
+
learning_rate: 1.0e-4
|
| 23 |
+
min_learning_rate: 1.0e-6
|
configs/phase2/s1_baseline.yaml
ADDED
|
@@ -0,0 +1,19 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
project:
|
| 2 |
+
name: DIPAug-SeverNet
|
| 3 |
+
phase: phase2
|
| 4 |
+
experiment_id: S1
|
| 5 |
+
dataset:
|
| 6 |
+
num_classes: 11
|
| 7 |
+
image_size: 384
|
| 8 |
+
split_seed: 42
|
| 9 |
+
inputs:
|
| 10 |
+
use_dgsm_channel: false
|
| 11 |
+
pretraining:
|
| 12 |
+
use_simclr: false
|
| 13 |
+
model:
|
| 14 |
+
use_segmentation_decoder: false
|
| 15 |
+
loss:
|
| 16 |
+
multitask: false
|
| 17 |
+
training:
|
| 18 |
+
warmup_epochs: 20
|
| 19 |
+
joint_epochs: 80
|
configs/phase2/s2_segmentation.yaml
ADDED
|
@@ -0,0 +1,20 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
project:
|
| 2 |
+
name: DIPAug-SeverNet
|
| 3 |
+
phase: phase2
|
| 4 |
+
experiment_id: S2
|
| 5 |
+
dataset:
|
| 6 |
+
num_classes: 11
|
| 7 |
+
image_size: 384
|
| 8 |
+
split_seed: 42
|
| 9 |
+
inputs:
|
| 10 |
+
use_dgsm_channel: false
|
| 11 |
+
pretraining:
|
| 12 |
+
use_simclr: false
|
| 13 |
+
model:
|
| 14 |
+
use_segmentation_decoder: true
|
| 15 |
+
loss:
|
| 16 |
+
multitask: false
|
| 17 |
+
segmentation: bce_dice
|
| 18 |
+
training:
|
| 19 |
+
warmup_epochs: 20
|
| 20 |
+
joint_epochs: 80
|
configs/phase2/s3_dgsm.yaml
ADDED
|
@@ -0,0 +1,20 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
project:
|
| 2 |
+
name: DIPAug-SeverNet
|
| 3 |
+
phase: phase2
|
| 4 |
+
experiment_id: S3
|
| 5 |
+
dataset:
|
| 6 |
+
num_classes: 11
|
| 7 |
+
image_size: 384
|
| 8 |
+
split_seed: 42
|
| 9 |
+
inputs:
|
| 10 |
+
use_dgsm_channel: true
|
| 11 |
+
pretraining:
|
| 12 |
+
use_simclr: false
|
| 13 |
+
model:
|
| 14 |
+
use_segmentation_decoder: true
|
| 15 |
+
loss:
|
| 16 |
+
multitask: false
|
| 17 |
+
segmentation: bce_dice
|
| 18 |
+
training:
|
| 19 |
+
warmup_epochs: 20
|
| 20 |
+
joint_epochs: 80
|
configs/phase2/s4_simclr.yaml
ADDED
|
@@ -0,0 +1,20 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
project:
|
| 2 |
+
name: DIPAug-SeverNet
|
| 3 |
+
phase: phase2
|
| 4 |
+
experiment_id: S4
|
| 5 |
+
dataset:
|
| 6 |
+
num_classes: 11
|
| 7 |
+
image_size: 384
|
| 8 |
+
split_seed: 42
|
| 9 |
+
inputs:
|
| 10 |
+
use_dgsm_channel: true
|
| 11 |
+
pretraining:
|
| 12 |
+
use_simclr: true
|
| 13 |
+
model:
|
| 14 |
+
use_segmentation_decoder: true
|
| 15 |
+
loss:
|
| 16 |
+
multitask: false
|
| 17 |
+
segmentation: bce_dice
|
| 18 |
+
training:
|
| 19 |
+
warmup_epochs: 20
|
| 20 |
+
joint_epochs: 80
|
configs/phase2/s5_full.yaml
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
project:
|
| 2 |
+
name: DIPAug-SeverNet
|
| 3 |
+
phase: phase2
|
| 4 |
+
experiment_id: S5
|
| 5 |
+
dataset:
|
| 6 |
+
num_classes: 11
|
| 7 |
+
image_size: 384
|
| 8 |
+
split_seed: 42
|
| 9 |
+
inputs:
|
| 10 |
+
use_dgsm_channel: true
|
| 11 |
+
pretraining:
|
| 12 |
+
use_simclr: true
|
| 13 |
+
model:
|
| 14 |
+
use_segmentation_decoder: true
|
| 15 |
+
loss:
|
| 16 |
+
multitask: true
|
| 17 |
+
segmentation: bce_dice
|
| 18 |
+
uncertainty_weighting: true
|
| 19 |
+
training:
|
| 20 |
+
warmup_epochs: 20
|
| 21 |
+
joint_epochs: 80
|
dipauglib/__init__.py
ADDED
|
@@ -0,0 +1 @@
|
|
|
|
|
|
|
| 1 |
+
"""Shared library for DIPAug project components."""
|
dipauglib/sampling/__init__.py
ADDED
|
@@ -0,0 +1,5 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Sampling utilities."""
|
| 2 |
+
|
| 3 |
+
from .class_imbalance import build_weighted_sampler, class_weights_from_counts, minority_class_names
|
| 4 |
+
|
| 5 |
+
__all__ = ["class_weights_from_counts", "minority_class_names", "build_weighted_sampler"]
|
dipauglib/sampling/class_imbalance.py
ADDED
|
@@ -0,0 +1,30 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Class imbalance helpers for WeightedRandomSampler and augmentation scaling."""
|
| 2 |
+
|
| 3 |
+
from __future__ import annotations
|
| 4 |
+
|
| 5 |
+
from collections.abc import Sequence
|
| 6 |
+
|
| 7 |
+
import torch
|
| 8 |
+
|
| 9 |
+
|
| 10 |
+
def class_weights_from_counts(class_counts: dict[str, int]) -> dict[str, float]:
|
| 11 |
+
"""Compute inverse-frequency class weights."""
|
| 12 |
+
|
| 13 |
+
total = float(sum(class_counts.values()))
|
| 14 |
+
num_classes = float(len(class_counts))
|
| 15 |
+
return {name: total / (num_classes * max(1, count)) for name, count in class_counts.items()}
|
| 16 |
+
|
| 17 |
+
|
| 18 |
+
def minority_class_names(class_counts: dict[str, int], threshold_ratio: float = 0.15) -> set[str]:
|
| 19 |
+
"""Return minority class names under the requested ratio."""
|
| 20 |
+
|
| 21 |
+
total = float(sum(class_counts.values()))
|
| 22 |
+
return {name for name, count in class_counts.items() if count / max(total, 1.0) < threshold_ratio}
|
| 23 |
+
|
| 24 |
+
|
| 25 |
+
def build_weighted_sampler(labels: Sequence[str], class_counts: dict[str, int]) -> torch.utils.data.WeightedRandomSampler:
|
| 26 |
+
"""Create a weighted sampler from labels."""
|
| 27 |
+
|
| 28 |
+
weights = class_weights_from_counts(class_counts)
|
| 29 |
+
sample_weights = torch.as_tensor([weights[label] for label in labels], dtype=torch.double)
|
| 30 |
+
return torch.utils.data.WeightedRandomSampler(sample_weights, num_samples=len(sample_weights), replacement=True)
|
dipauglib/schedulers/__init__.py
ADDED
|
@@ -0,0 +1,5 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Training schedulers."""
|
| 2 |
+
|
| 3 |
+
from .adaptive import AdaptiveAugmentationScheduler
|
| 4 |
+
|
| 5 |
+
__all__ = ["AdaptiveAugmentationScheduler"]
|
dipauglib/schedulers/adaptive.py
ADDED
|
@@ -0,0 +1,27 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Adaptive augmentation scheduler."""
|
| 2 |
+
|
| 3 |
+
from __future__ import annotations
|
| 4 |
+
|
| 5 |
+
import math
|
| 6 |
+
from dataclasses import dataclass
|
| 7 |
+
|
| 8 |
+
|
| 9 |
+
@dataclass
|
| 10 |
+
class AdaptiveAugmentationScheduler:
|
| 11 |
+
"""Sigmoid curriculum for augmentation intensity."""
|
| 12 |
+
|
| 13 |
+
d_min: float = 0.1
|
| 14 |
+
d_max: float = 1.0
|
| 15 |
+
t_half: float = 50.0
|
| 16 |
+
tau: float = 10.0
|
| 17 |
+
|
| 18 |
+
def intensity_at(self, epoch: int) -> float:
|
| 19 |
+
"""Return augmentation intensity for an epoch."""
|
| 20 |
+
|
| 21 |
+
value = self.d_min + (self.d_max - self.d_min) * (1.0 / (1.0 + math.exp(-(epoch - self.t_half) / self.tau)))
|
| 22 |
+
return float(value)
|
| 23 |
+
|
| 24 |
+
def log_payload(self, epoch: int) -> dict[str, float]:
|
| 25 |
+
"""Return dashboard-friendly payload."""
|
| 26 |
+
|
| 27 |
+
return {"epoch": float(epoch), "augmentation_intensity": self.intensity_at(epoch)}
|
dipauglib/transforms/__init__.py
ADDED
|
@@ -0,0 +1,23 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Physics-aware image augmentation transforms."""
|
| 2 |
+
|
| 3 |
+
from .physics import (
|
| 4 |
+
CastShadow,
|
| 5 |
+
ColourFade,
|
| 6 |
+
ColourTempShift,
|
| 7 |
+
DefocusBlur,
|
| 8 |
+
DustOverlay,
|
| 9 |
+
IlluminationGradient,
|
| 10 |
+
MotionBlur,
|
| 11 |
+
SensorNoise,
|
| 12 |
+
)
|
| 13 |
+
|
| 14 |
+
__all__ = [
|
| 15 |
+
"IlluminationGradient",
|
| 16 |
+
"CastShadow",
|
| 17 |
+
"MotionBlur",
|
| 18 |
+
"DefocusBlur",
|
| 19 |
+
"ColourTempShift",
|
| 20 |
+
"ColourFade",
|
| 21 |
+
"DustOverlay",
|
| 22 |
+
"SensorNoise",
|
| 23 |
+
]
|
dipauglib/transforms/physics.py
ADDED
|
@@ -0,0 +1,239 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Physics-aware augmentation classes compatible with Albumentations.
|
| 2 |
+
|
| 3 |
+
All transforms inherit from Albumentations ``DualTransform`` so image-mask
|
| 4 |
+
consistency is preserved for downstream segmentation use.
|
| 5 |
+
"""
|
| 6 |
+
|
| 7 |
+
from __future__ import annotations
|
| 8 |
+
|
| 9 |
+
import math
|
| 10 |
+
from typing import Any
|
| 11 |
+
|
| 12 |
+
import cv2
|
| 13 |
+
import numpy as np
|
| 14 |
+
from albumentations.core.transforms_interface import DualTransform
|
| 15 |
+
|
| 16 |
+
|
| 17 |
+
def _ensure_uint8(image: np.ndarray) -> np.ndarray:
|
| 18 |
+
if image.dtype == np.uint8:
|
| 19 |
+
return image.copy()
|
| 20 |
+
clipped = np.clip(image, 0, 255)
|
| 21 |
+
return clipped.astype(np.uint8)
|
| 22 |
+
|
| 23 |
+
|
| 24 |
+
def _clip_float_image(image: np.ndarray) -> np.ndarray:
|
| 25 |
+
return np.clip(image, 0, 255).astype(np.uint8)
|
| 26 |
+
|
| 27 |
+
|
| 28 |
+
def _scaled_value(intensity: float, low: float, high: float) -> float:
|
| 29 |
+
return low + (high - low) * float(np.clip(intensity, 0.0, 1.0))
|
| 30 |
+
|
| 31 |
+
|
| 32 |
+
def _kelvin_to_rgb(cct_kelvin: float) -> np.ndarray:
|
| 33 |
+
temperature = max(1000.0, min(40000.0, cct_kelvin)) / 100.0
|
| 34 |
+
|
| 35 |
+
if temperature <= 66:
|
| 36 |
+
red = 255
|
| 37 |
+
green = 99.4708025861 * math.log(temperature) - 161.1195681661
|
| 38 |
+
blue = 0 if temperature <= 19 else 138.5177312231 * math.log(temperature - 10) - 305.0447927307
|
| 39 |
+
else:
|
| 40 |
+
red = 329.698727446 * ((temperature - 60) ** -0.1332047592)
|
| 41 |
+
green = 288.1221695283 * ((temperature - 60) ** -0.0755148492)
|
| 42 |
+
blue = 255
|
| 43 |
+
|
| 44 |
+
return np.clip(np.array([red, green, blue], dtype=np.float32), 0, 255)
|
| 45 |
+
|
| 46 |
+
|
| 47 |
+
class DIPDualTransform(DualTransform):
|
| 48 |
+
"""Base transform with image-only mutation and mask passthrough."""
|
| 49 |
+
|
| 50 |
+
def __init__(self, intensity: float = 1.0, always_apply: bool = False, p: float = 0.5):
|
| 51 |
+
super().__init__(always_apply=always_apply, p=p)
|
| 52 |
+
self.intensity = float(np.clip(intensity, 0.0, 1.0))
|
| 53 |
+
|
| 54 |
+
def apply_to_mask(self, mask: np.ndarray, **params: Any) -> np.ndarray:
|
| 55 |
+
return mask
|
| 56 |
+
|
| 57 |
+
def get_transform_init_args_names(self) -> tuple[str, ...]:
|
| 58 |
+
return ("intensity",)
|
| 59 |
+
|
| 60 |
+
|
| 61 |
+
class IlluminationGradient(DIPDualTransform):
|
| 62 |
+
"""Apply a bidirectional illumination gradient on LAB L-channel."""
|
| 63 |
+
|
| 64 |
+
def get_params(self) -> dict[str, float]:
|
| 65 |
+
return {
|
| 66 |
+
"angle": float(np.random.uniform(0.0, 360.0)),
|
| 67 |
+
"strength": float(_scaled_value(self.intensity, 0.3, 0.8)),
|
| 68 |
+
}
|
| 69 |
+
|
| 70 |
+
def apply(self, img: np.ndarray, angle: float = 0.0, strength: float = 0.5, **params: Any) -> np.ndarray:
|
| 71 |
+
image = _ensure_uint8(img)
|
| 72 |
+
h, w = image.shape[:2]
|
| 73 |
+
yy, xx = np.mgrid[0:h, 0:w].astype(np.float32)
|
| 74 |
+
cx, cy = w / 2.0, h / 2.0
|
| 75 |
+
theta = np.deg2rad(angle)
|
| 76 |
+
projection = ((xx - cx) * np.cos(theta) + (yy - cy) * np.sin(theta))
|
| 77 |
+
projection = projection / (np.max(np.abs(projection)) + 1e-6)
|
| 78 |
+
gradient = 1.0 + projection * strength * 0.5
|
| 79 |
+
|
| 80 |
+
lab = cv2.cvtColor(image, cv2.COLOR_RGB2LAB).astype(np.float32)
|
| 81 |
+
lab[..., 0] = np.clip(lab[..., 0] * gradient, 0, 255)
|
| 82 |
+
return cv2.cvtColor(_clip_float_image(lab), cv2.COLOR_LAB2RGB)
|
| 83 |
+
|
| 84 |
+
|
| 85 |
+
class CastShadow(DIPDualTransform):
|
| 86 |
+
"""Apply polygonal canopy shadow with penumbra blur."""
|
| 87 |
+
|
| 88 |
+
def get_params(self) -> dict[str, float]:
|
| 89 |
+
return {
|
| 90 |
+
"area": float(_scaled_value(self.intensity, 0.1, 0.4)),
|
| 91 |
+
"blur_sigma": float(_scaled_value(self.intensity, 5.0, 20.0)),
|
| 92 |
+
"vertices": int(np.random.randint(3, 7)),
|
| 93 |
+
}
|
| 94 |
+
|
| 95 |
+
def apply(self, img: np.ndarray, area: float = 0.2, blur_sigma: float = 10.0, vertices: int = 4, **params: Any) -> np.ndarray:
|
| 96 |
+
image = _ensure_uint8(img)
|
| 97 |
+
h, w = image.shape[:2]
|
| 98 |
+
center = np.array([np.random.uniform(0, w), np.random.uniform(0, h)], dtype=np.float32)
|
| 99 |
+
radius = math.sqrt(max(1.0, area * h * w / math.pi))
|
| 100 |
+
angles = np.sort(np.random.uniform(0, 2 * math.pi, size=vertices))
|
| 101 |
+
polygon = []
|
| 102 |
+
for theta in angles:
|
| 103 |
+
scale = np.random.uniform(0.6, 1.2)
|
| 104 |
+
x = center[0] + radius * scale * math.cos(theta)
|
| 105 |
+
y = center[1] + radius * scale * math.sin(theta)
|
| 106 |
+
polygon.append([int(np.clip(x, 0, w - 1)), int(np.clip(y, 0, h - 1))])
|
| 107 |
+
|
| 108 |
+
mask = np.zeros((h, w), dtype=np.float32)
|
| 109 |
+
cv2.fillPoly(mask, [np.array(polygon, dtype=np.int32)], 1.0)
|
| 110 |
+
sigma = max(0.1, blur_sigma)
|
| 111 |
+
mask = cv2.GaussianBlur(mask, (0, 0), sigmaX=sigma, sigmaY=sigma)
|
| 112 |
+
alpha = np.expand_dims(np.clip(mask * 0.55, 0.0, 0.8), axis=-1)
|
| 113 |
+
shaded = image.astype(np.float32) * (1.0 - alpha)
|
| 114 |
+
return _clip_float_image(shaded)
|
| 115 |
+
|
| 116 |
+
|
| 117 |
+
class MotionBlur(DIPDualTransform):
|
| 118 |
+
"""Apply directional linear PSF motion blur."""
|
| 119 |
+
|
| 120 |
+
def get_params(self) -> dict[str, float]:
|
| 121 |
+
kernel_size = int(round(_scaled_value(self.intensity, 5.0, 25.0)))
|
| 122 |
+
if kernel_size % 2 == 0:
|
| 123 |
+
kernel_size += 1
|
| 124 |
+
return {
|
| 125 |
+
"kernel_size": float(kernel_size),
|
| 126 |
+
"angle": float(np.random.uniform(0.0, 180.0)),
|
| 127 |
+
}
|
| 128 |
+
|
| 129 |
+
def apply(self, img: np.ndarray, kernel_size: float = 9, angle: float = 0.0, **params: Any) -> np.ndarray:
|
| 130 |
+
image = _ensure_uint8(img)
|
| 131 |
+
size = int(kernel_size)
|
| 132 |
+
kernel = np.zeros((size, size), dtype=np.float32)
|
| 133 |
+
cv2.line(kernel, (0, size // 2), (size - 1, size // 2), 1, thickness=1)
|
| 134 |
+
rot_mat = cv2.getRotationMatrix2D((size / 2.0 - 0.5, size / 2.0 - 0.5), angle, 1.0)
|
| 135 |
+
kernel = cv2.warpAffine(kernel, rot_mat, (size, size))
|
| 136 |
+
kernel = kernel / (kernel.sum() + 1e-6)
|
| 137 |
+
return cv2.filter2D(image, -1, kernel)
|
| 138 |
+
|
| 139 |
+
|
| 140 |
+
class DefocusBlur(DIPDualTransform):
|
| 141 |
+
"""Apply circular pillbox defocus blur."""
|
| 142 |
+
|
| 143 |
+
def get_params(self) -> dict[str, float]:
|
| 144 |
+
return {"radius": float(_scaled_value(self.intensity, 3.0, 15.0))}
|
| 145 |
+
|
| 146 |
+
def apply(self, img: np.ndarray, radius: float = 5.0, **params: Any) -> np.ndarray:
|
| 147 |
+
image = _ensure_uint8(img)
|
| 148 |
+
radius_int = max(1, int(round(radius)))
|
| 149 |
+
size = radius_int * 2 + 1
|
| 150 |
+
kernel = np.zeros((size, size), dtype=np.float32)
|
| 151 |
+
cv2.circle(kernel, (radius_int, radius_int), radius_int, 1, thickness=-1)
|
| 152 |
+
kernel /= kernel.sum() + 1e-6
|
| 153 |
+
return cv2.filter2D(image, -1, kernel)
|
| 154 |
+
|
| 155 |
+
|
| 156 |
+
class ColourTempShift(DIPDualTransform):
|
| 157 |
+
"""Apply approximate correlated colour temperature shift."""
|
| 158 |
+
|
| 159 |
+
def get_params(self) -> dict[str, float]:
|
| 160 |
+
return {"cct_kelvin": float(_scaled_value(self.intensity, 3200.0, 8000.0))}
|
| 161 |
+
|
| 162 |
+
def apply(self, img: np.ndarray, cct_kelvin: float = 6500.0, **params: Any) -> np.ndarray:
|
| 163 |
+
image = _ensure_uint8(img).astype(np.float32)
|
| 164 |
+
rgb = _kelvin_to_rgb(cct_kelvin) / 255.0
|
| 165 |
+
neutral = _kelvin_to_rgb(6500.0) / 255.0
|
| 166 |
+
gain = rgb / np.maximum(neutral, 1e-6)
|
| 167 |
+
shifted = image * gain.reshape(1, 1, 3)
|
| 168 |
+
return _clip_float_image(shifted)
|
| 169 |
+
|
| 170 |
+
|
| 171 |
+
class ColourFade(DIPDualTransform):
|
| 172 |
+
"""Reduce saturation and adjust gamma on luminance."""
|
| 173 |
+
|
| 174 |
+
def get_params(self) -> dict[str, float]:
|
| 175 |
+
return {
|
| 176 |
+
"sat_factor": float(_scaled_value(self.intensity, 0.3, 0.7)),
|
| 177 |
+
"gamma": float(_scaled_value(self.intensity, 0.6, 1.4)),
|
| 178 |
+
}
|
| 179 |
+
|
| 180 |
+
def apply(self, img: np.ndarray, sat_factor: float = 0.5, gamma: float = 1.0, **params: Any) -> np.ndarray:
|
| 181 |
+
image = _ensure_uint8(img)
|
| 182 |
+
hsv = cv2.cvtColor(image, cv2.COLOR_RGB2HSV).astype(np.float32)
|
| 183 |
+
hsv[..., 1] *= 1.0 - sat_factor
|
| 184 |
+
faded = cv2.cvtColor(_clip_float_image(hsv), cv2.COLOR_HSV2RGB)
|
| 185 |
+
lab = cv2.cvtColor(faded, cv2.COLOR_RGB2LAB).astype(np.float32)
|
| 186 |
+
lab[..., 0] = 255.0 * np.power(np.clip(lab[..., 0] / 255.0, 0, 1), gamma)
|
| 187 |
+
return cv2.cvtColor(_clip_float_image(lab), cv2.COLOR_LAB2RGB)
|
| 188 |
+
|
| 189 |
+
|
| 190 |
+
class DustOverlay(DIPDualTransform):
|
| 191 |
+
"""Add semi-transparent dust particles using ellipses."""
|
| 192 |
+
|
| 193 |
+
def get_params(self) -> dict[str, float]:
|
| 194 |
+
return {
|
| 195 |
+
"n_particles": float(_scaled_value(self.intensity, 50.0, 300.0)),
|
| 196 |
+
"opacity": float(_scaled_value(self.intensity, 0.2, 0.6)),
|
| 197 |
+
}
|
| 198 |
+
|
| 199 |
+
def apply(self, img: np.ndarray, n_particles: float = 100, opacity: float = 0.3, **params: Any) -> np.ndarray:
|
| 200 |
+
image = _ensure_uint8(img).astype(np.float32)
|
| 201 |
+
h, w = image.shape[:2]
|
| 202 |
+
overlay = np.zeros_like(image, dtype=np.float32)
|
| 203 |
+
particle_count = int(max(1, round(n_particles)))
|
| 204 |
+
for _ in range(particle_count):
|
| 205 |
+
center = (int(np.random.randint(0, w)), int(np.random.randint(0, h)))
|
| 206 |
+
axes = (int(np.random.randint(1, 6)), int(np.random.randint(1, 6)))
|
| 207 |
+
angle = float(np.random.uniform(0, 180))
|
| 208 |
+
color = float(np.random.uniform(180, 255))
|
| 209 |
+
cv2.ellipse(overlay, center, axes, angle, 0, 360, (color, color, color), -1)
|
| 210 |
+
|
| 211 |
+
overlay = cv2.GaussianBlur(overlay, (0, 0), sigmaX=1.5, sigmaY=1.5)
|
| 212 |
+
dusted = image * (1.0 - opacity * 0.4) + overlay * opacity * 0.35
|
| 213 |
+
return _clip_float_image(dusted)
|
| 214 |
+
|
| 215 |
+
|
| 216 |
+
class SensorNoise(DIPDualTransform):
|
| 217 |
+
"""Apply mixed Gaussian, Poisson, and JPEG noise."""
|
| 218 |
+
|
| 219 |
+
def get_params(self) -> dict[str, float]:
|
| 220 |
+
return {
|
| 221 |
+
"sigma": float(_scaled_value(self.intensity, 5.0, 30.0)),
|
| 222 |
+
"jpeg_qf": float(_scaled_value(self.intensity, 40.0, 90.0)),
|
| 223 |
+
}
|
| 224 |
+
|
| 225 |
+
def apply(self, img: np.ndarray, sigma: float = 10.0, jpeg_qf: float = 80.0, **params: Any) -> np.ndarray:
|
| 226 |
+
image = _ensure_uint8(img).astype(np.float32)
|
| 227 |
+
gaussian = np.random.normal(0.0, sigma, size=image.shape).astype(np.float32)
|
| 228 |
+
noisy = image + gaussian
|
| 229 |
+
|
| 230 |
+
scaled = np.clip(noisy / 255.0, 0.0, 1.0)
|
| 231 |
+
poisson = np.random.poisson(scaled * 255.0).astype(np.float32) / 255.0
|
| 232 |
+
noisy = np.clip(poisson * 255.0, 0.0, 255.0).astype(np.uint8)
|
| 233 |
+
|
| 234 |
+
encode_params = [int(cv2.IMWRITE_JPEG_QUALITY), int(np.clip(jpeg_qf, 10, 100))]
|
| 235 |
+
success, encoded = cv2.imencode(".jpg", cv2.cvtColor(noisy, cv2.COLOR_RGB2BGR), encode_params)
|
| 236 |
+
if not success:
|
| 237 |
+
return noisy
|
| 238 |
+
decoded = cv2.imdecode(encoded, cv2.IMREAD_COLOR)
|
| 239 |
+
return cv2.cvtColor(decoded, cv2.COLOR_BGR2RGB)
|
dipauglib/transforms/pipeline.py
ADDED
|
@@ -0,0 +1,38 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Augmentation pipeline helpers."""
|
| 2 |
+
|
| 3 |
+
from __future__ import annotations
|
| 4 |
+
|
| 5 |
+
from albumentations import Compose, Normalize, Resize
|
| 6 |
+
|
| 7 |
+
from .physics import (
|
| 8 |
+
CastShadow,
|
| 9 |
+
ColourFade,
|
| 10 |
+
ColourTempShift,
|
| 11 |
+
DefocusBlur,
|
| 12 |
+
DustOverlay,
|
| 13 |
+
IlluminationGradient,
|
| 14 |
+
MotionBlur,
|
| 15 |
+
SensorNoise,
|
| 16 |
+
)
|
| 17 |
+
|
| 18 |
+
|
| 19 |
+
IMAGENET_MEAN = (0.485, 0.456, 0.406)
|
| 20 |
+
IMAGENET_STD = (0.229, 0.224, 0.225)
|
| 21 |
+
|
| 22 |
+
|
| 23 |
+
def build_dipaug_pipeline(intensity: float = 1.0, image_size: int = 384) -> Compose:
|
| 24 |
+
"""Build the full DIPAug pipeline."""
|
| 25 |
+
|
| 26 |
+
transforms = [
|
| 27 |
+
Resize(image_size, image_size),
|
| 28 |
+
IlluminationGradient(intensity=intensity, p=0.35),
|
| 29 |
+
CastShadow(intensity=intensity, p=0.35),
|
| 30 |
+
MotionBlur(intensity=intensity, p=0.25),
|
| 31 |
+
DefocusBlur(intensity=intensity, p=0.25),
|
| 32 |
+
ColourTempShift(intensity=intensity, p=0.35),
|
| 33 |
+
ColourFade(intensity=intensity, p=0.35),
|
| 34 |
+
DustOverlay(intensity=intensity, p=0.25),
|
| 35 |
+
SensorNoise(intensity=intensity, p=0.35),
|
| 36 |
+
Normalize(mean=IMAGENET_MEAN, std=IMAGENET_STD),
|
| 37 |
+
]
|
| 38 |
+
return Compose(transforms)
|
dipauglib/utils/__init__.py
ADDED
|
@@ -0,0 +1 @@
|
|
|
|
|
|
|
| 1 |
+
"""Shared project utilities."""
|
dipauglib/utils/dataset.py
ADDED
|
@@ -0,0 +1,53 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Dataset split and manifest helpers."""
|
| 2 |
+
|
| 3 |
+
from __future__ import annotations
|
| 4 |
+
|
| 5 |
+
from dataclasses import dataclass
|
| 6 |
+
from pathlib import Path
|
| 7 |
+
|
| 8 |
+
import pandas as pd
|
| 9 |
+
from sklearn.model_selection import StratifiedShuffleSplit
|
| 10 |
+
|
| 11 |
+
|
| 12 |
+
@dataclass
|
| 13 |
+
class SplitConfig:
|
| 14 |
+
"""Stratified split configuration."""
|
| 15 |
+
|
| 16 |
+
seed: int = 42
|
| 17 |
+
train_size: float = 0.70
|
| 18 |
+
val_size: float = 0.15
|
| 19 |
+
test_size: float = 0.15
|
| 20 |
+
|
| 21 |
+
|
| 22 |
+
def build_split_manifest(records: pd.DataFrame, label_column: str, config: SplitConfig = SplitConfig()) -> pd.DataFrame:
|
| 23 |
+
"""Build reproducible 70/15/15 stratified splits."""
|
| 24 |
+
|
| 25 |
+
if abs(config.train_size + config.val_size + config.test_size - 1.0) > 1e-6:
|
| 26 |
+
raise ValueError("Split fractions must sum to 1.0")
|
| 27 |
+
|
| 28 |
+
splitter = StratifiedShuffleSplit(n_splits=1, test_size=(1.0 - config.train_size), random_state=config.seed)
|
| 29 |
+
train_idx, temp_idx = next(splitter.split(records, records[label_column]))
|
| 30 |
+
train_df = records.iloc[train_idx].copy()
|
| 31 |
+
temp_df = records.iloc[temp_idx].copy()
|
| 32 |
+
|
| 33 |
+
val_ratio_within_temp = config.val_size / (config.val_size + config.test_size)
|
| 34 |
+
splitter_temp = StratifiedShuffleSplit(n_splits=1, test_size=(1.0 - val_ratio_within_temp), random_state=config.seed)
|
| 35 |
+
val_inner, test_inner = next(splitter_temp.split(temp_df, temp_df[label_column]))
|
| 36 |
+
|
| 37 |
+
val_df = temp_df.iloc[val_inner].copy()
|
| 38 |
+
test_df = temp_df.iloc[test_inner].copy()
|
| 39 |
+
|
| 40 |
+
train_df["split"] = "train"
|
| 41 |
+
val_df["split"] = "val"
|
| 42 |
+
test_df["split"] = "test"
|
| 43 |
+
manifest = pd.concat([train_df, val_df, test_df], ignore_index=True)
|
| 44 |
+
return manifest
|
| 45 |
+
|
| 46 |
+
|
| 47 |
+
def save_manifest(manifest: pd.DataFrame, output_path: str | Path) -> Path:
|
| 48 |
+
"""Save manifest CSV."""
|
| 49 |
+
|
| 50 |
+
path = Path(output_path)
|
| 51 |
+
path.parent.mkdir(parents=True, exist_ok=True)
|
| 52 |
+
manifest.to_csv(path, index=False)
|
| 53 |
+
return path
|
dipauglib/utils/io.py
ADDED
|
@@ -0,0 +1,14 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Configuration and file IO utilities."""
|
| 2 |
+
|
| 3 |
+
from __future__ import annotations
|
| 4 |
+
|
| 5 |
+
from pathlib import Path
|
| 6 |
+
from typing import Any
|
| 7 |
+
|
| 8 |
+
import yaml
|
| 9 |
+
|
| 10 |
+
|
| 11 |
+
def load_yaml(path: str | Path) -> dict[str, Any]:
|
| 12 |
+
"""Load a YAML configuration file."""
|
| 13 |
+
|
| 14 |
+
return yaml.safe_load(Path(path).read_text(encoding="utf-8"))
|
dipauglib/utils/repro.py
ADDED
|
@@ -0,0 +1,19 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Reproducibility helpers."""
|
| 2 |
+
|
| 3 |
+
from __future__ import annotations
|
| 4 |
+
|
| 5 |
+
import random
|
| 6 |
+
|
| 7 |
+
import numpy as np
|
| 8 |
+
import torch
|
| 9 |
+
|
| 10 |
+
|
| 11 |
+
def seed_everything(seed: int = 42) -> None:
|
| 12 |
+
"""Fix Python, NumPy, and PyTorch seeds."""
|
| 13 |
+
|
| 14 |
+
random.seed(seed)
|
| 15 |
+
np.random.seed(seed)
|
| 16 |
+
torch.manual_seed(seed)
|
| 17 |
+
torch.cuda.manual_seed_all(seed)
|
| 18 |
+
torch.backends.cudnn.deterministic = True
|
| 19 |
+
torch.backends.cudnn.benchmark = False
|
dipaugnet/__init__.py
ADDED
|
@@ -0,0 +1 @@
|
|
|
|
|
|
|
| 1 |
+
"""Phase 1 DIPAug-Net package."""
|
dipaugnet/evaluation/__init__.py
ADDED
|
@@ -0,0 +1 @@
|
|
|
|
|
|
|
| 1 |
+
"""Phase 1 evaluation package."""
|
dipaugnet/evaluation/metrics.py
ADDED
|
@@ -0,0 +1,27 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Evaluation helpers for Paper 1."""
|
| 2 |
+
|
| 3 |
+
from __future__ import annotations
|
| 4 |
+
|
| 5 |
+
import numpy as np
|
| 6 |
+
from sklearn.metrics import accuracy_score, cohen_kappa_score, f1_score, precision_recall_fscore_support
|
| 7 |
+
|
| 8 |
+
|
| 9 |
+
def classification_summary(y_true: list[int], y_pred: list[int]) -> dict[str, float]:
|
| 10 |
+
"""Compute core classification metrics."""
|
| 11 |
+
|
| 12 |
+
precision, recall, f1, _ = precision_recall_fscore_support(y_true, y_pred, average="macro", zero_division=0)
|
| 13 |
+
return {
|
| 14 |
+
"accuracy": float(accuracy_score(y_true, y_pred)),
|
| 15 |
+
"macro_precision": float(precision),
|
| 16 |
+
"macro_recall": float(recall),
|
| 17 |
+
"macro_f1": float(f1),
|
| 18 |
+
"cohen_kappa": float(cohen_kappa_score(y_true, y_pred)),
|
| 19 |
+
}
|
| 20 |
+
|
| 21 |
+
|
| 22 |
+
def relative_performance_retention(clean_metric: float, distorted_metric: float) -> float:
|
| 23 |
+
"""Compute relative performance retention."""
|
| 24 |
+
|
| 25 |
+
if clean_metric == 0:
|
| 26 |
+
return 0.0
|
| 27 |
+
return float(distorted_metric / clean_metric)
|
dipaugnet/models/__init__.py
ADDED
|
@@ -0,0 +1,5 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Phase 1 model definitions."""
|
| 2 |
+
|
| 3 |
+
from .dipaugnet import DIPAugNet
|
| 4 |
+
|
| 5 |
+
__all__ = ["DIPAugNet"]
|
dipaugnet/models/dipaugnet.py
ADDED
|
@@ -0,0 +1,84 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""DIPAug-Net architecture skeleton."""
|
| 2 |
+
|
| 3 |
+
from __future__ import annotations
|
| 4 |
+
|
| 5 |
+
import torch
|
| 6 |
+
from torch import nn
|
| 7 |
+
import torch.nn.functional as F
|
| 8 |
+
import timm
|
| 9 |
+
|
| 10 |
+
|
| 11 |
+
class FeatureProjector(nn.Module):
|
| 12 |
+
"""Project backbone feature maps into a shared embedding dimension."""
|
| 13 |
+
|
| 14 |
+
def __init__(self, in_channels: int, out_channels: int = 512):
|
| 15 |
+
super().__init__()
|
| 16 |
+
self.proj = nn.Sequential(
|
| 17 |
+
nn.Conv2d(in_channels, out_channels, kernel_size=1, bias=False),
|
| 18 |
+
nn.BatchNorm2d(out_channels),
|
| 19 |
+
nn.GELU(),
|
| 20 |
+
)
|
| 21 |
+
|
| 22 |
+
def forward(self, x: torch.Tensor) -> torch.Tensor:
|
| 23 |
+
return self.proj(x)
|
| 24 |
+
|
| 25 |
+
|
| 26 |
+
class DIPAugNet(nn.Module):
|
| 27 |
+
"""Dual-branch CNN + Transformer classifier with cross-attention fusion."""
|
| 28 |
+
|
| 29 |
+
def __init__(self, num_classes: int = 11, embed_dim: int = 512, attn_heads: int = 8, dropout: float = 0.4):
|
| 30 |
+
super().__init__()
|
| 31 |
+
self.cnn = timm.create_model(
|
| 32 |
+
"efficientnet_b3.ra2_in1k",
|
| 33 |
+
pretrained=True,
|
| 34 |
+
features_only=True,
|
| 35 |
+
out_indices=(2, 3),
|
| 36 |
+
)
|
| 37 |
+
self.transformer = timm.create_model(
|
| 38 |
+
"swin_tiny_patch4_window7_224.ms_in1k",
|
| 39 |
+
pretrained=True,
|
| 40 |
+
features_only=True,
|
| 41 |
+
out_indices=(1, 2),
|
| 42 |
+
)
|
| 43 |
+
|
| 44 |
+
cnn_channels = self.cnn.feature_info.channels()
|
| 45 |
+
tr_channels = self.transformer.feature_info.channels()
|
| 46 |
+
|
| 47 |
+
self.cnn_proj_8 = FeatureProjector(cnn_channels[0], embed_dim)
|
| 48 |
+
self.cnn_proj_16 = FeatureProjector(cnn_channels[1], embed_dim)
|
| 49 |
+
self.tr_proj_8 = FeatureProjector(tr_channels[0], embed_dim)
|
| 50 |
+
self.tr_proj_16 = FeatureProjector(tr_channels[1], embed_dim)
|
| 51 |
+
|
| 52 |
+
self.cross_attn = nn.MultiheadAttention(embed_dim=embed_dim, num_heads=attn_heads, batch_first=True)
|
| 53 |
+
self.fusion_proj = nn.Linear(embed_dim, embed_dim)
|
| 54 |
+
self.head = nn.Sequential(
|
| 55 |
+
nn.Linear(embed_dim * 3, embed_dim),
|
| 56 |
+
nn.GELU(),
|
| 57 |
+
nn.Dropout(dropout),
|
| 58 |
+
nn.Linear(embed_dim, num_classes),
|
| 59 |
+
)
|
| 60 |
+
|
| 61 |
+
def forward(self, x: torch.Tensor) -> dict[str, torch.Tensor]:
|
| 62 |
+
cnn_8, cnn_16 = self.cnn(x)
|
| 63 |
+
tr_8, tr_16 = self.transformer(x)
|
| 64 |
+
|
| 65 |
+
cnn_8 = self.cnn_proj_8(cnn_8)
|
| 66 |
+
cnn_16 = self.cnn_proj_16(cnn_16)
|
| 67 |
+
tr_8 = self.tr_proj_8(tr_8)
|
| 68 |
+
tr_16 = self.tr_proj_16(tr_16)
|
| 69 |
+
|
| 70 |
+
tr_8 = F.interpolate(tr_8, size=cnn_8.shape[-2:], mode="bilinear", align_corners=False)
|
| 71 |
+
tr_16 = F.interpolate(tr_16, size=cnn_16.shape[-2:], mode="bilinear", align_corners=False)
|
| 72 |
+
|
| 73 |
+
queries = tr_16.flatten(2).transpose(1, 2)
|
| 74 |
+
keys = cnn_16.flatten(2).transpose(1, 2)
|
| 75 |
+
values = keys
|
| 76 |
+
fused_tokens, attn_weights = self.cross_attn(queries, keys, values)
|
| 77 |
+
fused_tokens = self.fusion_proj(fused_tokens)
|
| 78 |
+
|
| 79 |
+
pooled_fused = fused_tokens.mean(dim=1)
|
| 80 |
+
pooled_cnn = F.adaptive_avg_pool2d(cnn_8, output_size=1).flatten(1)
|
| 81 |
+
pooled_tr = F.adaptive_avg_pool2d(tr_8, output_size=1).flatten(1)
|
| 82 |
+
|
| 83 |
+
logits = self.head(torch.cat([pooled_fused, pooled_cnn, pooled_tr], dim=1))
|
| 84 |
+
return {"logits": logits, "attention_weights": attn_weights}
|
dipaugnet/training/__init__.py
ADDED
|
@@ -0,0 +1 @@
|
|
|
|
|
|
|
| 1 |
+
"""Phase 1 training package."""
|
dipaugnet/training/engine.py
ADDED
|
@@ -0,0 +1,65 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Phase 1 training engine skeleton."""
|
| 2 |
+
|
| 3 |
+
from __future__ import annotations
|
| 4 |
+
|
| 5 |
+
from dataclasses import dataclass
|
| 6 |
+
from typing import Any
|
| 7 |
+
|
| 8 |
+
import torch
|
| 9 |
+
from torch.cuda.amp import GradScaler, autocast
|
| 10 |
+
|
| 11 |
+
from dipauglib.schedulers.adaptive import AdaptiveAugmentationScheduler
|
| 12 |
+
|
| 13 |
+
|
| 14 |
+
@dataclass
|
| 15 |
+
class TrainState:
|
| 16 |
+
"""Training state bundle."""
|
| 17 |
+
|
| 18 |
+
epoch: int = 0
|
| 19 |
+
best_val_f1: float = 0.0
|
| 20 |
+
patience_counter: int = 0
|
| 21 |
+
|
| 22 |
+
|
| 23 |
+
def train_one_epoch(
|
| 24 |
+
model: torch.nn.Module,
|
| 25 |
+
loader: Any,
|
| 26 |
+
optimizer: torch.optim.Optimizer,
|
| 27 |
+
criterion: torch.nn.Module,
|
| 28 |
+
device: torch.device,
|
| 29 |
+
use_amp: bool = True,
|
| 30 |
+
) -> float:
|
| 31 |
+
"""Train one epoch."""
|
| 32 |
+
|
| 33 |
+
model.train()
|
| 34 |
+
scaler = GradScaler(enabled=use_amp and device.type == "cuda")
|
| 35 |
+
losses: list[float] = []
|
| 36 |
+
|
| 37 |
+
for batch in loader:
|
| 38 |
+
images = batch["image"].to(device)
|
| 39 |
+
targets = batch["target"].to(device)
|
| 40 |
+
optimizer.zero_grad(set_to_none=True)
|
| 41 |
+
with autocast(enabled=use_amp and device.type == "cuda"):
|
| 42 |
+
logits = model(images)["logits"]
|
| 43 |
+
loss = criterion(logits, targets)
|
| 44 |
+
|
| 45 |
+
scaler.scale(loss).backward()
|
| 46 |
+
scaler.unscale_(optimizer)
|
| 47 |
+
torch.nn.utils.clip_grad_norm_(model.parameters(), max_norm=1.0)
|
| 48 |
+
scaler.step(optimizer)
|
| 49 |
+
scaler.update()
|
| 50 |
+
losses.append(float(loss.detach().cpu()))
|
| 51 |
+
|
| 52 |
+
return float(sum(losses) / max(1, len(losses)))
|
| 53 |
+
|
| 54 |
+
|
| 55 |
+
def fit_phase1(model: torch.nn.Module, optimizer: torch.optim.Optimizer, scheduler: Any | None = None) -> dict[str, Any]:
|
| 56 |
+
"""Placeholder fit entry for config-driven runners."""
|
| 57 |
+
|
| 58 |
+
aug_scheduler = AdaptiveAugmentationScheduler()
|
| 59 |
+
return {
|
| 60 |
+
"status": "scaffold_only",
|
| 61 |
+
"message": "Phase 1 training loop skeleton created.",
|
| 62 |
+
"initial_aug_intensity": aug_scheduler.intensity_at(0),
|
| 63 |
+
"mid_aug_intensity": aug_scheduler.intensity_at(50),
|
| 64 |
+
"scheduler_present": scheduler is not None,
|
| 65 |
+
}
|
dipaugnet/training/losses.py
ADDED
|
@@ -0,0 +1,33 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Classification losses for DIPAug-Net."""
|
| 2 |
+
|
| 3 |
+
from __future__ import annotations
|
| 4 |
+
|
| 5 |
+
import torch
|
| 6 |
+
from torch import nn
|
| 7 |
+
import torch.nn.functional as F
|
| 8 |
+
|
| 9 |
+
|
| 10 |
+
class WeightedFocalLoss(nn.Module):
|
| 11 |
+
"""Weighted focal loss."""
|
| 12 |
+
|
| 13 |
+
def __init__(self, class_weights: torch.Tensor | None = None, gamma: float = 2.0):
|
| 14 |
+
super().__init__()
|
| 15 |
+
self.class_weights = class_weights
|
| 16 |
+
self.gamma = gamma
|
| 17 |
+
|
| 18 |
+
def forward(self, logits: torch.Tensor, targets: torch.Tensor) -> torch.Tensor:
|
| 19 |
+
ce = F.cross_entropy(logits, targets, weight=self.class_weights, reduction="none")
|
| 20 |
+
pt = torch.exp(-ce)
|
| 21 |
+
return ((1.0 - pt) ** self.gamma * ce).mean()
|
| 22 |
+
|
| 23 |
+
|
| 24 |
+
class DIPAugNetLoss(nn.Module):
|
| 25 |
+
"""Combined focal and weighted CE loss."""
|
| 26 |
+
|
| 27 |
+
def __init__(self, class_weights: torch.Tensor | None = None):
|
| 28 |
+
super().__init__()
|
| 29 |
+
self.focal = WeightedFocalLoss(class_weights=class_weights, gamma=2.0)
|
| 30 |
+
self.ce = nn.CrossEntropyLoss(weight=class_weights)
|
| 31 |
+
|
| 32 |
+
def forward(self, logits: torch.Tensor, targets: torch.Tensor) -> torch.Tensor:
|
| 33 |
+
return 0.6 * self.focal(logits, targets) + 0.4 * self.ce(logits, targets)
|
dipaugsevernet/__init__.py
ADDED
|
@@ -0,0 +1 @@
|
|
|
|
|
|
|
| 1 |
+
"""Phase 2 DIPAug-SeverNet package."""
|
dipaugsevernet/evaluation/__init__.py
ADDED
|
@@ -0,0 +1 @@
|
|
|
|
|
|
|
| 1 |
+
"""Phase 2 evaluation package."""
|
dipaugsevernet/evaluation/metrics.py
ADDED
|
@@ -0,0 +1,33 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Evaluation metrics for segmentation and severity."""
|
| 2 |
+
|
| 3 |
+
from __future__ import annotations
|
| 4 |
+
|
| 5 |
+
import numpy as np
|
| 6 |
+
from scipy.stats import pearsonr
|
| 7 |
+
from sklearn.metrics import cohen_kappa_score, mean_absolute_error, mean_squared_error
|
| 8 |
+
|
| 9 |
+
|
| 10 |
+
def segmentation_iou(pred_mask: np.ndarray, true_mask: np.ndarray, eps: float = 1e-6) -> float:
|
| 11 |
+
"""Compute binary IoU."""
|
| 12 |
+
|
| 13 |
+
pred = pred_mask.astype(bool)
|
| 14 |
+
truth = true_mask.astype(bool)
|
| 15 |
+
intersection = np.logical_and(pred, truth).sum()
|
| 16 |
+
union = np.logical_or(pred, truth).sum()
|
| 17 |
+
return float((intersection + eps) / (union + eps))
|
| 18 |
+
|
| 19 |
+
|
| 20 |
+
def severity_summary(y_true: np.ndarray, y_pred: np.ndarray) -> dict[str, float]:
|
| 21 |
+
"""Compute severity regression metrics."""
|
| 22 |
+
|
| 23 |
+
return {
|
| 24 |
+
"pearson_r": float(pearsonr(y_true, y_pred).statistic) if len(y_true) > 1 else 0.0,
|
| 25 |
+
"mae": float(mean_absolute_error(y_true, y_pred)),
|
| 26 |
+
"rmse": float(np.sqrt(mean_squared_error(y_true, y_pred))),
|
| 27 |
+
}
|
| 28 |
+
|
| 29 |
+
|
| 30 |
+
def ordinal_summary(y_true: np.ndarray, y_pred: np.ndarray) -> dict[str, float]:
|
| 31 |
+
"""Compute ordinal metrics."""
|
| 32 |
+
|
| 33 |
+
return {"qwk": float(cohen_kappa_score(y_true, y_pred, weights="quadratic"))}
|
dipaugsevernet/models/__init__.py
ADDED
|
@@ -0,0 +1,6 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Phase 2 models."""
|
| 2 |
+
|
| 3 |
+
from .dgsm import DGSM
|
| 4 |
+
from .dipaugsevernet import DIPAugSeverNet
|
| 5 |
+
|
| 6 |
+
__all__ = ["DGSM", "DIPAugSeverNet"]
|
dipaugsevernet/models/dgsm.py
ADDED
|
@@ -0,0 +1,46 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""DIP-guided saliency module."""
|
| 2 |
+
|
| 3 |
+
from __future__ import annotations
|
| 4 |
+
|
| 5 |
+
import cv2
|
| 6 |
+
import numpy as np
|
| 7 |
+
|
| 8 |
+
|
| 9 |
+
class DGSM:
|
| 10 |
+
"""Non-trainable OpenCV-based saliency generator."""
|
| 11 |
+
|
| 12 |
+
def __init__(self, blur_sigma: float = 15.0, min_component_area: int = 50):
|
| 13 |
+
self.blur_sigma = blur_sigma
|
| 14 |
+
self.min_component_area = min_component_area
|
| 15 |
+
|
| 16 |
+
def __call__(self, image_rgb: np.ndarray) -> np.ndarray:
|
| 17 |
+
image = image_rgb.astype(np.uint8)
|
| 18 |
+
lab = cv2.cvtColor(image, cv2.COLOR_RGB2LAB)
|
| 19 |
+
hsv = cv2.cvtColor(image, cv2.COLOR_RGB2HSV)
|
| 20 |
+
|
| 21 |
+
a_channel = lab[..., 1]
|
| 22 |
+
s_channel = hsv[..., 1]
|
| 23 |
+
|
| 24 |
+
mask_a = cv2.adaptiveThreshold(a_channel, 255, cv2.ADAPTIVE_THRESH_GAUSSIAN_C, cv2.THRESH_BINARY, 31, -2)
|
| 25 |
+
mask_s = cv2.adaptiveThreshold(s_channel, 255, cv2.ADAPTIVE_THRESH_GAUSSIAN_C, cv2.THRESH_BINARY_INV, 31, 2)
|
| 26 |
+
mask = cv2.bitwise_or(mask_a, mask_s)
|
| 27 |
+
|
| 28 |
+
kernel = np.ones((7, 7), dtype=np.uint8)
|
| 29 |
+
mask = cv2.morphologyEx(mask, cv2.MORPH_CLOSE, kernel)
|
| 30 |
+
|
| 31 |
+
num_labels, labels, stats, _ = cv2.connectedComponentsWithStats(mask, connectivity=8)
|
| 32 |
+
filtered = np.zeros_like(mask)
|
| 33 |
+
for component_idx in range(1, num_labels):
|
| 34 |
+
area = stats[component_idx, cv2.CC_STAT_AREA]
|
| 35 |
+
if area >= self.min_component_area:
|
| 36 |
+
filtered[labels == component_idx] = 255
|
| 37 |
+
|
| 38 |
+
saliency = cv2.GaussianBlur(filtered.astype(np.float32) / 255.0, (0, 0), sigmaX=self.blur_sigma, sigmaY=self.blur_sigma)
|
| 39 |
+
return np.clip(saliency, 0.0, 1.0)
|
| 40 |
+
|
| 41 |
+
def concatenate_channel(self, image_rgb: np.ndarray) -> np.ndarray:
|
| 42 |
+
"""Append saliency as 4th channel."""
|
| 43 |
+
|
| 44 |
+
saliency = self(image_rgb)[..., None]
|
| 45 |
+
image = image_rgb.astype(np.float32) / 255.0
|
| 46 |
+
return np.concatenate([image, saliency], axis=-1)
|
dipaugsevernet/models/dipaugsevernet.py
ADDED
|
@@ -0,0 +1,96 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""DIPAug-SeverNet architecture scaffold."""
|
| 2 |
+
|
| 3 |
+
from __future__ import annotations
|
| 4 |
+
|
| 5 |
+
import torch
|
| 6 |
+
from torch import nn
|
| 7 |
+
import torch.nn.functional as F
|
| 8 |
+
import timm
|
| 9 |
+
|
| 10 |
+
|
| 11 |
+
class DecoderBlock(nn.Module):
|
| 12 |
+
"""Simple U-Net style decoder block."""
|
| 13 |
+
|
| 14 |
+
def __init__(self, in_channels: int, skip_channels: int, out_channels: int):
|
| 15 |
+
super().__init__()
|
| 16 |
+
self.block = nn.Sequential(
|
| 17 |
+
nn.Conv2d(in_channels + skip_channels, out_channels, kernel_size=3, padding=1, bias=False),
|
| 18 |
+
nn.BatchNorm2d(out_channels),
|
| 19 |
+
nn.GELU(),
|
| 20 |
+
nn.Conv2d(out_channels, out_channels, kernel_size=3, padding=1, bias=False),
|
| 21 |
+
nn.BatchNorm2d(out_channels),
|
| 22 |
+
nn.GELU(),
|
| 23 |
+
)
|
| 24 |
+
|
| 25 |
+
def forward(self, x: torch.Tensor, skip: torch.Tensor) -> torch.Tensor:
|
| 26 |
+
x = F.interpolate(x, size=skip.shape[-2:], mode="bilinear", align_corners=False)
|
| 27 |
+
return self.block(torch.cat([x, skip], dim=1))
|
| 28 |
+
|
| 29 |
+
|
| 30 |
+
class SeverityHead(nn.Module):
|
| 31 |
+
"""Severity regression and ordinal head."""
|
| 32 |
+
|
| 33 |
+
def __init__(self, in_features: int):
|
| 34 |
+
super().__init__()
|
| 35 |
+
self.backbone = nn.Sequential(
|
| 36 |
+
nn.Linear(in_features, 256),
|
| 37 |
+
nn.GELU(),
|
| 38 |
+
nn.Dropout(0.2),
|
| 39 |
+
nn.Linear(256, 64),
|
| 40 |
+
nn.GELU(),
|
| 41 |
+
)
|
| 42 |
+
self.regression = nn.Linear(64, 1)
|
| 43 |
+
self.ordinal = nn.Linear(64, 5)
|
| 44 |
+
|
| 45 |
+
def forward(self, x: torch.Tensor) -> dict[str, torch.Tensor]:
|
| 46 |
+
features = self.backbone(x)
|
| 47 |
+
return {
|
| 48 |
+
"severity_score": torch.sigmoid(self.regression(features)),
|
| 49 |
+
"severity_ordinal_logits": self.ordinal(features),
|
| 50 |
+
}
|
| 51 |
+
|
| 52 |
+
|
| 53 |
+
class DIPAugSeverNet(nn.Module):
|
| 54 |
+
"""Shared-encoder segmentation and severity model scaffold."""
|
| 55 |
+
|
| 56 |
+
def __init__(self, num_classes: int = 11):
|
| 57 |
+
super().__init__()
|
| 58 |
+
self.encoder = timm.create_model(
|
| 59 |
+
"efficientnet_b4.ra2_in1k",
|
| 60 |
+
pretrained=True,
|
| 61 |
+
in_chans=4,
|
| 62 |
+
features_only=True,
|
| 63 |
+
out_indices=(1, 2, 3, 4),
|
| 64 |
+
)
|
| 65 |
+
|
| 66 |
+
channels = self.encoder.feature_info.channels()
|
| 67 |
+
self.decoder3 = DecoderBlock(channels[3], channels[2], 256)
|
| 68 |
+
self.decoder2 = DecoderBlock(256, channels[1], 128)
|
| 69 |
+
self.decoder1 = DecoderBlock(128, channels[0], 64)
|
| 70 |
+
self.segmentation_head = nn.Conv2d(64, 1, kernel_size=1)
|
| 71 |
+
|
| 72 |
+
encoder_out_channels = channels[-1]
|
| 73 |
+
self.classifier = nn.Linear(encoder_out_channels, num_classes)
|
| 74 |
+
self.severity_head = SeverityHead(encoder_out_channels * 2)
|
| 75 |
+
self.log_sigmas = nn.Parameter(torch.zeros(4))
|
| 76 |
+
|
| 77 |
+
def forward(self, x: torch.Tensor) -> dict[str, torch.Tensor]:
|
| 78 |
+
f1, f2, f3, f4 = self.encoder(x)
|
| 79 |
+
d3 = self.decoder3(f4, f3)
|
| 80 |
+
d2 = self.decoder2(d3, f2)
|
| 81 |
+
d1 = self.decoder1(d2, f1)
|
| 82 |
+
mask_logits = self.segmentation_head(d1)
|
| 83 |
+
mask_pred = torch.sigmoid(F.interpolate(mask_logits, size=x.shape[-2:], mode="bilinear", align_corners=False))
|
| 84 |
+
|
| 85 |
+
encoded = F.adaptive_avg_pool2d(f4, output_size=1).flatten(1)
|
| 86 |
+
lesion_weighted = F.adaptive_avg_pool2d(f4 * F.interpolate(mask_pred, size=f4.shape[-2:], mode="bilinear", align_corners=False), output_size=1).flatten(1)
|
| 87 |
+
severity_features = torch.cat([encoded, lesion_weighted], dim=1)
|
| 88 |
+
|
| 89 |
+
outputs = {
|
| 90 |
+
"mask_logits": mask_logits,
|
| 91 |
+
"mask_pred": mask_pred,
|
| 92 |
+
"class_logits": self.classifier(encoded),
|
| 93 |
+
"log_sigmas": self.log_sigmas,
|
| 94 |
+
}
|
| 95 |
+
outputs.update(self.severity_head(severity_features))
|
| 96 |
+
return outputs
|
dipaugsevernet/training/__init__.py
ADDED
|
@@ -0,0 +1 @@
|
|
|
|
|
|
|
| 1 |
+
"""Phase 2 training package."""
|
dipaugsevernet/training/engine.py
ADDED
|
@@ -0,0 +1,15 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Phase 2 training engine skeleton."""
|
| 2 |
+
|
| 3 |
+
from __future__ import annotations
|
| 4 |
+
|
| 5 |
+
from typing import Any
|
| 6 |
+
|
| 7 |
+
|
| 8 |
+
def fit_phase2(model: Any) -> dict[str, str]:
|
| 9 |
+
"""Placeholder entry for phase 2 training."""
|
| 10 |
+
|
| 11 |
+
return {
|
| 12 |
+
"status": "scaffold_only",
|
| 13 |
+
"message": "Phase 2 multi-task training skeleton created.",
|
| 14 |
+
"model_name": type(model).__name__,
|
| 15 |
+
}
|
dipaugsevernet/training/losses.py
ADDED
|
@@ -0,0 +1,39 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Loss functions for DIPAug-SeverNet."""
|
| 2 |
+
|
| 3 |
+
from __future__ import annotations
|
| 4 |
+
|
| 5 |
+
import torch
|
| 6 |
+
from torch import nn
|
| 7 |
+
import torch.nn.functional as F
|
| 8 |
+
|
| 9 |
+
|
| 10 |
+
def dice_loss(logits: torch.Tensor, targets: torch.Tensor, eps: float = 1e-6) -> torch.Tensor:
|
| 11 |
+
"""Binary dice loss on logits."""
|
| 12 |
+
|
| 13 |
+
probs = torch.sigmoid(logits)
|
| 14 |
+
intersection = (probs * targets).sum(dim=(1, 2, 3))
|
| 15 |
+
union = probs.sum(dim=(1, 2, 3)) + targets.sum(dim=(1, 2, 3))
|
| 16 |
+
dice = (2 * intersection + eps) / (union + eps)
|
| 17 |
+
return 1.0 - dice.mean()
|
| 18 |
+
|
| 19 |
+
|
| 20 |
+
def corn_ordinal_loss(logits: torch.Tensor, labels: torch.Tensor) -> torch.Tensor:
|
| 21 |
+
"""Simple CORN-style cumulative ordinal loss approximation."""
|
| 22 |
+
|
| 23 |
+
losses = []
|
| 24 |
+
for threshold_idx in range(logits.shape[1]):
|
| 25 |
+
targets = (labels > threshold_idx).float()
|
| 26 |
+
losses.append(F.binary_cross_entropy_with_logits(logits[:, threshold_idx], targets))
|
| 27 |
+
return torch.stack(losses).mean()
|
| 28 |
+
|
| 29 |
+
|
| 30 |
+
class UncertaintyWeightedMultiTaskLoss(nn.Module):
|
| 31 |
+
"""Kendall uncertainty-weighted multi-task loss."""
|
| 32 |
+
|
| 33 |
+
def forward(self, losses: dict[str, torch.Tensor], log_sigmas: torch.Tensor) -> torch.Tensor:
|
| 34 |
+
ordered = [losses["segmentation"], losses["classification"], losses["severity_regression"], losses["severity_ordinal"]]
|
| 35 |
+
total = torch.zeros(1, device=log_sigmas.device, dtype=log_sigmas.dtype)
|
| 36 |
+
for idx, loss in enumerate(ordered):
|
| 37 |
+
sigma_sq = torch.exp(2.0 * log_sigmas[idx])
|
| 38 |
+
total = total + (0.5 / sigma_sq) * loss + log_sigmas[idx]
|
| 39 |
+
return total.squeeze(0)
|
figures/README.md
ADDED
|
@@ -0,0 +1,9 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Figures
|
| 2 |
+
|
| 3 |
+
This directory is reserved for:
|
| 4 |
+
|
| 5 |
+
- QC grids
|
| 6 |
+
- architecture diagrams
|
| 7 |
+
- confusion matrices
|
| 8 |
+
- CEDB charts
|
| 9 |
+
- Grad-CAM++ overlays
|
notebooks/README.md
ADDED
|
@@ -0,0 +1,8 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Notebooks
|
| 2 |
+
|
| 3 |
+
Use this directory for:
|
| 4 |
+
|
| 5 |
+
- augmentation QC grids
|
| 6 |
+
- CEDB result analysis
|
| 7 |
+
- Grad-CAM++ inspection
|
| 8 |
+
- severity error analysis
|
requirements.txt
CHANGED
|
@@ -1,2 +1,22 @@
|
|
| 1 |
-
gradio
|
| 2 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
gradio==6.10.0
|
| 2 |
+
albumentations>=1.4.18
|
| 3 |
+
numpy>=1.26.0
|
| 4 |
+
opencv-python>=4.10.0
|
| 5 |
+
Pillow>=10.4.0
|
| 6 |
+
PyYAML>=6.0.2
|
| 7 |
+
scikit-image>=0.24.0
|
| 8 |
+
scikit-learn>=1.5.2
|
| 9 |
+
scipy>=1.14.1
|
| 10 |
+
matplotlib>=3.9.2
|
| 11 |
+
seaborn>=0.13.2
|
| 12 |
+
pandas>=2.2.3
|
| 13 |
+
pytest>=8.3.3
|
| 14 |
+
pytest-cov>=5.0.0
|
| 15 |
+
timm>=1.0.11
|
| 16 |
+
torch>=2.4.0
|
| 17 |
+
torchvision>=0.19.0
|
| 18 |
+
segmentation-models-pytorch>=0.3.4
|
| 19 |
+
lightly>=1.5.14
|
| 20 |
+
wandb>=0.18.3
|
| 21 |
+
pytorch-grad-cam>=1.5.4
|
| 22 |
+
coral-pytorch>=1.4.0
|
results/README.md
ADDED
|
@@ -0,0 +1,8 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Results
|
| 2 |
+
|
| 3 |
+
This directory is reserved for:
|
| 4 |
+
|
| 5 |
+
- CSV metrics
|
| 6 |
+
- checkpoints
|
| 7 |
+
- manifests
|
| 8 |
+
- ablation summaries
|
scripts/evaluate_phase1.py
ADDED
|
@@ -0,0 +1,5 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Phase 1 evaluation entrypoint."""
|
| 2 |
+
|
| 3 |
+
from __future__ import annotations
|
| 4 |
+
|
| 5 |
+
print("Evaluation scaffold for Phase 1 is ready.")
|
scripts/evaluate_phase2.py
ADDED
|
@@ -0,0 +1,5 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Phase 2 evaluation entrypoint."""
|
| 2 |
+
|
| 3 |
+
from __future__ import annotations
|
| 4 |
+
|
| 5 |
+
print("Evaluation scaffold for Phase 2 is ready.")
|