IJmond Smoke Segmentation — Pixel-wise Soft Weighting for Noisy Pseudo-labels
Master's Thesis · Vrije Universiteit Amsterdam · 2026
Author: Wang Mingjia | Supervisor: Yen-Chia Hsu
Overview
This repository contains all model checkpoints, experiment notebooks, and results from a thesis studying whether pixel-wise soft-weighting can improve smoke segmentation when supervision comes from noisy SAM-generated pseudo-labels.
Short answer: No — under multi-seed evaluation, no weighting variant reliably outperforms the plain pseudo-label baseline (M2). All six methods achieve 0.164–0.185 mIoU with overlapping standard deviations. The single-run apparent range (0.060–0.220 mIoU) was an artefact of random initialisation, not a reliable property of the loss designs.
Architecture
| Component | Detail |
|---|---|
| Model | DeepLabV3+ |
| Encoder | MobileNetV2 (depthwise separable convolutions) |
| Parameters | ~5.8 M |
| Input size | 400 × 400 px |
| Output | 2-class Softmax (smoke / background) |
| Pre-training | Smoke5K dataset (clean pixel labels) |
| Fine-tuning | IJmond pseudo-labels (167 images, SAM-generated) |
| Test set | IJmond human annotations (2,074 images, never seen during training) |
| Optimizer | Adam · lr=1e-4 · wd=1e-4 · cosine annealing · 30 epochs · batch 8 |
Methods
Six loss formulations are compared. All share the same DeepLabV3+ network at inference; differences are only in how per-pixel cross-entropy losses are aggregated during training.
Notation
- $P_{\text{label}}(i)$ — Softmax probability of the pseudo-labeled class at pixel $i$ (detached from the gradient graph)
- $P_{\text{local}}(i)$ — 5×5 average-pool of $P_{\text{label}}$ (spatial neighbourhood mean)
Method table
| ID | Name | Weight $W(i)$ | Loss formula |
|---|---|---|---|
| M2 | Plain pseudo-labeling (baseline) | 1 | $\frac{1}{N}\sum_i \ell_i$ |
| M3 | Hard confidence threshold | $\mathbf{1}[\max_c P_c(i) \geq 0.8]$ | normalised masked CE |
| M1-Conf | Confidence only | $P_{\text{label}}(i)$ | $\frac{\sum W_i \ell_i}{\sum W_i + \varepsilon}$ |
| M1-Cons | Consistency only | $1 - |P_{\text{label}}(i) - P_{\text{local}}(i)|$ | same |
| M1-Both | Confidence × Consistency | $P_{\text{label}}(i) \times (1 - |P_{\text{label}}(i) - P_{\text{local}}(i)|)$ | same |
| M4 | Mean Teacher (EMA) | — | $\mathcal{L}_{\text{CE}} + \mathcal{L}_{\text{MSE}}(\text{student}, \text{teacher})$, $\alpha=0.99$ |
Key design choice: $P_{\text{label}}$ is detached from the computation graph before being used as a weight. Without
detach, the model can minimise the loss by driving $P_{\text{label}} \to 0$ (zeroing the weight) instead of improving predictions.
Results
Primary evidence — Multi-seed evaluation (3 seeds × 6 methods)
Mean ± 1 std over 3 independent training runs (seeds 0, 1, 2). Error bars show seed standard deviation. Threshold = 0.5.
| Method | mIoU | Dice | Precision | Recall |
|---|---|---|---|---|
| M2 (Plain Pseudo) | 0.1847 ± 0.019 | 0.3116 ± 0.027 | 0.4054 ± 0.026 | 0.2564 ± 0.046 |
| M3 (Hard Threshold 0.8) | 0.1810 ± 0.005 | 0.3065 ± 0.007 | 0.4146 ± 0.033 | 0.2445 ± 0.019 |
| M1-Conf (Confidence only) | 0.1843 ± 0.024 | 0.3107 ± 0.035 | 0.4471 ± 0.036 | 0.2382 ± 0.031 |
| M1-Cons (Consistency only) | 0.1780 ± 0.028 | 0.3016 ± 0.040 | 0.4073 ± 0.027 | 0.2447 ± 0.060 |
| M1-Both (Conf × Cons) | 0.1726 ± 0.023 | 0.2940 ± 0.033 | 0.4826 ± 0.091 | 0.2141 ± 0.030 |
| M4 (Mean Teacher) | 0.1637 ± 0.025 | 0.2808 ± 0.037 | 0.4810 ± 0.045 | 0.2023 ± 0.045 |
No weighting variant reliably outperforms M2. All differences lie within seed standard deviation.
Notable cross-seed patterns that are reliable:
- M3 has the lowest seed variance (σ = 0.005): binary keep/discard is more robust to initialisation than soft weighting.
- M4 has the highest mean Precision (0.481) but the lowest Recall (0.202) across all three seeds: the EMA teacher acts as a conservative regulariser that suppresses uncertain detections.
Per-seed breakdown
Click to expand full per-seed mIoU table
| Method | Seed 0 | Seed 1 | Seed 2 | Mean | Std |
|---|---|---|---|---|---|
| M2 | 0.2057 | 0.1800 | 0.1685 | 0.1847 | 0.019 |
| M3 | 0.1836 | 0.1756 | 0.1837 | 0.1810 | 0.005 |
| M1-Conf | 0.1992 | 0.1561 | 0.1974 | 0.1843 | 0.024 |
| M1-Cons | 0.2060 | 0.1776 | 0.1506 | 0.1780 | 0.028 |
| M1-Both | 0.1985 | 0.1565 | 0.1628 | 0.1726 | 0.023 |
| M4 | 0.1876 | 0.1652 | 0.1383 | 0.1637 | 0.025 |
Full metrics (mIoU, Dice, Precision, Recall) per seed: results/multiseed/per_seed_results.csv
Threshold sensitivity
Threshold sweep 0.3 → 0.7 on standalone checkpoints (single run). Method rankings under the standalone run are not reproduced in multi-seed evaluation; the sweep confirms the chosen threshold of 0.5 is not a cherry-picked operating point.
Label budget (RQ2 sub-experiment)
Each method re-trained at 25 / 50 / 75 / 100% of the 167-image training pool (single seed, fixed split).
| Method | 25% (41 imgs) | 50% (83 imgs) | 75% (125 imgs) | 100% (167 imgs) |
|---|---|---|---|---|
| M2 | 0.139 | 0.153 | 0.189 | 0.180 |
| M3 | 0.151 | 0.153 | 0.176 | 0.142 |
| M1-Conf | 0.088 | 0.089 | 0.133 | 0.153 |
| M1-Cons | 0.136 | 0.152 | 0.183 | 0.178 |
| M1-Both | 0.088 | 0.089 | 0.130 | 0.162 |
M1-Cons tracks M2 closely across all budget levels. M1-Conf and M1-Both are most unstable at very small budgets (normalised-sum denominator can collapse when batch contains few confident pixels).
Deployment Benchmarks (RQ3)
All M1 variants and M2/M3 share the same deployed network — weighting only affects the training loss, not the model architecture. M4 also deploys as a single network (student only).
Inference latency
| Batch size | CPU latency | CPU FPS | GPU latency | GPU FPS |
|---|---|---|---|---|
| 1 | 152 ms | 6.5 | 10 ms | 99.9 |
| 4 | 843 ms | 4.7 | 22 ms | 178 |
| 8 | 1671 ms | 4.8 | 45 ms | 179 |
5 warm-up + 20 timed runs. CPU: Apple M-series. GPU: Tesla P100.
Training-step overhead vs M2
| Method | CPU overhead | GPU overhead |
|---|---|---|
| M2 | — | — |
| M3 | −8.1% | +0.4% |
| M1-Conf | −6.9% | +1.5% |
| M1-Cons | ~−1.0% | ~+1.9% |
| M1-Both | −0.8% | +1.9% |
| M4 | +24.5% | +29.3% |
M4's overhead comes from a second forward pass (teacher) + EMA parameter update every step. CPU fluctuations for M3/M1-Conf are within timing noise.
Qualitative Analysis
Three representative prediction patterns from the 2,074-image test set.
Overlays show raw Softmax smoke probability; pixels with P < 0.05 are hidden.
Precision-contrast example — M2 over-predicts, weighting variants stay focused
M2 (purple) generates a large false-positive blob covering industrial structures. M1-Conf (red), M1-Cons (orange), and M1-Both (teal) assign substantially lower probabilities to the same non-smoke region.
Positive-detection example — all methods agree on genuine smoke
All four methods consistently localise the chimney plume with similar spatial extent. Weighting variants do not suppress genuine smoke signals when pseudo-labels are unambiguous.
Consistency-effect example — M1-Conf and M1-Cons diverge
M1-Conf (red) assigns moderate probability to a diffuse region near the chimney top. M1-Cons (orange) suppresses this prediction because the local 5×5 neighbourhood average deviates from the pixel's own confidence — illustrating how consistency weighting can reduce ambiguous false positives but may also suppress genuine detections.
40 full-resolution figures (fp / tp / cons categories) are in
figures/qualitative/.
How to Load a Checkpoint
import torch
import segmentation_models_pytorch as smp
def load_model(ckpt_path, device="cpu"):
model = smp.DeepLabV3Plus(
encoder_name="mobilenet_v2",
encoder_weights=None,
in_channels=3,
classes=2,
).to(device)
model.load_state_dict(torch.load(ckpt_path, map_location=device))
model.eval()
return model
# Example: load M2 multi-seed seed-0 checkpoint
from huggingface_hub import hf_hub_download
ckpt = hf_hub_download(
repo_id="Minggg0518/ijmond-smoke-segmentation",
filename="checkpoints/multiseed/M2_seed0_best.pth",
)
model = load_model(ckpt)
Run inference
import numpy as np
import torch
import torch.nn.functional as F
# Load a 400×400 image (.npy format as used in IJmond dataset)
image_np = np.load("your_image.npy").astype(np.float32) # H×W×3
t = torch.from_numpy(image_np).permute(2, 0, 1).unsqueeze(0) / 255.0
t = F.interpolate(t, size=(400, 400), mode="bilinear", align_corners=False)
with torch.no_grad():
logits = model(t) # 1×2×400×400
prob_smoke = F.softmax(logits, dim=1)[0, 1] # 400×400
pred_mask = (prob_smoke > 0.5).int() # binary, threshold=0.5
Repository Structure
├── checkpoints/
│ ├── multiseed/ # 18 checkpoints — 6 methods × 3 seeds (PRIMARY)
│ ├── standalone/ # 6 standalone reference checkpoints
│ └── pretrained_smoke5k.pth
├── experiments/
│ ├── 00_handbook.ipynb # Unified experiment handbook (all chapters)
│ ├── 01_training.ipynb
│ ├── 02_qualitative_analysis.ipynb
│ ├── 03_effectiveness_evaluation.ipynb
│ ├── 04_label_budget_analysis.ipynb
│ └── 05_deployability_cost.ipynb
├── results/
│ ├── multiseed/ # per_seed_results.csv · summary_mean_std.csv
│ ├── standalone/ # standalone evaluation CSVs + figures
│ ├── budget/ # label budget CSVs + figures
│ └── cost/ # inference + training overhead CSVs
└── figures/
├── qual_precision_contrast.png
├── qual_positive_detection.png
├── qual_consistency_effect.png
└── qualitative/ # 40 full-resolution comparison figures
Key Takeaways
Multi-seed evaluation is essential. A single training run produced apparent extremes (M1-Cons 0.220, M1-Conf 0.060) that did not reproduce across seeds. The true cross-seed range is only ~0.012 mIoU.
No weighting variant is reliably better than plain pseudo-labeling (M2) under the tested conditions (167-image fine-tuning set, Smoke5K pre-training, 30-epoch cosine annealing).
M1-Conf has a latent confirmation-bias failure mode. When the model drifts background-biased, $W = P_{\text{label}}$ gates out the corrective gradients, and the normalised-sum amplification accelerates convergence to a degenerate minimum. This occurred once (mIoU 0.060) but did not reproduce in 3-seed evaluation.
M4 (Mean Teacher) costs +25–30% training compute per step with no consistent accuracy gain — a poor accuracy-cost trade-off on this dataset.
All M1 variants are free at inference (same model, same speed). They are safe to try even when compute budget is tight.
Citation
Wang Mingjia (2026). Pixel-wise Soft Weighting for Smoke Segmentation under
Noisy Pseudo-labels: A Multi-seed Ablation on IJmond Industrial Data.
Master's Thesis, Vrije Universiteit Amsterdam.
Supervisor: Yen-Chia Hsu.