Update README.md
Browse files
README.md
CHANGED
|
@@ -1,3 +1,135 @@
|
|
| 1 |
-
---
|
| 2 |
-
license: cc0-1.0
|
| 3 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: cc0-1.0
|
| 3 |
+
library_name: pytorch
|
| 4 |
+
pipeline_tag: image-segmentation
|
| 5 |
+
tags:
|
| 6 |
+
- remote-sensing
|
| 7 |
+
- earth-observation
|
| 8 |
+
- cloud-detection
|
| 9 |
+
- cloud-segmentation
|
| 10 |
+
- satellite-imagery
|
| 11 |
+
- spot-vgt
|
| 12 |
+
- proba-v
|
| 13 |
+
- semantic-segmentation
|
| 14 |
+
- pytorch
|
| 15 |
+
---
|
| 16 |
+
|
| 17 |
+
# ☁️ FDR4VGT-CLOUD
|
| 18 |
+
|
| 19 |
+
<p align="center">
|
| 20 |
+
<img src="https://raw.githubusercontent.com/JulioContrerasH/DocProbaSpot/refs/heads/main/assets/logo_proba_spot.png" alt="FDR4VGT-CLOUD" width="90%"/>
|
| 21 |
+
</p>
|
| 22 |
+
|
| 23 |
+
Official model release accompanying the paper:
|
| 24 |
+
|
| 25 |
+
> **A multisensor deep learning framework for robust cloud segmentation in SPOT-VGT and Proba-V**
|
| 26 |
+
> Julio Contreras, Cesar Aybar, Luis Gómez-Chova
|
| 27 |
+
> *IEEE Geoscience and Remote Sensing Letters*, 2026.
|
| 28 |
+
|
| 29 |
+
This model is the **operational cloud masking algorithm** selected for the **ESA FDR4VGT** archive reprocessing, delivering consistent cloud detection across the full SPOT-VGT (VGT1 1998–2003, VGT2 2002–2014) and Proba-V (2013–2020) record — a single sensor-agnostic model for the three missions.
|
| 30 |
+
|
| 31 |
+
---
|
| 32 |
+
|
| 33 |
+
## ✨ Overview
|
| 34 |
+
|
| 35 |
+
- **Architecture:** Hybrid **DeepLabV3+ (MobileNetV2 backbone) + PixelWise MLP** (`PW-DL3+`)
|
| 36 |
+
- **Input:** 4 Top-of-Atmosphere reflectance bands (Blue, Red, NIR, SWIR) — sensor-agnostic
|
| 37 |
+
- **Supported sensors:** SPOT-VGT1, SPOT-VGT2, Proba-V
|
| 38 |
+
- **Input shape:** `[B, 4, 512, 512]`
|
| 39 |
+
- **Parameters:** 12.65M (57.29 MB)
|
| 40 |
+
- **Training:** Weak-to-strong supervision — large-scale pre-training on 3,647 weakly-labeled scenes, followed by fine-tuning on 109 hand-annotated hard-example scenes.
|
| 41 |
+
|
| 42 |
+
---
|
| 43 |
+
|
| 44 |
+
## 🚀 Quick start
|
| 45 |
+
|
| 46 |
+
### Installation
|
| 47 |
+
|
| 48 |
+
```bash
|
| 49 |
+
pip install mlstac rasterio torch==2.5.1
|
| 50 |
+
```
|
| 51 |
+
|
| 52 |
+
### Inference
|
| 53 |
+
|
| 54 |
+
```python
|
| 55 |
+
import torch
|
| 56 |
+
import mlstac
|
| 57 |
+
import rasterio as rio
|
| 58 |
+
|
| 59 |
+
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
|
| 60 |
+
|
| 61 |
+
# 1. Load the model
|
| 62 |
+
framework = mlstac.download(
|
| 63 |
+
file="https://huggingface.co/isp-uv-es/FDR4VGT-CLOUD/resolve/main/single/multisensor_single_1dpwdeeplabv3.json",
|
| 64 |
+
output_dir="FDR4VGT/single",
|
| 65 |
+
)
|
| 66 |
+
model = framework.model
|
| 67 |
+
|
| 68 |
+
# 2. Load a 4-band image (Blue, Red, NIR, SWIR)
|
| 69 |
+
with rio.open("https://huggingface.co/isp-uv-es/FDR4VGT-CLOUD/resolve/main/ensemble/rgb.tif") as src:
|
| 70 |
+
image = src.read()
|
| 71 |
+
|
| 72 |
+
# 3. Run large-scene inference (sliding window + Hann blending)
|
| 73 |
+
prob = framework.predict_large(
|
| 74 |
+
image=image,
|
| 75 |
+
model=model,
|
| 76 |
+
device=device,
|
| 77 |
+
batch_size=8, # increase on GPU to speed up; lower on CPU
|
| 78 |
+
num_workers=8,
|
| 79 |
+
nodata=0, # pixel value treated as invalid/padding
|
| 80 |
+
)
|
| 81 |
+
|
| 82 |
+
# 4. Binarize with the operational threshold
|
| 83 |
+
cloud_mask = (prob.squeeze() > 0.5).astype("uint8")
|
| 84 |
+
```
|
| 85 |
+
|
| 86 |
+
> The binarization threshold (default `0.5`) can be tuned per use case; the paper uses the F₂-optimal threshold on the validation set.
|
| 87 |
+
|
| 88 |
+
---
|
| 89 |
+
|
| 90 |
+
## 📊 Performance
|
| 91 |
+
|
| 92 |
+
Results on the manually-annotated test set (`PW-DL3+`, Multi-FT strategy) — mean over scenes:
|
| 93 |
+
|
| 94 |
+
| Sensor | F₂ | IoU | κ |
|
| 95 |
+
|-----------|:------:|:------:|:------:|
|
| 96 |
+
| Proba-V | 0.891 | 0.842 | 0.808 |
|
| 97 |
+
| SPOT-VGT | 0.949 | 0.898 | 0.829 |
|
| 98 |
+
|
| 99 |
+
The model substantially outperforms the legacy BS1 (physical thresholds) and BS2 (pixel-wise MLP) baselines on both sensors, with the largest gain on SPOT-VGT (ΔF₂ = +0.090 over BS1). Temporal analysis across the 1998–2020 archive shows no statistically significant discontinuity at the VGT→Proba-V transition (Mann-Whitney U, p > 0.05), in contrast to the legacy record.
|
| 100 |
+
|
| 101 |
+
---
|
| 102 |
+
|
| 103 |
+
## 📁 Repository layout
|
| 104 |
+
|
| 105 |
+
| Path | Description |
|
| 106 |
+
|------|-------------|
|
| 107 |
+
| `single/multisensor_single_1dpwdeeplabv3.json` | Operational single-model weights (`PW-DL3+`) |
|
| 108 |
+
| `ensemble/rgb.tif` | Example test scene (4-band TOA reflectance) |
|
| 109 |
+
|
| 110 |
+
---
|
| 111 |
+
|
| 112 |
+
## 📄 Citation
|
| 113 |
+
|
| 114 |
+
If you use this model, please cite:
|
| 115 |
+
|
| 116 |
+
```bibtex
|
| 117 |
+
@article{contreras2026fdr4vgt,
|
| 118 |
+
title = {A multisensor deep learning framework for robust cloud segmentation in SPOT-VGT and Proba-V},
|
| 119 |
+
author = {Contreras, Julio and Aybar, Cesar and G{\'o}mez-Chova, Luis},
|
| 120 |
+
journal = {IEEE Geoscience and Remote Sensing Letters},
|
| 121 |
+
year = {2026},
|
| 122 |
+
}
|
| 123 |
+
```
|
| 124 |
+
|
| 125 |
+
---
|
| 126 |
+
|
| 127 |
+
## 🙏 Acknowledgements
|
| 128 |
+
|
| 129 |
+
This work was supported by the **European Space Agency (ESA)** within the **FDR4VGT: Fundamental Data Record for VGT** project, and by the Spanish Ministry of Science, Innovation and Universities (grant PID2023-148485OB-C21 funded by MCIU/AEI/10.13039/501100011033 ERDF, EU).
|
| 130 |
+
|
| 131 |
+
Developed at the Image Processing Laboratory (IPL), University of Valencia.
|
| 132 |
+
|
| 133 |
+
## 📜 License
|
| 134 |
+
|
| 135 |
+
[CC0-1.0](https://creativecommons.org/publicdomain/zero/1.0/)
|