ESA PROBA-1 CHRIS Reprocessing: Cloud and Cloud-Shadow Detection Module
Labels clouds and cloud shadows in CHRIS/PROBA-1 scenes as clear, cloud or
shadow, plus 255 for nodata. One ensemble covers all six acquisition modes.
CHRIS records tens of narrow bands, in a different order in every mode, so the loader first builds the four inputs the model was trained on (red, green, blue and NIR) from the bands of that mode, and then segments the whole scene.
pip install mlstac pytorch-lightning segmentation-models-pytorch timm rasterio
import mlstac
model = mlstac.load("https://huggingface.co/isp-uv-es/CHRIS-PROBA1/resolve/main/mlm.json")
local = model.download("CHRIS-PROBA1")
net = local.compiled_model(device="cuda")
mask = local.module.predict_chris("scene_TOA.tif", model=net, mode_n=3, source="toa")
# 0 = clear, 1 = cloud, 2 = shadow, 255 = nodata
Accuracy
On 55 hand-painted CHRIS scenes, split by flight and never seen in training, each
one going through the full inference path in load.py:
| Metric | Value | Aggregation |
|---|---|---|
| mIoU | 0.708 | mean per scene |
| IoU clear / cloud / shadow | 0.898 / 0.831 / 0.652 | pixels pooled |
| Macro F1 | 0.881 | pixels pooled |
| Cohen's kappa | 0.854 | pixels pooled |
| Accuracy | 0.921 | pixels pooled |
The first row averages scenes, the rest pool every valid pixel. By mode: 0.784 (12 scenes), 0.697 (5), 0.699 (12), 0.576 (5), 0.658 (3) and 0.711 (18) for modes 1 to 5 and 20. Modes 2, 4 and 5 have five scenes or fewer, so read those as indicative.
The four inputs, mode by mode
Each panel is one mode. Each lane is one input: the wide curve is the Sentinel-2 response and the blocks are the CHRIS bands of that mode that build it, darker when they weigh more. Mode 5 carries a fifth lane because a model tuned for that mode adds B5; in the other modes a per-mode model keeps the same four, and no extra band pays off.
Mode 3, for instance: B4 from bands 7 and 8, B3 from 4 and 5, B2 from 2 and 3, NIR
from 14, 15 and 16. The weights come from comparing each CHRIS response with the
Sentinel-2 one, and they are listed per mode in bands.json. You do not need to
touch this: predict_chris reads the mode and builds the four channels for you.
How it works
Three U-Nets, averaged:
| Checkpoint | Encoder |
|---|---|
checkpoints/convnext_a.ckpt |
ConvNeXtV2-nano |
checkpoints/convnext_b.ckpt |
ConvNeXtV2-nano |
checkpoints/regnety.ckpt |
RegNetY-004 |
Inference walks the scene in 640 px windows with 320 px overlap, snaps the last window to the edge so nothing is padded, normalises each window per band with a z-score, averages the softmax of the three checkpoints and blends the windows with a gradient mask. Around 0.3 s per 748 x 744 scene on an RTX A5500, 4 s on CPU.
Inputs
| File | source |
Notes |
|---|---|---|
| TOA product | "toa" |
reflectance x10000, the usual case |
| Radiance | "rad" |
what mode 20 is usually delivered as |
| Raw DN | "dn" |
accepted, less accurate |
Modes 1 to 5 and mode 20 are supported, passed as mode_n=1 โฆ mode_n=20. Mode 1
is binned to 372 x 374, so its window shrinks to 352 px; the rest are 744 px wide,
with 748 rows in most acquisitions and up to 1024 in some mode 2 products.
With your own four-band stack:
mask = local.module.predict_large(rgbn, model=net) # rgbn: (4, H, W) float
Per-pixel confidence:
probs = local.module.predict_chris("scene_TOA.tif", model=net, mode_n=3,
source="toa", return_probs=True)
Data
- Pretraining: CloudSEN12, 50,247 patches resampled to 640 x 640 at 10 m with 13 spectral bands, labelled clear / cloud / shadow. Eleven band combinations were trained with ConvNeXtV2-nano and RegNetY-004; those with four bands or more land within 0.008 mIoU of each other, so the four-band one was kept.
- Fine-tuning: 130 hand-painted CHRIS scenes, each sampled 15 times with random crops, plus 1,611 automatic masks over scenes nobody painted. A second round retrained on the model's own predictions, keeping a pixel only where the prediction was confident and stable.
- Test: 55 painted scenes, split by flight, never by scene, so two angles of the same acquisition never sit on both sides.
What it is not
- Shadow is the weak class (IoU 0.652). CHRIS covers 406 to 1005 nm: no SWIR, no thermal band, so a shadow and a dark surface look alike. Expect misses over water and dense vegetation.
- No thin-cloud class. Thick and thin are merged into
cloud. - Differences under 0.03 mIoU are noise with 55 test scenes.
- Mode 2 after 2015 is unusable: those products store a gain setting of -1, so the calibrated imagery is not calibrated and the digital numbers are meaningless.
- The three checkpoint files carry the optimizer state, which is why they weigh more than the networks.
Files
load.py mlstac loader
mlm.json model card, STAC/mlm format
bands.json CHRIS to red/green/blue/NIR table per acquisition mode
checkpoints/ the three networks
examples/mode_<n>/ one scene per mode, with its painted mask and the prediction
figures/ metrics, example, band mapping, gallery
notebook.ipynb Colab walkthrough
metrics.json per-class and per-mode numbers
MIT licence.



