xBD post-event building damage assessment
Semantic segmenters fine-tuned on real xBD post-event images. Each checkpoint is the best validation run of a baseline trained with horizontal and vertical flips and a cross-entropy plus Lovász loss.
Five classes:
| Index | Class |
|---|---|
| 0 | background |
| 1 | no-damage |
| 2 | minor |
| 3 | major |
| 4 | destroy |
Un-classified pixels are ignored in training. Scores below are validation micro IoU of the saved checkpoint.
| Checkpoint | Backbone | Val micro IoU |
|---|---|---|
segformer-b0/xbd-baseline/best |
MiT-B0 | 40.6 |
segformer-b1/xbd-baseline/best |
MiT-B1 | 39.8 |
segformer-b2/xbd-baseline/best |
MiT-B2 | 41.8 |
segformer-b3/xbd-baseline/best |
MiT-B3 | 47.0 |
segformer-b4/xbd-baseline/best |
MiT-B4 | 47.7 |
segformer-b5/xbd-baseline/best |
MiT-B5 | 43.3 |
upernet-convnextv2-nano/xbd-baseline/best |
ConvNeXt V2 Nano | 41.7 |
upernet-convnextv2-tiny/xbd-baseline/best |
ConvNeXt V2 Tiny | 44.8 |
upernet-convnextv2-base/xbd-baseline/best |
ConvNeXt V2 Base | 45.8 |
Usage
Inputs are RGB tiles in [0, 1], then ImageNet normalization (mean = (0.485, 0.456, 0.406), std = (0.229, 0.224, 0.225)). Training and evaluation use 1024×1024 tiles.
import torch
import torch.nn.functional as F
from PIL import Image
from torchvision.transforms.functional import to_tensor
from transformers import SegformerForSemanticSegmentation
repo = "BiliSakura/xBD-post-bda-models-transformers"
model = SegformerForSemanticSegmentation.from_pretrained(
repo, subfolder="segformer-b4/xbd-baseline/best"
).eval()
mean = torch.tensor([0.485, 0.456, 0.406]).view(1, 3, 1, 1)
std = torch.tensor([0.229, 0.224, 0.225]).view(1, 3, 1, 1)
pixel_values = to_tensor(Image.open("post.png").convert("RGB")).unsqueeze(0)
pixel_values = (pixel_values - mean) / std
with torch.no_grad():
logits = model(pixel_values=pixel_values).logits
logits = F.interpolate(logits, size=pixel_values.shape[-2:], mode="bilinear", align_corners=False)
pred = logits.argmax(dim=1)[0]
UperNet checkpoints load the same way with UperNetForSemanticSegmentation and subfolder="upernet-convnextv2-base/xbd-baseline/best".
License
CC BY-NC-SA 4.0, the license of the xBD dataset these weights were trained on.